vLLM Startup & Request Flow

vLLM Startup & Request Flow A sequence diagram generated by Archify. POST /v1/chat/completions (prompt) check cache miss download weights model files save to cache load model to GPU (PagedAttention init) Internal step: weights read from ./models into GPU memory stream tokens (SSE) Request Cache miss Load + stream Client · API consumer · Sequence participant Client API consumer vLLM Server · port 8000 · Sequence participant vLLM Server port 8000 Model Cache · ./models · Sequence participant Model Cache ./models Hugging Face Hub · model weights · Sequence participant · Hugging Face Hugging Face Hub model weights Legend request return async trace default message

Request Entry

  • • Client calls POST /v1/chat/completions on vLLM Server at port 8000
  • • The prompt triggers weight resolution before inference can start
  • • Response streams back as SSE tokens once the model is resident

Cold Start (Cache Miss)

  • • Model Cache at ./models is checked first and reports a miss
  • • Weights are downloaded from Hugging Face Hub on the miss path
  • • Downloaded files are saved back to ./models for future runs

Warm Restart (Cache Hit)

  • • Sequence IR is linear, so the cache-hit alternative is not drawn as an alt branch
  • • On a hit, cache returns the files and the download/save steps are skipped
  • • Startup goes straight from cache read to GPU load with PagedAttention init