vLLM Deployment on a GPU Workstation

vLLM Deployment on a GPU Workstation An architecture diagram generated by Archify. Developer / Client App · C# client · Architecture component · OpenAI-compatible API Developer / Client App C# client OpenAI-compatible API vLLM OpenAI Server · vllm/vllm-openai:latest · Workstation: NVIDIA GPU, Docker + CUDA · API :8000 vLLM OpenAI Server vllm/vllm-openai:latest API :8000 Model Cache · Host volume ./models · Workstation: NVIDIA GPU, Docker + CUDA · Cached models Model Cache Host volume ./models Cached models Hugging Face Model Hub · Public model repository · Architecture component Hugging Face Model Hub Public model repository /v1/* · HTTP :8000 · Bearer Read/Write weights Download models · HTTPS 443 Workstation: NVIDIA GPU, Docker + CUDA Legend Backend Database External

Workstation

  • • NVIDIA GPU with Docker + CUDA runtime
  • • vLLM serves the OpenAI-compatible API on port 8000

Model storage

  • • Weights are read/written in host volume ./models
  • • Missing models download from Hugging Face over HTTPS 443

Client access

  • • C# client calls /v1/* endpoints
  • • Bearer token authentication on requests