What changed in the latest vLLM releases, how to deploy it (Docker/K8s), tune throughput/latency, monitor with Prometheus, and when to pick vLLM vs TGI or TensorRT-LLM.
We wrote this to save your team time. It’s the guide we wished we’d had the last time we took vLLM to production: precise, copy-pasteable, and focused on the decisions that actually move your throughput, latency, and bill.
TL;DR (Why vLLM is still trending)
- Real speed, real features. Continuous batching + PagedAttention = high GPU utilization and stable tail latencies. New 0.10.x releases added multi-LoRA, FlashAttention 2 support across more GPUs, and a pile of API/server improvements.
- OpenAI-compatible server. Drop-in
/v1/chat/completions& friends, plus embeddings, rerank, pooling, and health/metrics endpoints. - Production knobs that matter. Max model length, tensor parallelism, GPU memory utilization, LoRA hot-swap, prefix/speculative decoding—documented and stable.
- Observability built-in. Native Prometheus
/metricsyou can scrape into Grafana.
What’s new in the latest vLLM (0.10.x)
Here are highlights relevant to folks running real traffic:
- v0.10.2 (recent): LoRA & multi-LoRA improvements, Flash-attn 2 updates, and bug fixes around scheduling/throughput. (See full GitHub release notes.)
- Ongoing 0.10.x: More models supported out-of-the-box, stability in OpenAI-compat endpoints, and performance tweaks in the engine. (Track the repo’s releases for day-by-day changes.)
Tip: If you’re upgrading from <0.9, scan the OpenAI server API docs—there are more handlers now (embeddings, rerank, pooling). Your client might “just work,” but your metrics and health checks should be updated.
Still deciding whether vLLM is the right layer? Local LLM servers compared puts it next to Ollama, LM Studio and llama.cpp on the variable that decides it: how many concurrent requests each one serves before it starts queueing.
Quickstart: the 5-minute path
1) Install or run the server
Option A: pip (single GPU):
pip install -U vllm
python -m vllm.entrypoints.openai.api_server \
--model meta-llama/Meta-Llama-3.1-8B-Instruct \
--host 0.0.0.0 --port 8000This launches an OpenAI-compatible REST server on :8000.
Option B: Docker (recommended in dev/staging):
docker run --gpus all --rm -p 8000:8000 \
-e HF_TOKEN=$HF_TOKEN \
vllm/vllm-openai:latest \
--model mistralai/Mistral-7B-Instruct-v0.3 \
--host 0.0.0.0 --port 8000Use vllm-openai images for the API server entrypoint.
Auth: Enable a simple bearer key with --api-key YOUR_KEY and send Authorization: Bearer YOUR_KEY from clients. vLLM’s server includes an Authentication middleware that checks this header.
2) Call it with the OpenAI SDK
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="YOUR_KEY")
resp = client.chat.completions.create(
model="mistralai/Mistral-7B-Instruct-v0.3",
messages=[{"role":"user","content":"Explain paged attention in 2 sentences."}],
temperature=0.2,
)
print(resp.choices[0].message.content)The server implements the OpenAI routes so standard clients work with a custom base_url.
“It has to be fast”: the 8 knobs that actually matter
Below are the flags and settings we consistently see move the needle in production:
- Batching & Queueing
- vLLM does continuous batching—you don’t code anything; you tune it. Start by letting the server fully batch under load; avoid overly strict per-request max token limits that block batching efficiency. (Profile with
/metrics.)
- Context length vs. throughput
- Longer
max_model_lenincreases KV cache size → lower throughput. Size honestly for your workload; don’t set 128k if you mostly use 8–16k. (Use the server’s model args to cap.)
- GPU memory utilization
- Use
--gpu-memory-utilization(or Docker--gpus all) to push VRAM usage. Watch OOMs: beyond ~0.92 you’ll see spikier tail latencies. (Validate with Prometheus.)
- Tensor parallelism
- Multi-GPU servers: set
--tensor-parallel-sizeequal to the number of GPUs for big models. Keep nodes homogeneous.
- LoRA & multi-tenant adapters
- Serve many fine-tunes with multi-LoRA; load/unload without restarting. Great for “per-team system prompts” and A/Bs.
- Prefix caching
- Enable prefix/prompt caching for workloads with shared instruction prefixes; it eliminates repeated prefill cost. (Feature: “prefix caching.”)
- Speculative decoding
- If you have a small draft model that tracks your target reasonably well, turn on speculative decoding for a free tokens/s boost. Test carefully—mismatch hurts acceptance rate.
- Quantization
- AWQ/GPTQ are supported; they trade a little accuracy for a lot of throughput and memory headroom—often worth it for chat/apps. Validate per model.
Observability & SRE basics
Prometheus & Grafana
The API server exposes a /metrics endpoint. Scrape it and build alerts for:
- p50/p95 gen time, tokens/sec, batch size, queue depth
- OOM rate, scheduler wait time, request failure rate
Docs and sample metrics are in the production section.
Health/Info endpoints
Use /health and /show_server_info for K8s readiness/liveness and quick debugging. (These routes are part of the server module.)
Reference deployments (copy-paste)
Docker Compose (single node)
services:
vllm:
image: vllm/vllm-openai:latest
ports: ["8000:8000"]
deploy:
resources:
reservations:
devices: [{ capabilities: ["gpu"] }]
environment:
- HF_TOKEN=${HF_TOKEN}
command: >
--model mistralai/Mixtral-8x7B-Instruct-v0.1
--host 0.0.0.0 --port 8000
--api-key ${VLLM_API_KEY}Server flags correspond to the documented OpenAI-compatible entrypoint.
Kubernetes (sketch)
- Use a Deployment with
nvidia.com/gpu: 1..N, a ReadinessProbe on/health, and a Service with session-affinity if you attach large LoRA sets per pod. - Scrape
/metricsvia a PodMonitor. (The endpoint is standard Prometheus.)
Practical playbooks (real workloads)
1) Multi-team chat (tenants & adapters)
- Shape: Many tiny prompts, short outputs, moderate concurrency.
- Settings: High batching; prefix caching; multi-LoRA; 8–16k context.
- Why vLLM: Best bang-for-buck throughput without major client changes; adapters hot-swap per team.
2) Long-form generation (RAG/summarization)
- Shape: Few long prompts, long outputs.
- Settings: Bigger
max_model_len, lower concurrency, quantize for memory headroom, watch p95. - Why vLLM: Stable long-context scheduling with PagedAttention.
3) High-QPS API
- Shape: Many short calls under strict p95, shared system prompt.
- Settings: Tight p95 SLOs, aggressive prefix caching, speculative decoding, autoscale on queue length.
vLLM vs. TGI vs. TensorRT-LLM (when to choose what)
| You need… | Pick vLLM | Pick TGI | Pick TensorRT-LLM (+ Triton) |
|---|---|---|---|
| OpenAI-compatible server w/ fast time-to-value | ✅ | ✅ (HF client-first; OpenAI shim exists) | ❌ |
| SOTA batching + paged KV cache out-of-the-box | ✅ | ✅ (continuous batching) | ⚠️ (you assemble pipelines) |
| Quantization & model zoo breadth | ✅ | ✅ | ✅ (best if you rebuild engines) |
| Max perf on NVIDIA with deep custom tuning | ✅ | ⚠️ | ✅✅ |
| Lowest ops burden (single container) | ✅ | ✅ | ❌ (engines + Triton) |
- TGI (Text Generation Inference) is excellent and battle-tested, especially if your stack is already HF-centric. If you want OpenAI SDK drop-in plus LoRA/prefix caching controls, vLLM is slightly simpler.
- TensorRT-LLM wins for squeezing every last token/sec on NVIDIA when you can invest in building engines and Triton pipelines; it’s more work.
Advanced features you’ll actually use
- LoRA hot-swap / multi-LoRA: Serve many adapters on one base model; route by header or org.
- Prefix caching: Big win for apps sharing long instructions/system prompts.
- Speculative decoding: Draft model proposes tokens; target model verifies; higher tokens/sec. Validate acceptance rate per pair.
- Quantized models (AWQ/GPTQ): Less VRAM, more throughput—check quality on your evals.
- Tensor parallelism: Clean horizontal scale within a node; keep GPU types identical.
Troubleshooting (field notes)
- “It’s slow in prod but fast locally.” You’re under-batching. Load test with realistic concurrency; watch average batch size and queue wait in Prometheus.
- “OOMs after upgrade.” New defaults can change memory footprints (e.g., flash kernels). Drop
max_model_lenor lower--gpu-memory-utilization. - “OpenAI client 429s.” Check vLLM queue limits and your orchestrator retry policy; enable backpressure rather than letting clients hot-loop.
- “Which quant?” Start with AWQ on Llama/Mistral; compare GPTQ per model. Accuracy can vary—don’t skip evals.
The model server is one layer of five. Self-hosted AI agents: the production stack covers the runtime, the tool boundary, memory and audit around vLLM, with costs for three deployment sizes.
Security & compliance quick hits
- Auth: Use
--api-key(or your own reverse-proxy auth). The server checksAuthorization: Bearer …. - Secrets: Avoid baking tokens into images; mount as env/secret.
- Egress: Pin outbound to model repos only; mirror artifacts if needed.
Copy-paste cookbook
OpenAI SDK + embeddings:
emb = client.embeddings.create(
model="nomic-ai/nomic-embed-text-v1.5",
input=["alpha", "beta"]
)
print(len(emb.data[0].embedding))vLLM implements the embeddings route via the same server.
Metrics:
curl -s http://localhost:8000/metrics | grep vllmYou’ll see counters/gauges for tokens, queue, batch sizes, errors.
Key takeaways
- vLLM remains the easiest path to high-throughput, OpenAI-compatible serving with first-class batching and KV management.
- The 0.10.x line is worth upgrading for performance and server API improvements—just re-check your metrics & health checks after.
- For most App/Platform teams, vLLM vs. TGI is a productivity tie—pick the ecosystem you prefer. If you need ultimate NVIDIA perf and can pay the complexity tax, TensorRT-LLM wins.
— Cohorte Engine Room
November 17, 2025.

