The High-Throughput and Memory-Efficient inference and serving engine for LLMs
Easy, fast, and cost-efficient LLM serving for everyone.
Easy
Deploy the widest range of open-source models on any hardware. Includes a drop-in OpenAI-compatible API for instant integration.
Fast
Maximize throughput with PagedAttention. Advanced scheduling and continuous batching ensure peak GPU utilization.
Cost Efficient
Slash inference costs by maximizing hardware efficiency. We make high-performance LLMs affordable and accessible to everyone.
What's new
Latest from vLLM
Releases, day-0 model support and the latest from the vLLM blog.
- BlogServing DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72
With P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU at a P90 of 196 tokens/s per user, 7.9x our aggregated single-node result.
- BlogGLM-5.3 Optimizations, Part 2: PCP, DCP, and Hybrid HiSparse on 16 H200s
vLLM serves GLM-5.3 on 16 H200s with PCP and DCP prefill and Hybrid HiSparse decode, reaching 1M-token contexts and 1.7–3.2× the per-user speed of data-parallel P/D on the AgentX agentic benchmark.
- ReleasevLLM v0.32.0 is out: low-precision KV cache everywhere
814 commits from 363 contributors (121 new). NVFP4 KV cache on more GPUs, Model Runner V2 with Elastic EP and n-gram speculation, and faster startup.
Quick Start
Select your preferences and run the install command. Stable represents the most currently tested and supported version of vLLM. Nightly is available if you want the latest builds.
📦 Requires Python 3.10+. Python 3.12+ recommended.
⚡ We recommend uv for faster and more reliable installation.
🔧 For other platforms, see docs.vllm.ai
🎉 See what's new in
💡 Compatible with all CUDA 13.x versions (13.0 - 13.1) · Troubleshooting
Sponsors
vLLM is a community project. Our compute resources for development and testing are supported by the following organizations. Thank you for your support!
Cash Donations
Compute Resources
Slack Sponsor
We collect donation through GitHub and OpenCollective. We plan to use the fund to support the development, maintenance, and adoption of vLLM.
Universal Compatibility
One engine, endless possibilities. Run any model on any hardware.
Hardware
Unified API across platforms
Open Models
Latest trending open-source models, optimized & production-ready
Got questions?
We're here to help.
Whether you're just getting started or debugging a complex deployment, our community is open to everyone. No question is too basic!
Resources
Explore recipes, benchmarks, and roadmap