vLLM
High-throughput LLM serving engine with paged attention and an OpenAI-compatible server.
by vLLM Project
No reviews yet
Open source
Free
Linux
API only
proto-v0.3.0
Updated 49 days ago
92
81.7
=
Visit website
Not verified by the vendor
Source code
Compare with Ollama
Is this your product? Claim it
☆ Favorite
👍 Recommend
🔔 Follow releases
Overview
High-throughput LLM serving engine with paged attention and an OpenAI-compatible server.
- Category
- LLM runtimes · AI tools
- Platforms
- Linux, API only
- Pricing
- Free
- License
- Apache-2.0
- Source code
- vllm-project/vllm
- Repository stars
- 92,120
- API
- No public API description
Release history
Updated 49 days ago4 releases in the last 12 months.
-
proto-v0.3.0
-
proto-v0.2.0
-
proto-v0.1.0
-
v0.29.0
What users say
No reviews yet
Ratings appear once verified reviews arrive. Until then the rank uses public signals only: release freshness and repository adoption.
Alternatives to vLLM
See all alternatives Ollama
4
94.9
=
Runtime to download and run large language models locally, with a CLI and a REST API.
Updated 49 days ago
llama.cpp
6
94.4
=
C/C++ inference engine for LLMs on CPUs and GPUs, using the GGUF format and quantization.
Updated 48 days ago
LocalAI
61
86.6
=
Self-hosted, OpenAI-compatible API that runs text, image and audio models on local hardware.
Updated 49 days ago
LM Studio
367
40.0
=
Discover, download and run local LLMs offline on Mac, Windows and Linux with an intuitive desktop interface and local OpenAI-like server.
Stale: 633 days without a release