llama.cpp
C/C++ inference engine for LLMs on CPUs and GPUs, using the GGUF format and quantization.
by ggml.ai
No reviews yet
Open source
Free
Linux
macOS
Windows
API only
b11043
Updated 48 days ago
6
94.4
=
Visit website
Not verified by the vendor
Source code
Compare with Ollama
Is this your product? Claim it
☆ Favorite
👍 Recommend
🔔 Follow releases
Overview
C/C++ inference engine for LLMs on CPUs and GPUs, using the GGUF format and quantization.
- Category
- LLM runtimes · AI tools
- Platforms
- Linux, macOS, Windows, API only
- Pricing
- Free
- License
- MIT
- Source code
- ggml-org/llama.cpp
- Repository stars
- 128,722
- API
- No public API description
Release history
Updated 48 days ago10 releases in the last 12 months.
-
b11043
-
b11042
-
b11040
-
b11039
-
b11037
-
b11036
-
b11035
-
b11034
-
b11033
-
b11030
What users say
No reviews yet
Ratings appear once verified reviews arrive. Until then the rank uses public signals only: release freshness and repository adoption.
Alternatives to llama.cpp
See all alternatives Ollama
4
94.9
=
Runtime to download and run large language models locally, with a CLI and a REST API.
Updated 49 days ago
LocalAI
61
86.6
=
Self-hosted, OpenAI-compatible API that runs text, image and audio models on local hardware.
Updated 49 days ago
vLLM
92
81.7
=
High-throughput LLM serving engine with paged attention and an OpenAI-compatible server.
Updated 49 days ago
LM Studio
367
40.0
=
Discover, download and run local LLMs offline on Mac, Windows and Linux with an intuitive desktop interface and local OpenAI-like server.
Stale: 633 days without a release