Run open models as fast as your hardware allows

Magnitude is an open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. It's over 2x faster than Ollama and works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU. One click connects it to the agent you already use (Pi, OpenCode, Hermes, Codex, and more).

⭐ Help us reach more developers and grow the Magnitude community. Star this repo!

Magnitude comes with a desktop app and a CLI. Install from your terminal:

macOS / Linux:

curl -fsSL https://magnitude.dev/install.sh | sh

Windows PowerShell:

irm https://magnitude.dev/install.ps1 | iex

Or download the installer:

• macOS: Apple silicon · Intel
• Windows: x64
• Linux x64: .deb · .rpm · .pkg.tar.zst
• Linux ARM64: .deb · .rpm

To run Magnitude as a service other computers can use, see Remote server. To run it in a container, see Docker.

• 2x faster than Ollama: on average across popular local models, on Mac and NVIDIA
• Tuned on your hardware: kernels are tuned for each model when you download it
• Best drafter built in: each model ships with the fastest speculative decoding drafter
• Memory that flexes: quantized KV halves memory per agent, freed when agents stop
• Fast concurrent sessions: sessions share prefix caches to prevent slowdown
• Works with your agent: one click to connect Pi, OpenCode, Hermes, Codex, and more
• Free, private, open source: no token costs, nothing leaves your machine, Apache 2.0

View full benchmarks →

An open source inference engine that optimizes each model for your exact hardware. It ships as a desktop app that runs open models and connects them to the agent you already use.

They ship one-size-fits-all kernels, because tuning each model for each chip by hand would take forever. Magnitude tunes them automatically. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math (how many GPU threads team up, how big a chunk each one takes, how far ahead to load data) and keeps the fastest combination that doesn't change the model's output. See the full benchmarks against Ollama.

Any Apple Silicon, NVIDIA, AMD, or Intel GPU, or nothing but a CPU. There is no fixed minimum. Smaller machines run smaller models, and more memory lets you run larger ones.

macOS, Linux, and Windows.

See the full list at magnitude.dev/models. We write optimized kernels for the most popular open-weight families, which is how we beat generalist engines.

One click connects Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Anything else works through the OpenAI-compatible API.

Yes. Prompts, files, and models stay on your machine. No internet needed once a model is downloaded.