Gufo is an open-source inference engine for one machine: AMD Strix Halo, the Ryzen AI MAX+ 395 with its Radeon 8060S (gfx1151) and up to 128 GiB of unified memory. It serves text, embedding, speech, image and video models over OpenAI-compatible APIs, from a single binary with kernels tuned for that chip. It runs on Linux, where releases are built and tested, and natively on Windows.

Gufo supports one model per use case, the best that fits, and checks each one against the original before optimizing it. Its cache is built for coding agents that resend long conversations, and extensions let you bring your own model.

Each model page lists the files to download, the modes and the measured speed. Gufo's own code is MIT; the weights keep their publishers' terms.

Extensions add more models without changing Gufo. With gufo set up as in the quickstart, install a reviewed one from the extension registry:

gufo extension install lfm3

Extensions explains what that runs and how to trust it; the catalogue lists the official and community extensions.

Download Qwen3.8 Flash-Next and its MTP head:

hf download unsloth/Qwen3.8-Flash-Next-GGUF \
  --revision d96201194f103c906994f283f504209dc8d2a10c \
  --include "UD-Q4_K_XL/*" \
  --include "MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf" \
  --local-dir models/flash-next

Define gufo to run in the container, with the GPU and the models folder:

gufo() {
  podman run --rm -it --network host \
    --userns=keep-id:uid=1000,gid=1000 --group-add keep-groups \
    --device /dev/kfd --device /dev/dri --ulimit memlock=-1 \
    -v ./models:/models:ro \
    -v gufo-extensions:/home/gufo/.local/share/gufo/extensions \
    ghcr.io/gufo-org/toolboxes/gufo-runtime:latest gufo "$@"
}

Serve the model, then ask it something from another terminal:

gufo serve llm --served-model-name flash-next \
  --model /models/flash-next/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
  --speculative mtp --mtp-model /models/flash-next/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "flash-next", "messages": [{"role": "user", "content": "Say something"}]}'

Any OpenAI client works with the base URL http://localhost:8080/v1: Open WebUI, the OpenAI SDKs, Codex and other agents. If the GPU isn't found, see troubleshooting. For the Debian package, Nix, a source build or Windows, see install.

The documentation is on gufo.blog/docs, one version per release, and in docs/ for the development version.

• Get started: install, models, command line, server API.
• Run it well: cache, troubleshooting, benchmarks, versions.
• Extend it: extensions, writing an extension, extension API.

The changelog and releases list what changed.

• One machine. Every kernel is tuned for Strix Halo with 128 GiB. Smaller memory configurations work for the models that fit.
• The best model per use case. Fewer models means less code to maintain and more time to optimize and test each one. Each model keeps its own kernels, so a change to one can't break another.
• Quality before speed. Every model has a quality report that compares it with an independent reference; an optimization that changes the output isn't one.
• Real workloads. Concurrent requests, cancellation and long conversations are first-class, not afterthoughts.
• Small production footprint. Few runtime dependencies; development and reference tools stay out of the package.

Contributions are welcome: Gufo needs the Strix Halo community to keep improving. Start with development and testing; AGENTS.md holds the rules that people and coding agents follow. To add a model without changing Gufo, write an extension.

Community projects built on Gufo include Gufo RDMA for Dual Strix Halo, which runs models across two machines and isn't merged yet.

Gufo's original code is MIT licensed. Its design builds on other open-source engines, listed in reference projects. Adapted code and dependencies keep their own notices in NOTICE, THIRD_PARTY_NOTICES.md and licenses/, installed under share/licenses/gufo. Model weights aren't bundled and keep their publishers' terms.