Pinned vLLM XPU, llama.cpp SYCL and EXL3-XPU recipes, failure notes, and measurement records for Intel Arc Pro B60/B70. Every published benchmark row carries its exact workload, a commit-pinned evidence link, and — for routes that decode with a drafter — whether the drafter changes the model's greedy answer. Start with what worked, what did not, and what to test next. For a first local run, use START.md.
| Your goal | Start here | Status |
|---|---|---|
| Choose a model for a job | Model comparison | 94-task quality screen of Bolt 74B-A9B (two cards), Qwen3.8-27B and Qwen3.6-35B-A3B (one card each) with power and heat; one sample per task, Qwen3.6 partial. |
| Serve on one B70 | Qwen3.8-27B EXL3 route | Primary route: trellis 4bpw + MTP3 + fp8 KV on the exl3xpu engine; the only measured route at full 262,144 context and clean on the exact-answer battery. MTP3 changes the greedy answer on 27 % of a fixed 329-item set — serve --set vllm.speculative_config=null when one-shot code must match, and see spec parity on every speculative row. The GPTQ/W4A16 route is legacy — that class fails the canary on this model. |
| Fastest lossless 27B at short context | Qwen3.8-27B TensorFold route | Screen 2026-11-08: TensorFold + DFlash2 at 128 tok/s with output identical to plain greedy; EXL3+MTP3 wins from ~16K up. Includes a +48 % long-context patch. |
| Long prompts on llama.cpp | KV type at depth | Screen 2026-11-08: on llama.cpp after 2026-10-29, q8_0 or q4_0 KV decodes 1.8–2.1× faster at 64K than before; q5_0/q4_1 and f16 do not gain. |
| Run GGUF or vision | Muse-Glimmer llama.cpp route | Measured on its own engine and workload; no cross-engine speed ranking without matched timing. |
| Run MoE or long context | Qwen3.6-35B-A3B recipe | Historical MTP speed record is stack-specific; a later matched comparison did not reproduce it. |
| Serve on two B70s | Independent-card setup | Default: one server per card. FP8 TP2 is a research route, not an automatic single-request win. |
| Diagnose or reproduce a number | Image and patch matrix → family recipe → benchmark catalog | Match image, patch, model, context, cache, power, and timing definition. |
Read the decision guide before tuning. It separates measured improvements, failed attempts, unpublished lab screens, and stop/go gates. A passing smoke canary does not establish model quality. No public performance claim from a different model, engine, host, or result generation transfers to this setup.
- Check your card and host with
scripts/doctor.py, then use the quick start for a pinned single-card run. - Check the image and ordered patch matrix for the exact family before launching. Do not mix family patch stacks.
- Connect a client using client setup; use the reliability guide if the server hangs.
- Confirm a recipe with the reproduction procedure and benchmark format. The numeric authority is
data/benchmarks.v1.json;docs/BENCHMARK-CATALOG.mdis generated.
More routes: Ripple-65B on two cards (new, -sm tensor 13.3 tok/s), Nemotron DFlash, Nemotron Bolt 74B (one card on llama.cpp, or two cards on vLLM), Flash-Next, Ornith-1.5-122B GGUF, Windows, and family-specific research. Research routes are not interchangeable with the first-run recipe.
Quantize a model yourself: how the Bolt 74B AutoRound W4A16 quant was made and checked on two B70s, with every command, fix and script.
Need a model or configuration not yet covered? Submit a recipe request with your target Hugging Face model and hardware setup.
To contribute a recipe directly to the B70 Launcher catalog, follow the recipe format specification to edit recipes.json, verify with python3 -m unittest tests.test_recipes, and open a pull request.
Use data/system-map.v1.json to select an intent. docs/AGENT-SYSTEM.md explains the authority layers; docs/ADDING-A-RECIPE.md explains publication. Keep historical evidence intact. Validate changes with python3 scripts/check-repo.py.
Code: MIT. Measurement reports and prose: CC BY 4.0. See LICENSE.