Runs models up to 284B on one 128GB mini PC, measured on a Beelink GTR9 Pro, with results from 13 systems and sources. Raw logs and caveats.

Independent guide, not endorsed by AMD, any OEM or the projects named above.

Run a local LLM on the GPU of a 128GB AMD Strix Halo / Ryzen AI MAX+ 395 (Radeon 8060S) with Ubuntu 24.04, Ollama and Vulkan/RADV.

1. BIOS: set the UMA frame buffer to 512MB if offered, otherwise the lowest value your vendor allows (often 2GB). Leave IOMMU enabled/default.
2. Ubuntu: install Ubuntu 24.04 LTS directly on the hardware.
3. Install: run the setup script below. It adds the 128GB boot parameters amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0, upgrades Mesa/RADV and configures Ollama for Vulkan with OLLAMA_VULKAN=1, OLLAMA_IGPU_ENABLE=1 and HIP_VISIBLE_DEVICES=-1.
4. Reboot when the script prints REBOOT REQUIRED.
5. Check: the boot check and the known-answer check must both end in PASS.
6. Chat: run ollama run qwen3.6:35b-a3b, then confirm that ollama ps shows 100% GPU.

Another memory size (64/96/192GB)? The 128GB values do not carry over: use the manual route and its memory section. On Windows: the Windows route.

Step 3: install. Run this as your normal user, not as root. It stops at the first command that fails:

sudo apt update &&
sudo apt install -y git curl python3 zstd software-properties-common &&
cd "$HOME" &&
git clone https://github.com/hogeheer499-commits/strix-halo-guide &&
cd "$HOME/strix-halo-guide" &&
bash setup.sh

Steps 4 to 6: reboot, check, chat. Reboot if the script printed REBOOT REQUIRED, then run:

cd "$HOME/strix-halo-guide" &&
python3 scripts/check_local_ai.py boot &&
python3 scripts/check_local_ai.py answer &&
OLLAMA_HOST=http://127.0.0.1:11434 ollama run qwen3.6:35b-a3b

Good: both checks end in PASS and the model answers your question. Type /bye to leave the chat, then check the GPU:

OLLAMA_HOST=http://127.0.0.1:11434 ollama ps

It should show 100% GPU. An empty list means the model has already unloaded: run the chat once more and check again right away. CPU or a CPU/GPU mix means this check has not passed: see CPU fallback.

Keep Ollama local. It has no password. The script binds it to 127.0.0.1 and the answer check confirms that.

Working? Choose a model for your RAM and task or add browser chat. Stuck? Stop at the first error and use If a step fails; Get local AI running, step by step shows what every step should print.

Existing installations, version pinning, the curl | bash route and how to undo the changes: Setup Script Details.

• A clear setup route. The settings that matter on this chip, from the BIOS memory option to the one Ollama setting without which measured builds quietly run on the CPU, in one step-by-step start route that shows the expected result of every step.
• A model for your RAM and task. What can I run with my RAM? gives a starting point from 32GB to 192GB and shows which route was tested here and which are candidates. Best Known Profiles has the measured settings and speeds.
• Know what's tested. See which settings have been tested, what is still experimental, and which known failures you can avoid. If something goes wrong, use the troubleshooting steps. Every headline number links to its raw evidence or carries an explicit caveat.

Also: website · Qwen3.8 · reproduce a result · vendor/reviewer brief.

• You own, ordered, review, sell, or are evaluating a Strix Halo / Ryzen AI MAX system for local AI.
• You want a working route and the evidence needed to understand where it transfers—or fails to transfer—to another system.
• You need independent buyer guidance that can coexist with future disclosed vendor or affiliate relationships.

A complete, practical guide to running large language models locally on AMD Strix Halo / Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151) and 96GB/128GB unified memory.

Covers BIOS config, Ubuntu 24.04/kernel setup, Ollama, llama.cpp Vulkan/RADV, ROCm/HIP experiments, vLLM notes, 70B/120B and selected 284B GGUF capacity evidence, benchmarks, raw logs, and reproducibility checks.

AMD now publicly frames Ryzen AI Halo-class systems as a local-AI and developer-platform direction. This repository is the independent practical layer: copyable setup, measured rows, raw evidence, failures, and community reproductions. It is not official AMD or OEM endorsement. See RYZEN_AI_HALO_CONTEXT.md.

Project website: https://strixhaloguide.com/. This GitHub repository remains the source of truth for setup commands, benchmark claims, and raw evidence.

What you get:

• A copyable Ubuntu + Vulkan/RADV setup for Ollama and llama.cpp.
• Workload-to-model/backend recommendations instead of one undifferentiated leaderboard.
• Direct, API/server, MTP/speculative, concurrency, capacity, power, NPU, RPC, first-party, and community claims kept separate.
• Headline rows linked to CSVs, raw logs, charts or explicit n/a, plus preserved failures and corrections.
• Cross-OEM evidence covering 13 systems or independent sources from 10 credited community benchmark contributors.

Measured primarily on one Beelink GTR9 Pro. Community results are kept separate from local headline claims. This repository ships docs, scripts, data, and charts only; no .exe, binary .zip, browser extensions, or model weights. Raw evidence, commands, caveats, and corrections are linked so results can be checked instead of taken on trust.

Quick Start | Qwen3.8 | Profiles | Current Models | Evidence | Community | Reproduce | Contribute

If this guide saves you setup or research time, a GitHub star helps other Strix Halo owners find it; a reproducible result or correction helps even more.

Maintenance update: November 8, 2026. Model choices by RAM and task, corrected setup evidence, selected release and issue states, 192GB order phases and service details: MAINTENANCE_REVIEW_2026-11-08.md. The November 9 broad evidence review follows this scoped update; historical measurements keep their original dates.

Current evidence state, November 9, 2026 review: Qwen3.8 27B is measured through the official Ollama route, labelled as Ollama-default MTP drafting; a matched control on Ollama 0.32.15 (2026-09-26) measured 12.89 t/s without drafting versus 22.71 t/s with default MTP. See the active-evidence review; the September 26 review keeps its original wording. The August 30 b10687 Vulkan sentinel and Flash-Next scout are indexed with their own stack and caveats; see the measured update. The November 9 availability check records Ollama 0.40.15, llama.cpp v0.9.0 and numbered prerelease b12211 as available, not measured or qualified here. The security recheck adds published reports affecting older pins, including Ollama's pull API and llama.cpp parser/GGUF/RPC paths. Storefront blocks and advisory-feed limits are explicit in the review. The machine-readable freshness record is data/public_state.json.

Older changes are in the Changelog.

This is the quick "what can I actually run on my AI PC?" view. It is not the full benchmark list; see What You Can Run for more models and Headline Evidence for the audit trail.

These are the practical decisions extracted from the primary Beelink runs plus community reports from Corsair, GMKtec, MS-S1-Max, Nimo, a second Beelink owner stack, and a ciru-ai GMKtec EVO-X2 NixOS/NPU artifact. Use them to avoid retesting dead ends first; follow the evidence links if your setup differs.

Have a Strix Halo / Ryzen AI MAX system? Please share results, even if they are slower, failed, or contradict this guide.

• Open a benchmark report with your system, BIOS UMA setting, kernel, Mesa/ROCm versions, backend, model, command, CSV/raw output, and notes.
• Open a power / efficiency report if you can measure wall power, UPS power, or validated board power during the same benchmark command.
• Open a model request if there is a model/backend combination that should be tested.
• Use Discussions for setup questions, comparisons, and early results that are not ready for a benchmark issue yet.
• Power telemetry is useful if you have reliable wall-power data; it helps turn raw tokens/sec into tokens-per-watt context for buying, cooling, and always-on server decisions.
• Current community reports live in COMMUNITY_RESULTS.md, COMMUNITY_NIMO.md, COMMUNITY_RPC.md, USB4_CLUSTER_TUNING.md, and the community_* CSVs listed in data/README.md.
• See SHARE.md for short Reddit/HN/forum text and the current social preview image if you want to share the guide.

This guide is primarily a technical resource for AMD Strix Halo local-AI users. For vendors, reviewers, and partners, it also documents a reproducible way to reduce buyer setup friction and validate real-world local-AI use cases without weakening independent benchmark discipline.

Start with the public vendor and reviewer overview, then inspect the linked GitHub evidence, raw artifacts, negative results, disclosure policy, and upstream contribution record here.

Start with ONE_PAGE_BRIEF.md and PARTNERSHIP.md. Supporting docs cover SPONSORSHIP.md, VENDOR_DISCLOSURE.md, BUYER_USE_CASES.md, SPONSOR_ROADMAP.md, and TRACTION.md. How vendors can report a factual error, how vendor responses are recorded and how hardware provenance is disclosed: How We Work With Vendors.

No affiliate links are present as of September 19, 2026. The links added on October 1, 2026 were checked by URL pattern for tracking parameters and affiliate-network domains and none was found (the destinations themselves were not inspected). If monetized product links are added later, each relevant link will be labeled and recorded in the public affiliate link registry. Affiliate commission does not determine benchmark conclusions, product inclusion, or ranking.

These are the AMD Strix Halo / Ryzen AI MAX+ 395 configurations behind this guide's measured results. Pick the starting point that matches what you want to do; each result applies to the exact versions it was recorded with.

New to local AI? Follow the complete start route. It keeps installation, the first answer, the GPU check and help with errors on one page, and treats a browser interface as the next step for daily use. The measured profiles below were recorded on an existing setup; they are not proof that a fresh run of the revised installer works.

Choose the backend by what you are trying to do:

Once your first answer has passed its checks, choose a model and an interface for your daily work from the start route. To compare numbers, use the exact commands and evidence links in Reproduce One Headline Result, the AI/search setup summary, headline claim index, and raw evidence under data/raw/.

The machine-readable index for these rows is data/headline_claims.csv.

Dates below are measurement dates. A row being from May does not mean it is stale; it means later checks did not produce a stronger replacement headline. The latest controls and current-model rows are documented in CURRENT_MODELS.md, BENCHMARKS.md, PERFORMANCE_NOTES.md, and the raw evidence links below.

Pick the row that matches what you want to verify. The balanced coding row is the best first reproduction target for most users. The direct 100 t/s row is useful if you specifically want to verify the fastest measured 30B-class Qwen speed scout. Keep them separate: they use different model files, quants, builds, and quality tradeoffs.

AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-upstream-2026-05-07/build-vulkan/bin/llama-bench \
  -m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
  -fa 1 -ngl 999 -mmp 0 -p 0 -n 128 -r 20 -o csv

Measured local result: 96.76 tg128 in the max-performance b9049 campaign: raw CSV. This is the practical balanced Qwen3-Coder row. The fastest first-party Qwen3-Coder row is now 100.99 tg128 with Q4_K_S on the official b9851 Vulkan release binary, but that is a speed-first quant rather than the balanced default: raw b9851 r50. The older b9179 strict-clean 98.51 t/s row remains preserved for historical host-state context: raw b9179 r50.

Use this only if you have the exact Qwen3-30B-A3B-Instruct-2507 IQ4_XS-3.63bpw GGUF and a comparable b9467 Vulkan/RADV build. Adjust the binary and model paths to your checkout; the flags, model hash, build, and raw CSV are the reproducibility anchors. This is a direct llama-bench result, but it is not Qwen3-Coder and not the balanced default.

AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-b9467/build-vulkan/bin/llama-bench \
  -m ~/models/Qwen3-30B-A3B-Instruct-2507-IQ4_XS-3.63bpw.gguf \
  -fa 1 -ngl 999 -mmp 0 -b 2048 -ub 512 -t 16 --poll 50 -p 512 -n 128 -r 50 -o csv

Measured local result: 100.04 tg128 and 1416.03 pp512 on the r50 confirmation: raw r50, model hash, scout notes.

• A first-party Beelink wall-power tokens-per-watt claim. Local amdgpu PPT telemetry exists, and community Corsair wall-power rows exist, but the guide still needs a Beelink wall-meter run before publishing Beelink J/token claims.
• Same-machine Windows versus Linux performance. Windows LM Studio and WSL2/HIP community rows now exist, but they are not same-machine, same-model, same-shape comparisons against native Linux Vulkan/RADV.
• A default or balanced first-party 100 t/s Qwen3-Coder claim. The first-party Qwen3-Coder Q4_K_S speed-first row now reaches 100.99 t/s on b9851, but the practical balanced UD-Q4_K_XL row remains in the 96-99.6 t/s class depending on build/repeat length. A tuned community GMKtec report touched 100.0 t/s on Qwen3-Coder, and a separate first-party Qwen3-30B-A3B-Instruct-2507 IQ4_XS route reached 100.04 t/s, but those are separate evidence categories.
• Production-ready NPU/FastFlowLM inference. In the May 16 preflight the kernel saw amdxdna and /dev/accel/accel0, but XRT/FastFlowLM user-space is not installed and no local NPU LLM row is published yet.
• A broadly useful DFlash/PFlash speedup on Strix Halo. The official Gemma 4 31B QAT target and matched DFlash sidecar now load and serve on b10066, but DFlash n_max=8 was 5.54% slower at 5,471 prompt tokens and 20.42% slower at 21,855 on the measured synthetic shapes because acceptance stayed low. Representative chat, coding, and reasoning prompts still need separate profiling.
• A vLLM/DFlash server path that competes with llama-server or Ollama for a real 35B Strix Halo use case. Plain AWQ without the gated DFlash drafter was only a smoke test here at about 25 t/s.
• A pinned current-HIP long-context comparison against the current Vulkan/RADV path. Historical external rocWMMA evidence exists, but upstream removed that kernel in 2026 and the older local lhl branch failed to load the current Qwen3.6 GGUFs.
• Multi-machine clustering numbers from this guide's own hardware. Community RPC and USB4 tuning data exists in COMMUNITY_RPC.md and USB4_CLUSTER_TUNING.md, but it is not a local Beelink headline claim.

These numbers are local measurements from one primary machine: a Beelink GTR9 Pro with Ryzen AI MAX+ 395 and 128GB LPDDR5X-8000. Treat the headline results as directional unless your setup matches the measured run closely.

Performance depends on the exact hardware SKU, RAM configuration, BIOS UMA setting, IOMMU setting, firmware, kernel, Mesa/RADV version, ROCm version, Vulkan ICD selection, power profile, GPU clocks, thermal state, backend commit/build flags/container image, model file, quant type, model hash/path, context length, prompt length, generated token count, batch size, parallel slots, request concurrency, API endpoint, environment variables, and background system load.

If your setup differs, rerun the benchmark scripts and cite the date, command, CSV, raw log, chart, model file, and backend version with any copied claim.

Other projects and sites already read the data in this repository (see TRACTION.md), and a number copied without its qualification can say more than the measurement shows. The repository license stays MIT (see LICENSE). When you cite a headline, keep its qualification next to it, and give the date, the commit or release you read, and the source file.

All first-party numbers come from one Beelink GTR9 Pro with 128GB; community rows are labeled as such. The notes for each headline claim are in data/headline_claims.csv. To cite this repository, use CITATION.cff and add the commit hash or release tag you used and the date you read it.

• Quick Start (6 Steps)
• Setup Script
• 20-Second Summary
• Use This Guide If
• Maintenance And Evidence Status
• Qwen3.8 On Strix Halo
• What You Can Run: Quick Snapshot
• Use This If You Want
• Community-Tested Rules Of Thumb
• Results Wanted
• Community Results
• Best Current Setup Tested Here
• Headline Evidence
• Reproduce One Headline Result
• Not Yet Proven Here
• Do Not Copy These Claims Without Matching Setup
• Reuse And Citing The Numbers
• Documentation Map
• Hardware
• What You Can Run
• Benchmark Results
  • Benchmark Charts
  • Ollama Vulkan (RADV)
  • llama-server Multi-User Serving
  • ROCm HIP (llama.cpp)
  • Backend Comparison
  • Hardware Comparison
  • Long Context Performance
• Backend Decision Guide
• Server Shootout
• Setup Script Details
• Phase 1: BIOS Configuration
• Phase 2: Ubuntu 24.04 Installation
• Phase 3: Kernel Configuration
• Phase 4: Performance Tuning
• Phase 5: Ollama Setup (Vulkan)
• Phase 6: Benchmarking
• Phase 7: ROCm with llama.cpp (Containers)
• Phase 8: vLLM Serving
• Phase 9: Multi-Node Clustering (RDMA)
• Phase 10: SSH and Remote Access
• Vulkan Driver Comparison
• Key Findings and Corrections
• Known Issues
• Troubleshooting
• Kernel and ROCm Compatibility
• Power Measurement Status
• Testing Checklist
• Model Recommendation Guide
• Cost: Local vs Cloud
• Buying Guide
• Glossary
• FAQ
• Community Resources
• Credits and References
• Contributing
• Changelog
• License

The count is 10 described owner systems plus 3 independently attributable external evidence sources. One physical owner system counts once regardless of how many results it contributes; separate physical machines count separately even when they are the same product or belong to the same contributor. External packages whose unique hardware identity cannot be safely proved remain labelled as sources. This makes the public 13 systems/sources claim auditable without treating every row as an equal-quality benchmark.

For a reviewer-friendly cross-OEM view of what each source proves, how the evidence classes differ, and which validation is still missing, see the AMD Strix Halo System Evidence Matrix. A machine-readable version is available at data/system_evidence_matrix.csv.

This table is a product/context map, not an endorsement list or the source of the thirteen-source count above. Rows without linked community evidence are useful hardware context only.

Why this hardware? 96GB/128GB unified memory shared between CPU and GPU means you can run 70B+ models entirely on the GPU -- something an RTX 4090 (24GB VRAM) cannot do. You trade raw bandwidth (~215 GB/s vs ~1 TB/s on this Beelink) for the ability to run larger model artifacts (not automatically higher task quality) on one compact machine. Price changes quickly by vendor; check the Buying Guide before making a purchase decision.

Real-world generation speeds measured on the Beelink GTR9 Pro, primarily with Vulkan/RADV. Speeds marked with * are via llama-bench direct; others are via Ollama unless noted. Use this table to choose a first model, then follow the evidence links above before copying a benchmark claim.

Benchmarks below were run on 2026-03-20, 2026-03-21, 2026-04-26, 2026-05-03, 2026-05-07, 2026-05-16, 2026-05-26, 2026-05-27, 2026-05-31, 2026-06-01, 2026-06-02, 2026-06-05, 2026-06-07, 2026-06-11, 2026-06-12, 2026-06-21, 2026-06-30, and 2026-07-02. Primary benchmark system: Beelink GTR9 Pro. Recorded local runs used kernel 6.19.4, Mesa RADV 26.0.2-26.1.3 where captured, AMDVLK removed, and tuned accelerator-performance where captured; individual raw directories and CSV rows are the source of truth for exact run metadata, and some older or scout rows intentionally record missing metadata as not recorded. Before new benchmarks, record and verify the selected power/background policy; require tuned only for a tuned-profile reproduction.

These rows are included because they answer practical setup questions: which model to try first, which backend removes the most friction, which paths are only experimental, and which results are strong enough to cite.

These generated SVGs summarize the current structured benchmark data. The CSV files in data/ and raw logs in data/raw/ remain the source of truth; regenerate charts with python3 scripts/generate_charts.py.

Multi-user serving	Long-context prompt scaling

Filled-KV decode	KV-cache quantization tradeoff

Real versus synthetic prompts	Backend spot check

Qwen3.6-35B-A3B (Q4_K_M, ~20-23GB, MoE):

Qwen3.5-35B-A3B (Q4_K_M, ~23GB, MoE -- Ollama 0.20.4):

Qwen3-Coder 30B-A3B (Q8_0, ~32GB, MoE):

Qwen3-Coder-Next (~51GB artifact, 80B total / 3B active MoE):

Other Models:

Llama 3.1 70B (Q4_K_M, 42GB, Dense -- the "doesn't fit on RTX 4090" showcase):

A rough weight-streaming estimate is 215 GB/s divided by 42 GB, or about 5.1 t/s, compared with the observed 4.8 t/s. This simplified model omits compute, KV/cache traffic and implementation effects; it is not a proven ceiling or evidence that optimization cannot help. The artifact cannot fit wholly in an RTX 4090's 24GB VRAM; partial CPU offload is a separate memory/latency tradeoff.

The historical combined Mesa/power-profile change was reported alongside higher generation rates. It does not isolate a tuned-only effect or establish a universal percentage across models.

Single-user llama-bench tells you the ceiling for one stream. For a real local API box, the more practical question is what happens when multiple tools or users hit llama-server at the same time.

Qwen3.6-35B-A3B (UD-Q4_K_M, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):

Takeaway: continuous batching makes Strix Halo look much stronger as a local API server than single-user tg numbers suggest. -np 8 delivers about 2.7x the -np 1 aggregate throughput while keeping TTFT around 0.3 seconds. -np 16 works with no errors in this test, but aggregate throughput barely improves while per-user speed drops sharply.

Qwen3-Coder 30B-A3B (UD-Q4_K_XL, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):

Coding server takeaway: -np 8 is also the best measured setting for Qwen3-Coder, but here -np 16 is actively worse. More parallel slots are not automatically better.

Raw data: data/multi_user.csv, data/raw/2026-05-03/multi-user/, and data/raw/2026-05-03/multi-user-coder/.

Historical Qwen3.5-35B-A3B comparison (2026-03-21): b8298 to b8460 improved pp512 by about 24% and tg128 by about 25% on the recorded Vulkan pair. The build includes changes such as PR #19625, PR #20551 and PR #20334, but this endpoint comparison does not isolate their individual contributions.

Important caveats:

• The 25% observation is scoped to that Qwen3.5 pair. Qwen3-Coder's recorded 86.81→87.11 t/s pair did not show that gain. Small changes on dense models do not prove a bandwidth ceiling.
• For kyuz0's containers, inspect the actual binary revision and image digest. A tag or rebuild policy does not prove an already-installed container contains a particular update.
• WARNING: AMDVLK silently overrides RADV. If AMDVLK is installed, its /etc/vulkan/icd.d/amd_icd64.json takes priority over RADV. This reduced pp speed by about 39% in the recorded pair (1080 -> 660 pp512) without any visible error. Always set AMD_VULKAN_ICD=RADV or uninstall AMDVLK entirely: sudo dpkg -r amdvlk && sudo rm -f /etc/vulkan/icd.d/amd_icd64.json. Check your driver: RADV shows (RADV STRIX_HALO) (radv) with shared memory: 65536 in llama-bench output. AMDVLK shows (AMD open-source driver) with shared memory: 32768. We originally reported this as a llama.cpp regression -- it wasn't.

Qwen3.5-35B-A3B (Q4_K_M, 19.9GB, MoE) -- the biggest improvement:

Beginner rule: use RADV for Vulkan. Do not install AMDVLK. In the newer b8460 Vulkan comparison, RADV is faster than AMDVLK on both pp (+63%) and tg (+1.2%), and AMDVLK can silently hijack your Vulkan driver. Advanced note: ROCm/HIP is a different backend, not a Vulkan driver. HIP can still be worth testing for long prompts, RAG ingest, and other prompt-processing-heavy workloads.

Extended context scaling (b8460 RADV):

pp is virtually flat from 512 to 8192 tokens. Only 3% drop at 8K context.

Qwen3-Coder 30B-A3B (Q4_K_S speed-first and UD-Q4_K_XL balanced, MoE):

A controlled May 2026 rerun moved the balanced Qwen3-Coder 30B headline from 87 t/s to 97 t/s on b9010 Vulkan RADV. The 2026-05-07 b9049 max-performance campaign measured 96.76 t/s on UD-Q4_K_XL. A later strict-clean b9179 run measured 98.51 t/s with Q4_K_S, and the official b9851 Vulkan release binary lifted the same speed-first quant to 100.99 t/s. This is still not the default balanced row.

Gemma 4 26B-A4B historical non-QAT baseline (UD-Q4_K_M, 15.7GB, MoE) -- tested on b8933 (earliest build with Gemma 4 support):

This row is historical and non-QAT. The separate Gemma 4 26B-A4B IT QAT UD-Q4_K_XL route measured 74.80 t/s direct and 102.69 cold / 107.42 T3-only / 110.00 best-repeat t/s through a matched MTP llama-server route. Sampled saved server coding answers failed in both no-spec and MTP records, so this QAT server profile is not a practical recommendation; see the saved-output review. Keep direct and server/speculative speed evidence separate from task correctness.

The older non-QAT Gemma row was slower than the recorded Qwen MoE routes. Attention dimensions, mixed attention and active-parameter traffic are possible contributors, alongside runtime implementation. These unmatched rows do not isolate a cause or rule out llama.cpp improvements; the later non-QAT b9851 result reached 55.45 t/s.

KV precision caveat: an external KL experiment reported q8_0 KV KL 0.377 for Gemma 26B-A4B versus 0.108 for Gemma 31B, about 3.49×. It compared output distributions against an f16-KV reference; this is not a factor of downstream task-quality loss and not a comparison with Qwen. Keep f16 KV as this guide's conservative tested profile; lower-precision KV needs artifact/workload-specific qualification.

Llama 4 Scout 109B (Q4_K_M, 60.9GB, MoE -- 109B total params, 17B active):

The recorded 109B artifact ran at 18.3 t/s on the 128GB system and cannot fit wholly in 24GB VRAM. A nominal 17B-active-at-4-bit bandwidth estimate gives roughly 25 t/s at 215 GB/s, but omits mixed-quant bytes, KV/cache traffic and compute; it is a rough model, not an established performance ceiling.

Qwen3-Next 80B-A3B (UD-Q4_K_XL, 42.90 GiB, MoE -- 80B total params, 3B active, 256K context):

80 billion parameters running at 59 t/s on a mini PC. This measured Qwen3-family MoE has 80B total with only 3B active parameters and a 256K context window. Despite being 42.90 GiB of reported llama-bench model bytes (46.06 decimal GB), the MoE routing keeps only 3B params active per token. The measured route reached 59 t/s versus 38 t/s for the separate Qwen3-Coder-Next 80B-A3B artifact/runtime row. This is not a matched architecture or quality comparison. The 2026-05-16 b9172 check improved this row, while Qwen3-Coder, Qwen3.6, and gpt-oss did not improve on the same latest-stack rerun.

Qwen3.6-35B-A3B (Q4_K_M, 19.9GB, MoE -- drop-in upgrade from Qwen3.5, released April 2026):

Qwen3.6 is a measured runtime-compatible successor on these routes; this guide has not established a general coding/reasoning-quality improvement over Qwen3.5. Older April data showed a 13% UD-Q4_K_M penalty, but the controlled May b9010 and b9049 reruns did not reproduce that large gap. Prefer plain Q4_K_M when you have a direct-compatible GGUF, but treat the old "UD is always 13% slower" warning as superseded until same-build plain-vs-UD is rerun.

The March/May b8460 route on kernel 6.19.4 required HSA_OVERRIDE_GFX_VERSION=11.5.1 + HSA_ENABLE_SDMA=0. Those variables belong to this dated comparison and are not the current default. Current ROCm builds that detect the iGPU natively as gfx1151 should be run without a global HSA architecture override.

ROCm's historical pair improved from 47.87 to 54.67 tg128 (+14%). Vulkan RADV remained faster in the recorded short-context pair: 1080 versus 1047 pp512 (+3%) and 64.85 versus 54.67 tg128 (+19%). These endpoint deltas cannot be decomposed into generic and Vulkan-specific optimization percentages without isolated A/B evidence. HIP batching and prompt-heavy workloads remain separate tests; the removed rocWMMA kernel is historical evidence.

ROCm HIP spot check (2026-05-03, b8460 HIP build):

This HIP run required LD_LIBRARY_PATH=/usr/local/lib/ollama/rocm plus the HSA override and emitted a missing TensileLibrary_lazy_gfx1151.dat warning. Treat it as a ROCm HIP baseline, not a tuned rocBLASLt/rocWMMA result. Vulkan RADV remains the recommended short-context generation backend.

Build version matters enormously:

The early Qwen3.5 pair showed a substantial build-to-build gain. That observation supports checking the exact runtime before tuning, but does not promise that newer builds improve every MoE or that dense models cannot improve.

Batch size and ubatch tuning results (b8298, for reference):

We swept batch sizes 64-2048 and ubatch sizes 32-1024. Result: default 512 is optimal. No headroom via tuning -- the improvement came from updating the build.

Pinned candidate build: llama.cpp v0.4.1 with Vulkan

This is a source/CLI-aligned candidate example, not a locally qualified replacement for the measured builds. Use a new directory; do not overwrite a working checkout. The v0.4.1 parsers use --load-mode instead of the removed load flags (upstream change). auto below is a starting policy, not a measured optimum. Historical commands retain their original flags and build IDs.

git clone --branch v0.4.1 --depth 1 https://github.com/ggml-org/llama.cpp llama.cpp-v0.4.1-vulkan
cd llama.cpp-v0.4.1-vulkan
CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -B build -S . \
  -DGGML_VULKAN=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -G "Unix Makefiles"
cmake --build build -j$(nproc)

# Inspect this binary's CLI before the first candidate smoke
./build/bin/llama-bench --help
# Candidate benchmark (not a published performance result)
AMD_VULKAN_ICD=RADV ./build/bin/llama-bench \
  -m ~/models/your-model.gguf \
  -fa on -ngl 999 --load-mode auto -p 512 -n 128

# Reproduce only the dated b8460 / kernel 6.19.x evidence:
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0
export ROCBLAS_USE_HIPBLASLT=1

Do not add this override globally by default for current ROCm images. First run llama-cli --list-devices; if the current stack already reports gfx1151, leave HSA_OVERRIDE_GFX_VERSION unset.

Llama 2 7B (Q4_K_M, 3.8GB, Dense):

AMDVLK was 3-4X slower on pp for the recorded Llama 2 7B pair. A 2 GiB allocation limit is a possible contributor, not an isolated causal measurement; the measured beginner route uses RADV.

Qwen3-0.6B (Q8_0, 762MB, Dense) -- maximum throughput:

Historical note (March 2026): The measured b8460/kernel 6.19.4 combination detected the iGPU incorrectly without HSA_OVERRIDE_GFX_VERSION=11.5.1 and HSA_ENABLE_SDMA=0. Preserve those variables when reproducing that exact row. Current ROCm images with native gfx1151 support should not inherit the workaround.

Previous results on kernel 6.18.14 (for reference -- these worked):

Vulkan llama-bench Direct (kyuz0 containers, b8298) -- March 2026:

Critical finding (b8298): The recorded AMDVLK Llama 2 7B pp result was 3-4X slower. Its reported 2 GiB single-buffer limit is a hypothesis for investigation, not a proven sole cause. On MoE models, AMDVLK was slightly faster on tg (+6.5%) with b8298, but this advantage disappeared with b8460 -- see the key build comparison. For beginners: keep AMDVLK removed and use RADV for Vulkan.

Vulkan RADV vs ROCm HIP (same build b8460, Qwen3.5-35B-A3B):

Vulkan RADV wins on both pp512 and tg128 for this Qwen3.5 b8460 short-context pair. ROCm works on kernel 6.19.x with the HSA override fix, and newer spot checks show HIP can win longer prompt-processing rows. Use llama-bench or llama-server directly instead of Ollama to avoid the measured short-context overhead on Qwen3.6. As upstream context rather than a local measurement, open ollama/ollama#15601, rechecked 2026-08-30, reports about 34 t/s through Ollama's b7437-era vendored llama.cpp versus 52-56 t/s standalone (roughly a 56% gap) because the vendored tree lacks newer Vulkan Wave32 FlashAttention and graphics-queue changes; whether Ollama 0.33.2 closed that gap is unknown.

Based on our measurements and lhl's detailed testing:

rocWMMA route removed (2026-08-30 note): the earlier "ROCm + rocWMMA (tuned) for long context" row is retracted. This guide measured a 73% regression on the ROCm 7.2 rocWMMA container (see Known Issues), and upstream llama.cpp removed the rocWMMA FlashAttention kernel entirely in PR #26046 (merged 2026-07-24); HIP builds now always use the newer MMA kernel. External rocWMMA columns in older tables below are historical evidence only.

Capacity and bandwidth are hardware context, not matched inference benchmarks.

Do not rank these systems from unmatched ranges or capacity alone. The earlier Beelink $4,349 US snapshot was checked August 29, 2026; other older price mentions are not a current exact-SKU comparison. Use the dated buyer snapshots and verify complete-system configuration, region and availability before a purchase decision. External context: NVIDIA, Framework discussion, lhl's tests.

Local prompt-processing scaling on the May 2026 b9010 Vulkan RADV stack:

Local result: long-prompt ingestion remains usable through 64K on both tested MoE models. This table measures prompt processing, not generation after a fully occupied KV cache. Raw data: data/long_context.csv and data/raw/2026-05-03/long-context/.

Filled-KV decode through llama-server on the same stack:

KV-cache takeaway: q4_0/q8_0 KV improves Qwen3.6 decode speed after the context is filled, but slows prompt ingestion enough that full first-turn wall time is worse than f16 in this benchmark. Use f16 for first-turn long prompts. Consider q4_0/q8_0 only when memory pressure or long continued generation matters more than prompt-ingest speed. The 128K f16 rows completed without truncation. Raw data: data/filled_kv_decode.csv, data/raw/2026-05-03/filled-kv-decode/, and data/raw/2026-05-03/filled-kv-decode-128k/.

Real-corpus 64K check using this guide's own documentation files:

Real-corpus takeaway: synthetic repeated-token prompts are optimistic for prompt ingest. Real documentation text slowed prompt eval by 24-33%, while decode-after-fill barely changed. Raw data: data/raw/2026-05-03/filled-kv-decode-real-corpus/.

Based on lhl's measurements with gpt-oss-120b (tg32):

Historical external result: at 32K context, standard ROCm dropped to 17.82 t/s and Vulkan held 31.54 t/s (1.8X faster), while lhl's tuned rocWMMA branch was the overall winner on that dated stack at 36.43 t/s—2X faster than standard ROCm and 15% faster than Vulkan. Upstream removal means this is not current build guidance.

Historical external evidence at 130K tokens from strixhalo.wiki is preserved below. The wiki's public mirror repository lists 2025-11-01 as the last change to that page (history, checked 2026-10-02), so its content may predate the 2026-07-24 rocWMMA removal and ROCm 7.x/10.0; do not use it as current build guidance.

Which backend should I use?

Note: an earlier version of this diagram recommended the rocWMMA-tuned ROCm branch for long context. That route measured a 73% regression on ROCm 7.2 and failed to load current Qwen3.6 GGUFs here — see Things That DON'T Work before copying it.

The short version is in Setup Script. This section keeps the full scope, version pinning, the unattended curl | bash route and how to undo the changes.

Scope before running: this automatic route is for 128GB-class systems with at least 120GiB visible RAM on Ubuntu 24.04. It is not a general 64GB/96GB installer, and it stops above about 136GiB visible RAM: 192GB-class systems (for example Ryzen AI Max+ PRO 495) are not qualified, so do not reuse the 128GB values there. The check reads OS-visible RAM only, so a 192GB system with a large BIOS UMA reserve can still pass it; confirm your installed memory first. Ubuntu 26.04 is not qualified by this guide; the script stops on any OS other than Ubuntu 24.04 unless you deliberately pass STRIX_HALO_ALLOW_UNQUALIFIED_OS=1 to it (for example STRIX_HALO_ALLOW_UNQUALIFIED_OS=1 bash setup.sh). Configuration handling has offline fixture coverage; the revised script's fresh-install and upgrade paths still need hardware qualification. Review the script and existing configuration first: a later conflict can stop a partially completed run; this is not a transactional installer with a complete automatic rollback.

If /etc/default/grub.d/ contains any *.cfg files, the script stops before changing the system and asks for manual review. These files can override the boot settings it writes. This conservative check also stops for harmless fragments; keep them and use the manual boot-settings reference.

Existing-user result, September 19: the scoped qualification passed text, separate-model image, executed tools and pinned WebUI on the existing Ollama 0.32.15 service after restart. The historical fresh-install reboot pin stays 0.31.2; isolated 0.34.2 is useful but not default. These tests are not a new clean-install or full-host-reboot qualification.

Fresh installations request Ollama 0.31.2, the reboot-qualified runtime. An existing installation is retained, with its version reported; it is not automatically downgraded. Set OLLAMA_VERSION explicitly to test another release. This pins only the requested runtime: model tags, Mesa and other packages can change, so record the actual stack and complete the reboot/GPU checks before comparing with a measured profile. See Ollama version selection.

Before you change anything: the complete start route walks through the prerequisites, disk checks, the expected output of each step and help when something fails. The script edits GRUB, adds the kisak Mesa PPA and runs a full apt upgrade -y; the manual undo table is untested on hardware. Ollama has no password, so keep its API on loopback. Published advisories include the historical 0.31.2 pin; read the security status. If you already have a working setup, back it up and test the restore before running the script.

Both the git clone route in Setup Script and the curl | bash route below run the unpinned main branch, which has not been fresh-install qualified on hardware. For a reproducible run, review the script and pin it to a tag or commit you have reviewed (git checkout <tag-or-commit> before bash setup.sh, or replace main in the raw URL with that commit). No tag is currently marked as fresh-install qualified for the revised script.

Inspect the script first with less setup.sh before running it: it edits GRUB boot parameters, adds the kisak Mesa PPA, and runs a full apt upgrade.

For unattended copy/paste installs, the same script can also be run as:

curl -fsSL https://raw.githubusercontent.com/hogeheer499-commits/strix-halo-guide/main/setup.sh | bash

This also adds the kisak Mesa PPA and runs a full apt upgrade -y, and it fetches unpinned main (see the pinning note above). Install the tools from the apt install line in Setup Script first; if a required tool is missing, the script stops before it changes anything.

This installs the Linux-side Vulkan/RADV + Ollama path, configures Ollama for Vulkan, pulls a model, and prepares a verification benchmark. If the script changes boot parameters, reboot first and then run bash ~/bench-ollama.sh.

Untested on hardware: these manual reversal steps have not been run on the test system. Review each one against your own configuration first, and only undo what the script actually changed on your machine.

Do this BEFORE installing the OS.

Navigate to Integrated Graphics then UMA Frame Buffer Size and set it to 512MB if your BIOS exposes that option. If your vendor BIOS only exposes 2GB as the minimum, leave it at 2GB; do not flash or fight the BIOS only to chase 512MB.

Why? On the primary Beelink 128GB system, the default BIOS setting reserved ~97GB for GPU VRAM and left only ~31GB visible to the OS. Setting UMA to 512MB lets Linux see almost all system RAM. Some vendor BIOSes use 2GB as the lowest fixed reserve; that is fine if Linux still sees the large system-memory pool. Vulkan/RADV uses GTT system memory, so the fixed UMA reserve is not the total memory available to the iGPU path. The practical check is free -h: a 128GB box should show roughly 124-126GiB usable, not ~31GiB.

For a normal system, leave IOMMU Enabled or at the firmware default. This preserves NPU access, RDMA/VFIO/passthrough support, and mobile suspend behavior.

Use amd_iommu=off only as an optional profile for an always-on desktop benchmark box where the NPU and suspend do not matter. lhl's memory-bandwidth testing measured about 6% faster memory reads with IOMMU off (234 vs 221 GB/s), and this guide's primary Beelink headline runs used that profile. It is a reproducibility choice, not a universal requirement: a GMKtec EVO-X2 community Vulkan/RADV run with translated IOMMU mode reproduced the guide within about 2%.

Laptop/tablet and NPU warning: a reproduced ROG Flow Z13 case found that amd_iommu=off prevented s0i3 hardware sleep, leaving 0% s0ix residency, spinning fans, heat, and battery drain. The Linux amdxdna driver also now refuses to run without IOMMU because the NPU firmware requires it. See kyuz0 issue #104 and Linux commit a8878e19.

Install Ubuntu 24.04 LTS Desktop with default settings. After installation:

sudo apt update && sudo apt upgrade -y

If a specific Ubuntu 24.04 remote-desktop, screen-sharing or monitoring tool needs X11, edit GDM's daemon section. Headless inference does not require this desktop change.

sudoedit /etc/gdm3/custom.conf

Under [daemon], uncomment or set WaylandEnable=false. If [daemon] is missing, create that section and place the key immediately beneath it. Keep other sections and settings intact; do not append the key under [debug]. Repeated edits should leave one active key in [daemon]. Reboot deliberately after saving your work.

Ubuntu 26.04 LTS (released April 2026) ships with Linux 7.0, Mesa 26.0, and native apt install rocm. However, 26.04 is Wayland-only (X11 switch above does not work) and it is not qualified by this guide; community 26.04 reports stay separate from first-party results. On 24.04, use the stock Ubuntu HWE kernel (linux-hwe-7.0 has been in noble-updates since 2026-05-28; the September 19 acceptance ran 7.0.0-31) and Mesa RADV from the kisak PPA. Mainline kernel PPA builds are unsupported and receive no security updates, so this guide no longer recommends them. AMD's RDNA3.5 system notes list Ubuntu 24.04 HWE 6.17.0-19.19~24.04.2 or later as the minimum. As of 2026-09-25, the point releases Ubuntu 24.04.5 (2026-09-10) and 26.04.1 (2026-08-27) exist; neither has been measured here. Upgrading is not needed for LLM performance. This guide stays on 24.04 LTS.

Measured setup: historical headline runs used kernel 6.19.4; the September 19 existing-user acceptance used 7.0.0-31. This is not a request to upgrade a working kernel.

• Older kernels may have gfx1151 stability or ROCm issues.
• The dated b8460/kernel 6.19.4 ROCm route used the documented 11.5.1 override because that stack otherwise reported the iGPU incorrectly.
• Current ROCm images that already detect native gfx1151 should run without a global HSA architecture override.
• Treat kernel guidance here as this guide's tested state, not a universal support matrix.

Check your kernel:

uname -r

The following is the recorded 128GB Beelink memory profile, with a small UMA reserve. It is not a preset for 96GB systems. No automatic smaller-RAM profile is qualified here; retain distribution defaults and investigate actual runtime allocation failures before choosing host-specific limits.

Use sudoedit /etc/default/grub and preserve unrelated parameters in GRUB_CMDLINE_LINUX_DEFAULT. For the selected 128GB profile, its relevant parameters are:

amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0

gttsize deprecation (checked 2026-09-25): upstream kernel master (7.3-rc4) logs that configuring gttsize via the module parameter is deprecated in favour of ttm.pages_limit. Because this profile's GTT size (128 GiB) differs from its TTM limit (120 GiB), that code also logs an "unusual" GTT/TTM mismatch warning. AMD's RDNA3.5 notes raise only the TTM limit (amd-ttm --set). The recorded profile is unchanged until a reversible reboot A/B (TTM-only vs this profile; cwsr_enable on vs off) is measured.

Leave IOMMU at its normal default for the buyer route. Preserve existing configuration and resolve conflicting parameter values before running:

sudo update-grub

Reboot deliberately, then compare /proc/cmdline and the live module parameters with the chosen profile. A second setup run before reboot still has pending boot changes. Kernel versions can change parameter support and effective limits.

Use one reviewed source for each module option. Earlier guide revisions wrote /etc/modprobe.d/amdgpu_llm_optimized.conf with gttsize=122800, conflicting with the GRUB value above. The script now stops for manual migration if that legacy file exists and does not create a competing modprobe profile.

Inspect all relevant modprobe.d files, preserve administrator options, and resolve duplicate/conflicting values deliberately. If you change module options embedded in initramfs, run sudo update-initramfs -u -k all and require success before rebooting. Do not infer live state from the presence of a file.

Use the distribution's device rules. Do not create world-writable (0666) KFD or DRM rules. Older revisions of this guide created /etc/udev/rules.d/99-amd-kfd.rules; review and back up that file, remove only the obsolete guide rules, and reload rules/reboot before checking permissions. Preserve any unrelated administrator rules. The script stops for manual review when that legacy file exists.

On the Ubuntu route, add your interactive user to both GPU groups, then log out and back in:

sudo usermod -aG render,video "$USER"

Inspect the actual device owners, modes and service identity:

id
ls -l /dev/dri/renderD* /dev/kfd
systemctl show ollama -p User -p Group -p SupplementaryGroups

Vulkan needs access to its render node; ROCm also needs KFD access. A missing KFD node does not by itself disqualify Vulkan. If Ollama runs as the ollama user, check id ollama separately: your login's groups do not grant the service access. Add only the group that owns the required device to the actual service account, then restart the service and verify GPU offload. Distribution rules and service restrictions vary; these checks are not a hardware qualification.

Choose and record one power policy per route/campaign. The script preserves the existing policy by default; POWER_POLICY=tuned bash setup.sh selects the older accelerator-performance reproduction profile and stops if a conflicting daemon is active. The commands below are for deliberately selecting that profile, not a universal benchmark prerequisite. Controlled desktop-performance/DPM-auto runs are also retained in Reproducibility.

sudo apt install tuned -y
sudo systemctl disable --now power-profiles-daemon
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance

Verify:

tuned-adm active
# Expected: Current active profile: accelerator-performance
systemctl is-active power-profiles-daemon
# Expected: inactive

Power-policy effects depend on the workload and stack. The historical combined driver/policy observations do not isolate a universal tuned percentage. The 221/234 GB/s external comparison concerns the IOMMU experiment, not a matched tuned-only test.

Avoid competing power-policy managers. For a tuned reproduction, verify tuned is active and power-profiles-daemon inactive. For a different recorded policy, verify that policy instead; do not alter it merely to pass an older checklist.

The default Mesa on Ubuntu 24.04 is significantly slower. Upgrade to Mesa 26.0.2 or newer from the kisak-mesa PPA. Exact driver metadata is recorded per run in the CSV and raw evidence when available.

sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt update
sudo apt upgrade -y

Verify:

vulkaninfo --summary 2>&1 | grep driverInfo
# Expected: Mesa 26.0.2+ RADV from kisak-mesa PPA.

Impact: Mesa 25.2.8 to 26.0.1 gave +9% prompt eval (87 to 96 t/s). Mesa 26.0.1 to 26.0.2 gave an additional small improvement.

Note: Investigate DKMS failures before treating an upgrade as complete; see Troubleshooting.

Record clocks under the actual workload. A 2900 MHz target belongs to the older high-DPM desktop profile, not every model, OEM or idle state:

cat /sys/class/drm/card*/device/pp_dpm_sclk
# Asterisk identifies the currently selected state; compare with the recorded profile.

A historical stack was observed stuck at 900 MHz. Low clocks can also reflect idle/light load, thermal or power policy; inspect utilization and the selected profile before diagnosing a fault.

dpkg-query -W 'linux-firmware*'

Firmware compatibility: linux-firmware-20251125 is an upstream firmware tag with a documented Strix Halo ROCm failure history in Fedora (rpm) packaging, where that update has since been recalled (toolbox source). Ubuntu version strings (for example 20240318.git3b128b60-...) never contain that tag, so on Ubuntu you cannot identify the affected state by that name. If symptoms began after a firmware update, inspect the exact package versions and your distribution's available versions. The guide's historical working Ubuntu package is recorded in Reproducibility; it is not a universal firmware guarantee.

Ubuntu 24.04 package split (checked 2026-09-25): since 2026-09-03, linux-firmware in noble-updates is a metapackage. The AMD GPU blobs ship in the separate linux-firmware-amd-graphics package. Checking, restoring or holding only linux-firmware no longer controls the amdgpu firmware; check, hold or restore linux-firmware-amd-graphics instead. Downgrading the metapackage to an older monolithic 0ubuntu2.x package reshuffles the split packages; review what apt proposes before confirming.

dpkg-query -W 'linux-firmware*'
apt-cache policy linux-firmware-amd-graphics linux-firmware

Select an unaffected package available for your distribution, install that exact version using its package manager, then reboot and repeat the GPU/model checks. A version from another distribution is not a safe substitute. No known-good linux-firmware-amd-graphics version has been qualified by the guide. apt-mark hold only prevents package changes: it does not downgrade firmware. If you temporarily hold a restored working package (on Ubuntu 24.04: linux-firmware-amd-graphics, not only the metapackage), record the reason and revisit the hold when a suitable update is verified.

Ollama is the easiest way to run LLMs on Strix Halo. With the right configuration, it works great.

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.31.2 sh

Version 0.31.2 is the reboot-qualified fresh-install baseline. The script retains existing runtimes; verify their actual running version separately. Newer-model routes, including Qwen3.8, have their own measured runtime requirements and are explicit opt-ins rather than an automatic baseline upgrade.

Historical update (April 2026): The measured Ollama 0.20.4/kernel 6.19.x ROCm route used HSA_OVERRIDE_GFX_VERSION=11.5.1 (ollama/ollama#14855). That row measured 42.4 t/s versus 46.6 t/s on Vulkan for Qwen3.5-35B. Current native-gfx1151 ROCm builds should be tested without a global HSA override; Vulkan remains the easier measured beginner path.

sudo systemctl edit ollama

Preserve existing settings and add the following between the comment lines. Check for existing assignments first; resolve conflicting values deliberately. The automated script uses a separate 60-strix-halo-guide.conf and refuses conflicts instead of replacing your override.conf.

[Service]
Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_IGPU_ENABLE=1"
Environment="HIP_VISIBLE_DEVICES=-1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="AMD_VULKAN_ICD=RADV"
Environment="VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json"
Environment="OLLAMA_NUM_PARALLEL=1"

Reload and inspect the effective environment before restarting. Later drop-ins can override values; EnvironmentFile and UnsetEnvironment can change the final environment too. Review those manually if present. Do not paste service environment output publicly: it can contain credentials.

sudo systemctl daemon-reload
systemctl show ollama -p Environment -p EnvironmentFiles -p UnsetEnvironment
sudo systemctl restart ollama

# Control MoE model (~22.6GB of blobs on 2026-10-14; tags move): measured for speed, not ranked for general or coding quality (see docs/models.md)
# The historical ~60 t/s 0.31.2 row used manifest 07d35212591f. The current tag has
# draft_num_predict 2 (MTP); a 2026-09-26 matched check stayed in that performance class.
# Record ollama list and ollama show --parameters before comparing results.
ollama pull qwen3.6:35b-a3b

# Current official dense multimodal model, measured route in this guide (~18GB)
# Requires Ollama 0.32.12 or later; measured on 0.32.13 (not the 0.31.2 pin above)
# The plain tag carries draft_num_predict 4, so it gives the MTP build (registry check 2026-09-30).
# The artifact is identified by its model blob (f5f1dd8920d4) next to the manifest ID; to reproduce
# the measurement exactly, pin qwen3.8:27b-mtp-q4_K_M (MTP) or qwen3.8:27b-q4_K_M (no draft).
ollama pull qwen3.8:27b

# Higher quality MoE, Q8_0 quantization (~32GB)
ollama pull qwen3-coder:30b-a3b-q8_0

# Google's MoE model, strong reasoning (~16GB)
ollama pull gemma4:26b-a4b

# Large coding MoE (80B total, 3B active) for complex tasks (~51GB)
ollama pull qwen3-coder-next

ollama run qwen3.6:35b-a3b

Verify a useful text response and inspect ollama ps plus runtime logs for GPU offload. Speed alone does not establish correct setup; match the exact model/runtime/profile before comparing rates.

From a local checkout of this guide:

bash scripts/ollama_smoke.sh
bash scripts/ollama_smoke.sh qwen3-coder-next 'Explain backpropagation in simple terms.'

The setup script also installs the same helper as ~/bench-ollama.sh. It serializes quoted/multiline prompts, bounds HTTP duration, requests up to 128 output tokens, and rejects HTTP errors, empty/incomplete responses and invalid timing counters. Natural completion can be shorter. This is a smoke check, not a controlled throughput comparison: inspect ollama ps and runtime logs to verify actual GPU offload, then use the reproducibility protocol.

bash scripts/ollama_smoke.sh qwen3.6:35b-a3b \
  'Review a Python web application architecture covering sessions, database access, caching, background jobs and error handling. Explain the tradeoffs with concrete examples.'

Record the actual input/output token counts, cache policy, warmup and repetitions before comparing rates. A longer instruction is not by itself a long-context test.

Prompt processing speed scales with prompt length due to GPU parallelism:

For ROCm-specific workloads, batch processing, and long-context experiments, use llama.cpp with ROCm via kyuz0 containers. For short-context MoE inference, current measured results still favor Vulkan RADV.

Current rule: start with no global HSA_OVERRIDE_GFX_VERSION and verify that llama-cli --list-devices reports gfx1151. A 2026-07-29 Beelink / Ubuntu validation found that a stale host HSA_OVERRIDE_GFX_VERSION=11.0.0 was inherited by Distrobox, changed detection to gfx1100, and crashed current ROCm 7.2.4. Removing it restored native gfx1151 detection and inference. The older 11.5.1 workaround is retained only for reproducing the dated b8460/kernel 6.19.4 rows.

sudo apt install podman distrobox -y

Note: Ubuntu 24.04 ships both distrobox (1.7.0 in noble universe, checked 2026-09-25) and a podman-toolbox package. This guide uses Distrobox; earlier guide notes reported GPU-access problems with the default toolbox on Ubuntu, which has not been re-qualified. If you need a newer Distrobox than the Ubuntu package, install a pinned upstream release after reviewing it; do not pipe an unpinned script into sudo sh.

The rocm-7.2 command below is a dated reproduction route, not a floating latest recommendation. The upstream toolbox now documents a rocm-10.0 stable route; use its current instructions and pin an image digest for a new qualification campaign. This guide has not promoted that available image to its measured default.

distrobox create llama-rocm-72 \
  --image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.2 \
  --additional-flags "--device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"

Container risk (checked 2026-09-30): this image comes from a third party and is selected by a moving tag, not a digest. The command runs it without the default seccomp profile (--security-opt seccomp=unconfined) and with the sudo group, and Distrobox gives the container access to your home directory. These flags come from the toolbox's own instructions, and this guide has not tested whether the image works without them. Record and pin an image digest, and see Containers and closed binaries.

distrobox enter llama-rocm-72
rocm-smi  # Should show your gfx1151 GPU

The following is a historical reproduction command, not a v0.4.1 example. The rocm-7.2 tag alone does not pin a binary: match the archived build/digest and inspect llama-bench --help before using its old -mmp flag. For a new candidate, use the explicitly versioned build example instead.

export ROCBLAS_USE_HIPBLASLT=1
llama-bench -m ~/models/your-model.gguf -fa 1 -ngl 999 -mmp 0 -p 128,512 -n 128

Critical flags:

The kyuz0 pre-built binary includes the critical compiler flag --amdgpu-unroll-threshold-local=600 which works around the LLVM compiler regression in ROCm 7+. Self-compiled binaries without this flag may be significantly slower.

Pinned v0.4.1 ROCm/HIP route: scoped Coder/Gemma controls passed, not all-model correctness. See the exact test limits. Use a separate checkout/build; retain the known control. See the HIP correctness queue before promotion.

# Inside a ROCm container
git clone --branch v0.4.1 --depth 1 https://github.com/ggml-org/llama.cpp llama.cpp-v0.4.1-hip
cd llama.cpp-v0.4.1-hip

# Current HIP build
cmake -B build -S . \
  -DGGML_HIP=ON \
  -DAMDGPU_TARGETS="gfx1151" \
  -DCMAKE_HIP_FLAGS="-mllvm --amdgpu-unroll-threshold-local=600" \
  -DCMAKE_BUILD_TYPE=Release

cmake --build build -j$(nproc)

Upstream change (2026-08-30 review): do not pass GGML_HIP_ROCWMMA_FATTN. llama.cpp PR #26046, merged 2026-07-24, removed the rocWMMA FlashAttention implementation; HIP builds now use the newer MMA kernel. The older local and external rocWMMA material in ROCM_ROCWMMA_BASELINE.md is historical evidence only.

kyuz0's vLLM toolboxes enable API serving on gfx1151. Treat vLLM as a separate serving benchmark path, not as something to install into the host Python environment.

The command below preserves the May :stable reproduction route. Upstream now uses :latest for its verified vLLM route and :dev for development. Those moving tags are not immutable benchmark identities: select and record a digest, then qualify the exact image before treating it as a new recommendation.

Local preflight status: vllm-gfx1151 was created and smoke-tested on 2026-05-03 with the :stable image. See VLLM_BASELINE.md. This is setup evidence, not a throughput benchmark.

distrobox create vllm-gfx1151 \
  --home "$HOME/distrobox/vllm-gfx1151" \
  --image docker.io/kyuz0/vllm-therock-gfx1151:stable \
  --additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined"

distrobox enter vllm-gfx1151
rocm-smi
# Historical small-model smoke shape; this moving image is not an immutable pin.
VLLM_DISABLE_COMPILE_CACHE=1 vllm serve Qwen/Qwen3-0.6B \
  --host 127.0.0.1 --port 18001 \
  --max-model-len 2048 --max-num-seqs 1 --gpu-memory-utilization 0.20 \
  --dtype auto --attention-backend TRITON_ATTN --enforce-eager
# In another terminal: curl --fail http://127.0.0.1:18001/health
# Inspect ss -ltn: port 18001 must be loopback-only; test other-device denial.

The same container caveat applies (checked 2026-09-30): third-party image, moving tag, --security-opt seccomp=unconfined and host integration. A separate home keeps application files apart but is not a security sandbox. Do not use the launcher's unspecified HOST default; the explicit vllm command above supplies --host. Keep the vLLM server on loopback; the text of AMD's clustering playbooks read on 2026-09-30 binds it to 0.0.0.0 and contains no authentication or exposure note (not exhaustively checked), see Clusters.

Record vLLM results separately from llama.cpp server results. At minimum, capture image tag, ROCm/TheRock build, model, quant, max context, concurrency, aggregate throughput, TTFT, p50/p95 latency, memory use, and any kernel compile/cache warmup behavior.

Known vLLM issues on gfx1151:

1. Qwen3.5 block_size validation (issue #28): Hybrid mamba/attention models compute block_size=1056 which gets rejected by a hardcoded whitelist. Fix available in the issue.
2. MIOpen encoder hang (issue #30): Vision models hang during kernel search because MIOpen lacks pre-compiled solver DBs for gfx1151. Workaround: disable encoder profiling.

Tested models on vLLM:

For models that exceed 128GB, you can cluster multiple Strix Halo machines using RDMA.

From kyuz0's vLLM clustering guide:

Hardware needed:

• 2x Strix Halo machines (e.g., Framework Desktop)
• 2x Intel E810-CQDA1 100GbE NICs
• 1x DAC cable (direct attach copper, no switch needed for 2 nodes)

Performance:

• ~50 Gbps bandwidth, ~5 us latency (vs ~70-100 us TCP/IP)
• TP=2 across two 128GB machines provides 256GB aggregate physical RAM, with per-node runtime/headroom constraints; it is not one universally shared allocation pool.
• This two-node RDMA/vLLM route is separate from AMD's February 25 demonstration: four Framework 128GB nodes, llama.cpp RPC, a 375GB Kimi K2.5 UD-Q2_K_XL artifact and 5Gbps Ethernet. That external capacity demonstration does not qualify trillion-parameter inference on the two-node recipe.

Additional kernel parameter for clustering:

pci=realloc

Network configuration:

# Set MTU to 9000 (jumbo frames)
sudo ip link set <interface> mtu 9000

sudo apt install openssh-server fail2ban -y

sudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo systemctl restart ssh

fail2ban starts automatically and blocks IPs after repeated failed login attempts. It reduces failed-login noise; it does not replace key login and a firewall (Steps 10.3 and 10.4).

On your client, create a key and install it on the box. Confirm that key login works from a second terminal before changing anything on the server:

ssh-keygen -t ed25519
ssh-copy-id <user>@<box>

Only then set PasswordAuthentication no in /etc/ssh/sshd_config (see sshd_config(5)) and restart ssh, keeping the first session open until the second login works. fail2ban reduces noise; key login and a firewall do more.

Then check what sshd actually uses, not what the file says:

sudo sshd -T | grep -E "^(passwordauthentication|permitrootlogin|pubkeyauthentication)"

sshd -T prints the effective configuration. The Ubuntu 24.04 manual page for sshd_config(5) says that for each keyword "the first obtained value will be used", and that the Debian openssh-server package includes /etc/ssh/sshd_config.d/*.conf at the start of the configuration file, so options set there override those in /etc/ssh/sshd_config (manual page, read 2026-10-02). A drop-in with PasswordAuthentication yes can therefore leave password login on although your own line says no. If the output shows passwordauthentication yes, find the file that sets it with sudo grep -ri passwordauthentication /etc/ssh/sshd_config /etc/ssh/sshd_config.d/ and change that one. The same check covers PermitRootLogin from Step 10.2.

• Allow port 22 only from your LAN or a VPN, for example sudo ufw allow from <lan-subnet> to any port 22 proto tcp, and do not forward it on your router. ufw is disabled after installation (ufw(8), Ubuntu 24.04, read 2026-10-02), so a rule does nothing until sudo ufw enable; allow your own SSH source before you enable it. Docker-published ports bypass ufw and Tailscale accepts tailnet traffic early; see Network exposure.
• Check the firewall with sudo ufw status numbered. It lists each rule with its number, and a firewall reported as inactive applies no rule. ufw(8) says rule ordering is important and the first match wins: a narrow allow rule does not close port 22 while a broader allow such as 22/tcp ALLOW Anywhere is in the list. Remove that rule by its number with sudo ufw delete <number>. With IPv6 enabled, ufw(8) says deleting a generic rule by number deletes only the specified rule, so look in the numbered list for the matching IPv6 entry (shown with (v6)), delete it too and run sudo ufw status numbered again to confirm that both are gone.
• Check IPv6: ss -ltn 'sport = :22' showing [::]:22 means a global address may be reachable. Whether your router blocks inbound IPv6 is router-specific, so test from outside your network.
• Reach Open WebUI remotely through a tunnel instead of an open port: ssh -N -L 127.0.0.1:3000:127.0.0.1:3000 <user>@<box>, then open http://127.0.0.1:3000 on your client. This matches the candidate's explicit HOST=127.0.0.1 and PORT=3000; verify the actual listener before relying on the tunnel.

These steps and checks are standard OpenSSH and ufw practice and are not tested here on this guide's own system. The statements about sshd_config and ufw come from the Ubuntu 24.04 manual pages (read 2026-10-02).

We tested both Vulkan drivers via llama-bench. Results depend heavily on the llama.cpp build version:

With kyuz0 containers (b8298):

With newer tested llama.cpp (b8460) -- AMDVLK advantage is gone:

AMDVLK is discontinued. Uninstall it -- even inactive, its ICD file silently hijacks Vulkan and halves your pp speed. See AMDVLK warning above.

Our recommendation: Use RADV. AMDVLK is discontinued (last release April 2025). In the recorded newer Vulkan driver comparison, RADV won both pp and tg. AMDVLK's reported 2 GiB buffer limit is a possible contributor to the 3-4X slower pp on the dense-model pair, not an isolated causal result. The measured beginner route uses RADV.

Optimal ubatch sizes per driver (from lhl's testing):

• AMDVLK: -ub 512
• RADV: -ub 1024
• ROCm HIP: -ub 2048

These findings correct several common recommendations found in other Strix Halo guides.

Before changing kernels, ROCm containers, Ollama deployment type, or DeepSeek environment variables, check the scoped ROCm and vLLM bugwatch. It currently tracks:

• one kernel-specific ComfyUI/FLUX load deadlock report;
• a reported ROCm mmap limit above 64GB on one Ubuntu 26.04 setup;
• a container-only Ollama UMA-reporting regression;
• a Windows bare-metal llama.cpp ROCm performance regression;
• a DeepSeek ROCm environment-variable trap where defining a variable as 0 can still enable its path; and
• an older ROCm playbook failure that AMD reports as passing with current pinned wheels.

These are troubleshooting signals, not proof that every Strix Halo system or every version is affected. Match the exact OS, kernel, backend, deployment type, and artifact before applying a workaround.

Fail-Safe's three-system Corsair AI Workstation 300 campaign found that two systems had previously booted without the out-of-tree ec_su_axb35 module after a kernel update, causing dependent custom fan/power services to fail. That is a plausible major contributor to the earlier sustained-load event, not a proven sole root cause and not evidence that every Corsair or Strix Halo system needs a clock cap.

If you use this custom fan path, verify the module and services after kernel updates before sustained inference. The contributor's measured 2400 MHz tradeoff is scoped to the tested fleet. See THERMAL_STABILITY.md for commands, charts, raw evidence, limitations, and the merged upstream fan-reset source change and installed-version qualification gap.

Symptoms on that dated stack: Without the workaround, ROCm containers segfaulted. ggml_cuda_init reported gfx1100 (0x1100) instead of gfx1151.

Dated fix: These variables were used for the measured b8460/kernel 6.19.4 route:

export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0

With this fix, ROCm worked on that measured setup and improved prompt processing versus the older 6.18.14 row. It is historical reproduction metadata, not the current default.

Current ROCm builds with native Strix Halo support should report gfx1151. Before debugging a current container, check the host and container:

printenv HSA_OVERRIDE_GFX_VERSION
llama-cli --list-devices

If an old override is set, remove it from host shell/service startup files and retry with:

env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devices

The Kyuz0 toolbox cross-OEM validation contains the exact Beelink/Ubuntu failure, kernel log, corrected repeats, image digests, and minimal rootless Podman commands.

Symptoms: Qwen3.5 models (35B-A3B and 27B) hang during load_tensors on ROCm. CPU pegs at 99.9%.

Historical status: the cited report is closed. The submitted conservative one-layer attempt also hung; closure does not establish that this guide's exact model/build is now qualified.

The reported --n-gpu-layers 1 attempt is a preserved failure, not a workaround. Match the model, artifact, runtime and logs against a known working route before retrying; do not infer success from newer availability alone.

Symptoms: GPU stays at 900 MHz instead of 2900 MHz, causing ~8% performance loss.

Check:

cat /sys/class/drm/card*/device/pp_dpm_sclk
# Should show: 2: 2900Mhz *

For a deliberate reproduction of the older high-DPM desktop profile only, record the existing state before a temporary change, then restore that exact state after the test. First identify the intended GPU device; do not write through a wildcard on a multi-GPU system.

# Read current state for the GPU you identified; change only as part of a selected profile.
cat /sys/class/drm/card*/device/power_dpm_force_performance_level

Newer kernels (6.18.4+) recognize gfx1151's 1.5X VGPR capacity compared to standard gfx11 chips. This enables better occupancy for compute shaders. If you're on an older kernel, you may not be getting full performance.

Error! Bad return status for module build on kernel: 6.18.14-061814-generic
dkms autoinstall failed for mt76-mt7925(10)

dkms status
lspci -k

# Check whether the model is actually on GPU
ollama ps
journalctl -u ollama --no-pager | grep -iE "vulkan|igpu|gpu" | tail -20

# Use the complete service block in Step 5.2 if this matches the selected route.

# Check current Ollama environment
systemctl show ollama | grep Environment

ollama ps
free -h
journalctl -u ollama -n 50 --no-pager

printenv HSA_OVERRIDE_GFX_VERSION
env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devices

# Check TTM pages limit
cat /sys/module/ttm/parameters/pages_limit

# Check GTT size
cat /sys/module/amdgpu/parameters/gttsize

# Check Vulkan driver
vulkaninfo --summary 2>&1 | grep -E "driverName|driverInfo"

# Check OS-visible RAM
free -h

# Check GPU memory allocation
for file in /sys/class/drm/card*/device/mem_info*; do
  echo "$file: $(cat $file)"
done

# Check status
tuned-adm active

# If not running:
sudo systemctl disable --now power-profiles-daemon
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance

# Verify it persists
tuned-adm active
systemctl is-active power-profiles-daemon

# Check current clock
cat /sys/class/drm/card*/device/pp_dpm_sclk

# Compare under load with the selected campaign's power policy.
# Do not make a temporary high-DPM diagnostic persistent by default.

Historical compatibility snapshot from the March/May test campaign:

As of 2026-08-30, ROCm Core SDK 10.0.0 is released and lists gfx1151 in its official supported-hardware table. Nothing in this guide has installed or qualified ROCm 10.0; this historical matrix and the measured ROCm 7.2/7.14 evidence remain unchanged.

Rules for interpreting this dated matrix:

• Kernel 6.18.4+ changed gfx1151 handling; use current ROCm builds/containers instead of old ROCm RC builds
• The measured b8460/kernel 6.19.4 route used HSA_OVERRIDE_GFX_VERSION=11.5.1
• The documented linux-firmware-20251125 failure (an upstream tag, reported in Fedora packaging) applies to the recorded stack; investigate exact package/runtime combinations, and on Ubuntu 24.04 the linux-firmware-amd-graphics package
• A later date alone does not qualify firmware; preserve known-working package metadata and retest after changes

Current measured recommendation: use a current ROCm build with native gfx1151 support and no global HSA architecture override. Keep the older variables only in commands that reproduce the dated rows. Before publishing benchmark numbers, also verify kernel, ROCm version, Mesa, AMDVLK removal, GPU clock, and tuned status.

Beelink wall-power efficiency is not published yet. powercap is empty on this system, but amdgpu exposes PPT telemetry through power1_average / power1_input. A 2026-05-16 local PPT run measured roughly 111-113 W during Qwen3-Coder/Qwen3.6 Vulkan workloads, but this is GPU/APU telemetry, not wall power.

Community wall-power data does exist from Corsair AI Workstation 300 systems. The current issue #6 cross-section measures about 150 W / 1.6 J/token for Qwen3-Coder, 148 W / 2.0 J/token for Qwen3.6, 174 W / 3.1 J/token for gpt-oss-120b, and 137 W / 3.4 J/token for Qwen3-Coder-Next during sustained generation. Use that as practical community context, not as a Beelink claim.

A separate three-system Corsair thermal/SCLK campaign reports Linux AMDGPU socket-power telemetry from 80.90 W at a 2200 MHz cap to 118.71 W at 2600 MHz, plus 119.54 W mean in bounded stock controls. Those are socket-power readings, not wall power or Beelink measurements. See THERMAL_STABILITY.md.

See POWER_BASELINE.md, COMMUNITY_RESULTS.md#whole-system-power, data/community_power.csv, data/beelink_power_telemetry.csv, and scripts/sample_power.py before adding tokens-per-watt claims.

After completing setup, verify each item:

• free -h shows most of your installed memory, not ~31GB (~124GiB on 128GB systems; lower on 96GB systems)
• vulkaninfo --summary shows the intended RADV driver; September 19 acceptance used Mesa 26.2.3, while historical benchmark rows retain their own driver metadata. Do not upgrade solely to match this checklist.
• Record the selected power policy and verify its active manager; tuned is required only for a tuned reproduction
• Check GPU clocks/utilization under the actual workload against that profile, not an idle universal 2900 MHz threshold
• Compare live GTT/TTM values with the selected RAM/kernel profile; 31457280 pages is the recorded 128GB-system limit, not a 96GB preset
• ollama --version returns without error
• A useful text task succeeds; ollama ps and runtime logs establish actual GPU offload independently of speed
• systemctl show ollama | grep Environment includes OLLAMA_VULKAN=1 and OLLAMA_IGPU_ENABLE=1
• cat /etc/default/grub | grep CMDLINE includes the GTT/TTM parameters; amd_iommu=off appears only if you deliberately selected the optional desktop benchmark profile
• Record the exact kernel/runtime pair; current native gfx1151 ROCm starts without a global HSA override, while dated reproduction commands retain theirs
• Record the exact firmware package and verify the selected runtime after any change; version dates alone do not prove compatibility

• kyuz0/amd-strix-halo-toolboxes -- Community standard containers for llama.cpp (1.2k+ stars)
• kyuz0/amd-strix-halo-vllm-toolboxes -- vLLM serving + RDMA clustering
• kyuz0/amd-strix-halo-gfx1151-toolboxes -- Meta repository with all toolboxes
• kyuz0 Backend Benchmarks Dashboard -- Interactive benchmark comparison
• lhl/strix-halo-testing -- Deep performance research and rocWMMA patches
• nabe2030/hip-vs-vulkan-evo-x2 -- Independent HIP versus Vulkan workload-crossover benchmark on Strix Halo
• hec-ovi/vllm-awq4-qwen -- Experimental Qwen3.6 AWQ/DFlash vLLM path for Strix Halo
• strixhalo.wiki and llm-tracker.info -- historical community references; their content may predate the 2026-07-24 rocWMMA removal and ROCm 7.x/10.0, so do not follow their rocWMMA advice as current guidance. Dates (checked 2026-10-02): the wiki's public mirror repository lists 2025-11-01 as the last change to its llama.cpp performance page and 2026-08-26 (commit message: internal link fixes) for its llama.cpp with ROCm page; the ROCm page opens with a recommendation dated August 2025 and carries its own warning that the rocWMMA path should not be used with ROCm 7.0.2 and later; llm-tracker.info shows the date May 17, 2025 on the page and the same date in its sitemap, without saying that it is an update date
• Level1Techs Forum -- Community benchmark results
• Framework Community -- Framework Desktop discussions
• ROCm Strix Halo Optimization Guide -- Official AMD guide

Use the Strix Halo model hub for a web view that separates models measured by this guide from artifacts verified to exist but not measured here, plus dated 128GB published-size fit tiers. The tables below remain the canonical in-repository setup guidance.

Not sure which model to run? Here's what we recommend based on use case. Recommendations reviewed 2026-08-30; wording corrected September 5. Speeds belong to different measured guide rows, not a matched quality evaluation. Advertised context is not a locally qualified context limit.

How to install any model:

# Via Ollama (easiest)
ollama pull qwen3.6:35b-a3b

# For llama-bench direct (need GGUF file)
# Download from huggingface.co, place in ~/models/

  Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf
  |     |      |   |   |        |  |
  |     |      |   |   |        |  +-- Quantization (see Glossary)
  |     |      |   |   |        +-- "Unsloth Dynamic" quant method
  |     |      |   |   +-- Fine-tuned for instructions
  |     |      |   +-- 3B Active parameters (MoE)
  |     |      +-- 30B Total parameters
  |     +-- Optimized for coding
  +-- Model family (by Alibaba)

Compare the exact task, quality requirement and actual bill. Local throughput does not establish equivalence with a hosted model, and buying hardware does not remove all software, power, maintenance or time costs.

Illustrative token-fee scenario, checked September 13, 2026: Claude Sonnet 5 standard pricing is $2 per million input tokens and $10 per million output tokens. Ten million uncached input tokens plus one million output tokens therefore costs $30 in base token fees. This excludes tools, cache writes/reads, batch discounts, regional premiums and tax. It is not a claim that a local Qwen model performs the same work at the same quality. Anthropic pricing.

For your own scenario:

API cost = input_tokens / 1e6 × input_tariff
         + output_tokens / 1e6 × output_tariff + cache/tools/other fees
Electricity = days × (active_W × active_hours + idle_W × idle_hours) / 1000 × price_per_kWh
Monthly ownership = purchase_cost / assumed_lifetime_months + electricity + maintenance

For example, assumed, unmeasured 120 W active for one hour/day plus 30 W idle for 23 hours/day costs $3.65 over 30 days at $0.15/kWh. At 140 W active it is $3.74. At Eurostat household electricity prices for the second half of 2025 (page checked 2026-09-30: about €0.2896/kWh EU average, about €0.3869/kWh Germany) the same assumed duty cycle costs €7.04 (EU average) or €9.40 (Germany) over 30 days. A continuous 140 W load (an assumption inside the community-reported sustained-generation range of 137.4-173.6 W) is about 1,226 kWh per year, or about €355 (EU average) and €474 (Germany). Tweakers measured desktop idle of 11.6 W (Framework Desktop) and 12.8 W (GMKtec EVO-X2) with CPU-load tests, not LLM inference. See electricity cost with EU tariffs. These duty cycles are explicit assumptions, not inferred from token counts. The dated $4,349 Beelink example amortized over an assumed 60 months adds $72.48 per month before maintenance/resale. Beelink wall power remains unmeasured here; amdgpu PPT is not a wall-meter substitute.

Break-even exists only when verified avoided spending exceeds incremental local costs, and hardware remains useful long enough. No payback period or subscription replacement is established by this guide.

Local inference can keep prompts on the machine after artifacts are provisioned, provided models, embeddings, tools and integrations also run locally. Optional cloud, web-search and remote-tool features change that boundary. Capacity and latency still limit requests, and refusal/filter behavior depends on the selected weights and application. Review Ollama's cloud/offline controls for the exact deployed version.

Ollama exposes API routes that some coding clients can use, but endpoint/schema compatibility does not establish a working client workflow. Choose the exact client version and follow its provider configuration, authentication and context guidance. For Claude Code, use the official Ollama integration recipe, including its documented native/compatible protocol requirements.

The local endpoint is http://127.0.0.1:11434; an OpenAI-compatible client normally uses its /v1 route. Confirm the model ID with the running server. Then verify one useful coding task, any required tool execution, error handling and restart behavior. This guide's direct llama-bench and API smokes do not qualify every Cursor, Continue or Claude Code version, nor measure coding-quality parity with hosted models.

Cursor is not an offline-local guarantee: its BYOK documentation (checked September 19, 2026) says requests pass through Cursor's servers for prompt building and Tab uses Cursor's own models. A localhost URL or your own API key does not establish that those features stay on-device. Verify the specific client/function/network path; do not expose an unauthenticated local model server to make a cloud-mediated client reach it.

Coding and computer-use agents that read repository files, web pages or tool output are exposed to prompt injection as well; see RAG and agents.

For a new setup, use the Linux loopback-only candidate and acceptance checklist. It pins Open WebUI 0.11.4 by registry digest, sets the UI host and port explicitly, and reaches a loopback-only native Ollama without widening its API listener. This candidate has not passed a hardware/client/reboot qualification here; terminal Ollama is the first checkpoint.

Historical, scoped client result: Open WebUI 0.10.2 passed discovery, visible Qwen3.6 output and restart checks against the existing Ollama 0.32.15 route. The measured host was LAN-reachable, not local-only. Keep that qualification as historical evidence; its bridge command is not a fresh local-only installation recipe.

Security status, read 2026-10-03: 47 of the 165 published Open WebUI advisories include 0.10.2 in their affected ranges (15 high, 31 medium, 1 low). The same advisory review found no ranges including 0.11.4; that is not a general safety guarantee. Turning community sharing off addresses one condition and does not patch the old image. Environment settings normally lose to persistent database values; the new candidate uses a separate volume and explicit nonpersistent configuration. The November 9 recheck does include 0.11.4 in a later channel-webhook advisory and adds new Ollama/llama.cpp reports; the older count is historical. See SECURITY.md for the source and scope.

The historical 0.10.2 digest remains in the qualification report. A patched release is not qualified here; priority 40 in the test queue tracks text, image, tool, login, restart, reboot and network acceptance. Use the same UI port in your SSH tunnel.

Privacy depends on models, embeddings, tools and integrations. Provision artifacts first and verify that every enabled component stays local before claiming offline operation; a local UI alone does not prove this.

For querying your own documents locally:

# 1. Pull an embedding model
ollama pull nomic-embed-text

# 2. Use Open WebUI's built-in RAG (easiest)
#    or set up LangChain + ChromaDB for custom pipelines

Documents are untrusted input (checked 2026-09-30). A document, web page or repository file that a RAG assistant or an agent reads can carry instructions that the model may follow (indirect prompt injection, OWASP LLM01:2025). Treat retrieved text as data, not instructions; do not load images or links from answers automatically; run agents in a sandbox without secrets; and have a person approve actions such as file writes, shell commands and network calls. That section summarises public sources; the guide has not tested these controls. See RAG and agents.

For a more current local embedding route, NVIDIA Llama Nemotron Embed 1B v2 now has a first-party CPU sanity pass. It ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector in a fresh offline process. Follow the pinned reproduction and raw evidence. Keep this scoped to local functionality: a real Dutch/English corpus, long documents, batch throughput, memory, and ROCm acceleration still need measurement.

The measured Unsloth Strix Halo path uses an isolated ROCm 7.2 container. It completed a Radeon GPU gate, one-step SFT smoke, checkpoint inference, Q4_K_M export, ROCm llama.cpp inference, and a post-restart load from the host-persisted artifact.

Start there before attempting a longer QLoRA campaign. The smoke proves the workflow, not useful adaptation quality or large-model training performance.

The official Qwen3-ASR 0.6B Q8_0 GGUF now has a first-party llama.cpp b10107 Vulkan/RADV pass. It transcribed a known short English sample as Front, center. and returned the exact transcript after a fresh process. Use the exact command in the reproduction and raw evidence.

This proves a local load/transcription path, not general ASR quality. Dutch/multilingual accuracy, long audio, streaming, real-time factor, the 1.7B tradeoff, and word-error rate remain open; llama.cpp currently labels audio support experimental.

The official LFM2.5-VL 1.6B Q4_0 GGUF plus Q8_0 projector now has a first-party llama.cpp b10107 Vulkan/RADV pass. It correctly read the guide title, AMD Strix Halo platform, 101.0 t/s, 140.4 t/s, and 128 GB from the repository image, then repeated the exact answer in a fresh process. Use the exact command in the reproduction and raw evidence.

This is the low-download, lightweight vision route in the measured profiles. It is one image-functionality check, not a scored screenshot/photo/document benchmark.

This guide has not run image generation. A community ComfyUI toolbox is a ROCm container with bundled ComfyUI workflows for gfx1151. Its README (checked 2026-10-02) lists HunyuanVideo 1.5, LTX-2.3, MiniMax-H3, Qwen Image, Qwen Image Edit and Wan 2.2 workflows; Flux is not on that list. For Vulkan-only: the stable-diffusion.cpp README lists a Vulkan backend (checked 2026-10-02); whether it works on gfx1151 with the RADV driver was not checked here. Third-party run times and open problems are in the model hub.

AMD also publishes an official native-Windows ComfyUI route for Windows 11 24H2, current Adrenalin drivers, and ROCm 7.2.1, covering SDXL, Flux, and WAN workflows. That is useful official setup guidance, but this guide has not reproduced it on the Beelink or compared it with the measured Linux paths.

The current model evidence records a narrow Qwen3-TTS English speech generation and ASR back-check. A complete listen-answer-speak workflow, Dutch quality and streaming remain unqualified. Chatterbox and third-party voice applications are not established by that test. Third-party speech routes, none run here, are listed in the model hub.

The MAX+ 395 systems compared here share that APU; other Strix Halo variants must be checked separately. AMD's CES 2026 announcement (2026-01-05) added the Ryzen AI Max+ 392 (12 cores) and 388 (8 cores) with the full 40-CU Radeon 8060S. The later Ryzen AI Max PRO 490/485 use 32-CU Radeon 8050S graphics. None of these is measured here, and 395 results do not qualify them. The web buyer comparison carries the dated source links, the Ryzen AI Max PRO 400 hardware context and a 128GB or 192GB? section. Memory options include 64GB, 96GB, or 128GB LPDDR5X-8000 depending on vendor and variant; the Ryzen AI Max PRO 495 systems add a 192GB option (most vendors state LPDDR5X-8533; neither that speed nor the 192GB setup is measured or qualified here). The differentiators are memory size, form factor, cooling, ports, support, stock status, price, and how much public evidence exists for the exact chassis.

Newest storefront snapshot: observations of September 30, 2026. The exact-SKU snapshot (rows also in data/buyer_price_snapshot_2026-09-30.csv) records, with country, currency and tax status per row: US store prices in USD, EU manufacturer-store prices with seller terms as stated on the pages, Dutch and German retailer listings, and the first Ryzen AI Max PRO 495 (192GB) systems. For example, Framework's NL store listed a 192GB Desktop DIY Edition at €7,659 (VAT included per the store; pre-order, ships in November) and the 128GB DIY edition as out of stock. These are seller listings, not delivered checkout quotes.

Earlier storefront snapshot: September 19, 2026. Use the exact-SKU snapshot for source URLs, variant identifiers, stock/ETA, seller terms and unresolved checkout fields. US-facing/USD examples: GMKtec EVO-X2 128GB/2TB $3,649.99 (64GB/1TB is a different $2,199.99 offer), Beelink GTR9 Pro 128GB/2TB $4,349 pre-sale, Bosgame M5 128GB/2TB $2,999, Minisforum MS-S1 MAX 128GB/2TB $3,799 with early-October shipping, and Nimo 128GB/2TB $3,899.99 with delivery ETA unresolved. These are not delivered checkout quotes.

In the September 19 snapshot, Framework's 128GB mainboard was not a complete PC and its quote was unresolved (the Framework NL store prices are in the September 30 snapshot). Corsair's exact 128GB/4TB SKU was out of stock with price unresolved. HP's selected laptop quote was unresolved on that date. No older price was silently refreshed. GMKtec EVO-X3 uses MAX+ 395 in the observed offers; EVO-X2 evidence does not automatically qualify EVO-X3.

The July CSV and September 13 snapshot remain historical. The first Ryzen AI Max PRO 400 Series (announced by AMD on 2026-05-20) systems with 192GB opened for order between 2026-09-28 and 2026-09-30, with shipping from mid-October to November (dated table: PRO 400 systems, checked 2026-09-30); none is measured here, and none inherits 128GB results. AMD calls Micro Center the "first global launch partner" for its Ryzen AI Halo reference platform (AMD post of 2026-07-06); no Micro Center price is used here. A footnote on AMD's Ryzen AI Halo product page (checked 2026-09-30) states a retail price of $3,999 (USD; testing as of May 2026); that is a price AMD states in a performance footnote, not a store observation.

Board/NIC revision note (Beelink GTR9 Pro): Some, but not all, original v1.0 systems with Intel E610-XT2 networking have reported NIC recovery/disconnection failures. Beelink has published update guidance and later introduced the v2.2 board with Realtek RTL8127 networking. Because field reports vary by unit and revision, confirm the exact board/NIC version with the seller and contact Beelink support with the serial number when troubleshooting. See the Beelink forum thread and Beelink's Q1 2026 BIOS summary. Independent press (ComputerBase, 2025-12-03, before the v2.2 board) reported reproducible system crashes under a local LLM run via the Intel E610 LAN ports; the ports worked again only after the power cable was removed.

How to compare the systems: compare memory configuration, dated delivered price/availability, evidence depth, cooling/thermals, firmware/support, ports, expandability and workload fit. Beelink's deeper first-party evidence is not a universal value win; Framework's modularity is not measured support superiority; network ports alone do not qualify a cluster. Affiliate commission is not a selection criterion.

Important: many Chinese mini PCs in this class, including Bosgame, GMKtec, and Beelink, appear to use closely related Sixunited platform designs. Do not assume every config is literally identical, but the first native GMKtec EVO-X2 community run reproduced the guide's Qwen3.6 Vulkan/RADV row within about 2%. Pick based on price, memory size, ports, cooling, and support.

Model fit and setup effort depend on configuration and workload on all three routes; neither a fixed usable-memory allowance nor lower setup effort is established here.

Linux is strongly recommended for Strix Halo LLM work because it is the path with the strongest native Vulkan/RADV evidence. Windows is now represented by a community MS-S1-Max LM Studio report: Qwen3.6 Q4_K_M through LM Studio measured a 89.49 tok/s script average across mixed prompts, with long 512-token prompt rows around 69-70 tok/s. That is useful Windows serving/API evidence, not a same-shape comparison against native Linux llama-bench. One GMKtec EVO-X2 community report also measured WSL2/HIP at 44.05 t/s on a TG512 Qwen3.6 generation-only run, while the same contributor's native Ubuntu Vulkan/RADV run measured 61.52 t/s on the guide's TG128 shape. Treat both as useful Windows-path baselines, not a clean same-machine Windows-vs-Linux conclusion.

New to local LLMs? Here's what the technical terms mean.

# Start llama-server with your model (OpenAI-compatible API on port 8080)
# From its parent directory, enter the v0.4.1 Vulkan checkout built above.
# Candidate only, not an arbitrary latest binary.
cd llama.cpp-v0.4.1-vulkan
./build/bin/llama-server --help
AMD_VULKAN_ICD=RADV ./build/bin/llama-server \
  -m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
  -ngl 999 -fa on --load-mode auto -c 8192 \
  --cache-ram 0 --no-cache-idle-slots \
  --host 127.0.0.1 --port 8080

• kyuz0 -- Maintainer of the Strix Halo toolbox ecosystem, community standard containers
• lhl -- Deep performance research, rocWMMA patches, IOMMU/bandwidth testing
• pablo-ross -- Original GMKtec EVO-X2 setup guide
• TechnigmaAI / Hardware Corner -- Alternative optimization guide
• AMD -- Trillion-parameter LLM clustering article
• Lychee-Technology -- Pre-built llama.cpp binaries for gfx1151
• kisak-mesa PPA -- Latest Mesa drivers for Ubuntu
• GPUOpen-Drivers/AMDVLK -- Discontinued AMD Vulkan driver; kept here only as context for the ICD hijacking issue

Found something that's wrong, outdated, or missing?

See CONTRIBUTING.md for the full contribution path and the highest-value data currently wanted.

1. Open a benchmark report with hardware, BIOS, kernel, driver, model, backend, command, and raw output.
2. Open a power / efficiency report with wall-power or board-power readings, idle baseline, sample interval, raw readings, and the exact benchmark command.
3. Use Discussions for setup questions, early results, and comparisons.
4. PRs welcome -- especially from other Strix Halo systems (Framework, GMKtec, HP ZBook).
5. If you find a new optimization, include before/after benchmarks.

This guide stays free and evidence-first. If it saved you setup time or helped you choose hardware/software, the most useful support is a GitHub star, a benchmark report, a correction, or a pull request.

Financial support may fund hardware, storage, model downloads, testing time, and ongoing maintenance, but it does not influence benchmark conclusions. See SUPPORT.md for the funding policy.

For private setup help, an independent reproduction, or a scoped OEM buyer-path pilot, see SERVICES.md. The public guide remains free, and paid work does not buy positive conclusions.

• Qwen3.8 matched control recorded everywhere: the 20.42 t/s result is an Ollama-default MTP result. The matched no-draft control on Ollama 0.32.15 (2026-09-26) measured 12.89 t/s without drafting versus 22.71 t/s with default MTP; the model pages, profile and claim indexes, the route matrix and the share text now give the measured result instead of "queued", and EVIDENCE_CORRECTIONS.md records the corrected label. data/mtp_speculative.csv gained the Ollama API rows and data/benchmarks.csv gained speculation and evidence_class columns. The 2026-09-26 entry below keeps its original wording.
• Labels: the 2026-09-26 b11146 re-check is renamed "controlled re-check (not a fully idle host)"; "latest control" labels carry dates; routine scout rows for Nemotron 3.5 Lightning, Laguna XS 2.1 and Muse Glimmer 30B are in CURRENT_MODELS.md; an artifact is identified by its model-blob digest next to the manifest ID.
• Dated upstream state (2026-09-30): bugwatch recheck (Ollama 0.35.0 and 0.35.1-rc0, marked pre-release on 2026-09-30, with 0.35.0 listed as the latest release by 2026-10-01 and unqualified; llama.cpp b11265, Lemonade v2026.40.0 stable, an amdgpu kernel fix, the llama-server prompt-cache report, ROCm-runner carry-over reports, AMD playbook CI failures, AMD security bulletins), new rows in data/current_test_queue.csv, and a raw-folder index for 2026-08-15 to 2026-09-26 in data/README.md.
• Security: new SECURE_LOCAL_AI.md checklist (network exposure, shared servers, community recipes, containers, clusters, SSH, RAG and agents, firmware) and a dated advisory status in SECURITY.md. The setup commands gained key-login, firewall, prompt-cache and container notes; steps that are not tested here say so.
• Buyer pages: a storefront snapshot of 2026-09-30 with country, currency and tax status per row, the first Ryzen AI Max PRO 495 (192GB) listings (none measured here), a 128GB-or-192GB section and European buying notes in the web buyer comparison.
• New pages: ENGINES.md, WINDOWS_START.md and LAPTOPS_AND_HOMELAB.md collect community and vendor sources and label what this guide did not verify; docs/models.md gained a capability-versus-fit table and an other-workloads section.
• Vendor pages and reuse: How We Work With Vendors describes the correction route; TRACTION.md lists where the data is reused; a new README section asks readers to keep each headline's qualification when citing it. Issue and pull-request templates and the CI workflows were updated, and the tests are documented as needing Linux or WSL.
• Review record: ACTIVE_EVIDENCE_REVIEW_2026-10-01.md records what was checked on 2026-09-30 and what that date does not certify.

• Qwen3.8 route relabelled, numbers unchanged: the 292.49/20.42 t/s Ollama result is now labelled as an Ollama API result with Ollama-default MTP drafting (draft_num_predict 4), not a no-draft result; a matched no-draft control is queued. qwen3.8:27b requires Ollama 0.32.12 or later. The qwen3.6:35b-a3b beginner row is tied to its measured manifest because the tag now resolves to an MTP build.
• Setup safety: setup.sh stops on non-24.04 systems (explicit override) and above the 128GB class; the one-liner is documented as unpinned main; firmware checks target linux-firmware-amd-graphics; the recorded boot profile, including amdgpu.cwsr_enable=0, is quoted identically everywhere; an undo section was added.
• Security: SECURITY.md lists advisories affecting pinned Ollama and Open WebUI versions, a lookalike-repository warning, the canonical site, and a private route for script vulnerabilities.
• Dated upstream state (2026-09-25): runtime availability, ROCm 10 vendor scope, new models and dates, and a re-triaged data/current_test_queue.csv. Availability does not upgrade any measurement.
• Buyer and vendor pages: announced PRO 400-series hardware (not measured), dated Apple/NVIDIA context, EU prices, daily-use links, per-OEM coverage, a new dated star snapshot and a private-first contact route.

• Active recommendation surfaces reviewed: ACTIVE_EVIDENCE_REVIEW_2026-09-19.md records what was reviewed and which boundaries apply. It is not a remeasurement date for historical benchmarks.
• Existing-user and isolated runtime qualification: RUNTIME_QUALIFICATION_2026-09-19.md separates the existing Ollama 0.32.15 service/client pass, the useful-but-not-default isolated 0.34.2 candidate, and llama.cpp v0.4.1 direct/server/HIP controls. Official Qwen3.8 on the candidate and a new full-host reboot remain unqualified.
• Buyer and vendor evidence refreshed: a new storefront snapshot and a vendor proof summary were added; the September 13 snapshot became historical.
• Canonical publication recorded: active guidance and canonical publication sources were synchronized, and the verified deployment was recorded in data/public_state.json without refreshing the evidence date.

• Setup and scripts repaired: existing configuration and raw evidence were preserved while setup and measurement scripts were corrected; offline regression tests (python3 -m unittest discover -s tests) were added and run in CI next to the validator.
• Evidence corrections recorded: EVIDENCE_CORRECTIONS.md corrects or qualifies authored interpretation while retaining original submissions and raw measurements; measurement semantics and provenance were corrected across benchmark, community and data docs.
• Dated snapshots refreshed: upstream state and a buyer snapshot were refreshed and public claims qualified.

• Issue #4 follow-up imported: the GMKtec native contributor's four-model follow-up was added to community results, MTP notes, current-model guidance, contributor credit and the system matrix, with the submitted CSVs preserved under data/raw/2026-08-26/community-gmktec-issue4/. It stays community evidence, separate from first-party rows.

• August 30 evidence integrated: the b10687 sentinel and first Qwen3.8-Flash-Next rows were reviewed into structured claims, benchmarks and the model hub, with setup and model claims corrected.
• Buyer comparisons clarified: the buyer page now separates memory fit and dated price evidence more clearly.
• Guide mirrors point to project-domain pages: mirrored web pages now point to the verified strixhaloguide.com routes.

• Auditable coverage split: the evidence count is documented as described owner systems plus independently attributable external sources, with a per-row identity basis, and the validator enforces it.
• Runtime targets refreshed: Ollama 0.33.2 and llama.cpp b10687 became current checked, still-unqualified targets; measured claims stayed pinned to their tested versions.
• Stale claims fixed: the retracted rocWMMA long-context row was removed after upstream removed the rocWMMA FlashAttention kernel; ROCm 10.0 and Lemonade 11.8.0 were recorded as checked but unqualified; buyer pricing was refreshed with dated spot-checks.
• New web pages and title rule: troubleshooting and model-hub pages were published with validator coverage, and the maintainer title convention ("software engineer") was documented.
• New raw sentinel lane: a b10687 current-stack sentinel and a first Qwen3.8-Flash-Next measurement were added as raw evidence, labelled as not comparable with the pinned headline controls.

• Current Qwen3.8 decision layer published: the guide now separates its measured official Ollama route from external 262K-class validation, stock community MTP, tuned ROCm/DFlash reports, and an unpublished-sidecar performance lead. A matched route ladder is the next performance test; community values are not promoted into guide-owned headlines.
• Selected public-state checks: the validator checks evidence-review age, required text fragments on named local pages, coverage arithmetic and declared affiliate-registry state. It does not prove all numbers or prose are synchronized, absence of undisclosed links, rendered website correctness or deployed-site freshness. Scheduled validation is not a substitute for claim/source review.
• Buyer, contributor, and vendor routes tightened: the protected SEO/hook introduction now leads directly into audience-specific decisions; a dedicated Qwen web page and share card, current responsible-share copy, current traction evidence, contribution guardrails, and explicit affiliate-ranking firewall remove ambiguity without turning the README into an advertisement.
• mottledMantis' Gemma row fully imported: preserved the issue CSV and provenance, added the stock b9235 GMKtec result (1209.08 pp512 / 53.02 tg128) to community data, benchmarks, current-model guidance, contributor credit, and the system matrix. Decode is in the same practical band as the newer 54.18-55.45 t/s Beelink controls; unmatched model bytes/hash, build, Mesa, mmap, repeats, and host state prevent an OEM ranking.
• Qwen3.8 evidence split sharpened: the measured Ollama 0.32.13 Beelink route remains the buyer baseline through 50,059 prompt tokens. A corrected 96GB GMKtec public package adds 13/13 exact retrieval up to a 261,130-token needle and a 6/6 image pilot, proving 262K-class operation can work while remaining an advanced patched-HIP route.
• HIP correctness warning corrected: the earlier b10046 result is now scoped to small-model full-UMA allocation evidence. Open llama.cpp issue #26209 and PR #25863 make a stock b10622-versus-fix exact-output A/B the highest-priority runtime test before long-context, vision, or multi-slot HIP guidance is promoted.
• Runtime targets refreshed: llama.cpp v0.3.0 / b10622 and Ollama 0.32.15 are current checked releases, not locally qualified replacements. The Ollama qualification now includes the reported Qwen3.8 Anthropic-compatible xhigh failure mode.
• Community map expanded to 13 systems/sources and 10 contributors: the Kyanite Labs GMKtec evidence package is counted separately from first-party and other GMKtec owner rows.

• Community map expanded to 12 systems/sources and 9 contributors: Ornery_Specialist_83 independently reproduced the first-party Beelink Qwen3.6 ~62-63 t/s figure on a second GTR9 Pro unit and documented the silent llama.cpp -ub > -b clamp (+28.9% pp512 after correction) plus a measured stock-Mesa-to-kisak uplift (+16.9% pp512). Added to the rules of thumb, troubleshooting, community results, and the evidence matrix.

• Official storefronts rechecked: the buyer table now records the displayed configuration, price, and availability state for GMKtec, Bosgame, Framework, Beelink, Corsair, Minisforum, Nimo, and HP. GMKtec's official EU listing is recorded separately because its 128GB price and availability differ materially from the US listing. The configuration-level source, price, and fulfillment state are preserved in data/buyer_price_snapshot_2026-07-27.csv. The dated snapshot is deliberately not a tax-, shipping-, coupon-, or regional-cart quote.
• Material changes reflected: Beelink's listed 128GB+2TB price is now $4,349 and remains pre-sale with a stated 35-day ship window; Corsair's listed 128GB/4TB price is $3,399.99 and currently out of stock; and Minisforum's listed 128GB+2TB Max AI Compute Edition price is $3,639 with mid-August estimated shipping. The current Beelink value is also used in the hardware comparison and break-even examples.

• Ollama 0.32.3 isolated qualification passed: same-cache Qwen3.6 measured 73.13 t/s versus 73.20 t/s on 0.31.2 with exact outputs. Qwen2.5-VL fully offloaded to the iGPU, read the test image correctly, and repeated after process restart. The installed 0.31.2 service remains the default until current 0.32.5 passes a normal package upgrade and full-host reboot.
• Official llama.cpp b10107 capability routes added: LFM2.5-VL 1.6B correctly read the guide image and Qwen3-ASR 0.6B transcribed a short known English sample. Both reproduced exactly after a fresh process. These are scoped function/restart smokes, not broad vision or speech-quality benchmarks.
• Modern local embedding route added: Llama Nemotron Embed 1B v2 ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector offline. The measured route used CPU and remains a tiny retrieval sanity check rather than multilingual-corpus or ROCm evidence.
• PR #25666 independently checked: the Qwen3.6 MTP stock-versus-PR A/B preserved exact generated outputs and 80.187% draft acceptance. Warm repeats were only 0.55% faster on the PR head, so the result is recorded as correctness/no-regression evidence rather than a performance headline.

• Strict three-system campaign imported: Fail-Safe's matched Corsair AI Workstation 300 sweep now has normalized CSV rows, two generated charts, a complete raw bundle, analyzer, contamination checks, cap/reset harness, and bounded stock controls.
• Scoped buyer guidance: 2400 MHz was the best measured conservative tradeoff on this fleet; moving to 2600 MHz added 6.39% prompt throughput and 0.87% generation while mean AMDGPU socket power increased 21.70%. This is not a universal cap recommendation.
• Root-cause framing corrected: historical logs showed missing ec_su_axb35 modules and failed dependent services after kernel updates on two systems. The guide records this as a plausible major confounder, keeps the root cause unresolved, and tracks the then-open fan-reset patch (merged August 20; installed-version qualification remains separate).

• Current direct capacity: DeepSeek V4 Flash 284B UD-IQ2_XXS loaded and generated from a pinned 90.86GB ordinary GGUF on official llama.cpp b10034 at 155.64 pp512 / 13.27 tg128. A deterministic smoke answered correctly; this remains low-bit capacity/current-model evidence rather than a speed or broad quality recommendation.
• Current agent/server route: Step 3.7 Flash 198B-A11B ROCmFPX Q3 plus its Q8 MTP draft measured 34.50 t/s at 4K and 33.83 t/s at 16K, passed a native tool call, and allocated 256K context on one 128GB Beelink.
• ROCm 7.14 workaround reproduced: in the pinned official ROCm 7.14 / PyTorch 2.11 / vLLM image, enabling hipBLASLt improved Qwen3-0.6B FP16 aggregate throughput by 40.50% / 38.96% / 41.54% at concurrency 8/9/16. Concurrency 1 was unchanged and 4 was slightly slower, so this is a batch-8+ server profile rather than a universal tuning rule.
• HIP integrated-host-buffer allocation reproduced (historical scope): official llama.cpp b10046 detected the full Strix Halo UMA pool and allocated ROCm_Host model/output/compute buffers without a gfx-version override. The 2026-08-25 recheck limits this to small-model allocation/setup evidence because later upstream long-context/multimodal correctness reports were outside this smoke's coverage.
• Current runtime checks: official b10034 still showed the Vulkan MoE 8-to-9 sequence cliff on two model shapes; a same-cache Ollama 0.31.1/0.31.2/0.32.0 comparison found no material version regression; current Nemotron Omni, Cascade, AgentWorld, Audex, and CHADROCK routes were also rechecked with scoped evidence.

• Official b9888 Vulkan sentinel checked: Qwen3-Coder 30B-A3B Q4_K_S measured 1404.73 pp512 / 98.12 tg128 r50 on b9888 with the b9851/b9859-matching command shape, and 98.59 t/s generation-only. UD-Q4_K_XL measured 1410.82 pp512 / 96.53 tg128 r5.
• Claim boundary preserved: b9888 is the latest official llama.cpp runtime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline.
• DeepSeek V4 Flash route triaged again: current routes include very large ordinary GGUF artifacts, a 92.8GiB IQ2_M GGUF candidate whose download was too slow to complete in this pass, and a smaller 46.98GiB REAP route that needs a separate ds4 runtime. No local pass/fail or speed claim is published yet.

• Ollama easy-path sanity check updated: a user-local Ollama 0.31.1 binary on port 11435 measured 71.82 t/s warm API generation mean on qwen3.6:35b-a3b, with a 71.62-72.05 t/s warm range after one cold run. This is Ollama API / buyer-path evidence, not direct llama-bench.
• Important setup fix: current Ollama builds can drop the Strix Halo iGPU during GPU discovery unless OLLAMA_IGPU_ENABLE=1 is set. The setup script and README environment guidance now include that variable.
• Official b9859 Vulkan sentinel checked: Qwen3-Coder 30B-A3B Q4_K_S measured 1413.38 pp512 / 98.48 tg128 r50 on b9859 with the b9851-matching command shape, and 99.09 t/s generation-only. UD-Q4_K_XL measured 1411.76 pp512 / 97.01 tg128 r5; Gemma 4 26B-A4B UD-Q4_K_M measured 1323.39 pp512 / 54.18 tg128 r5.
• Claim boundary preserved: b9859 is a useful current-runtime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline.

• Fail-Safe added a Corsair MiMo V2.5 capacity row: issue #26 reports MiMo-V2.5 UD-IQ2_M, described by llama-bench as mimo2 310B.A15B IQ2_M - 2.7 bpw, on Corsair AI Workstation 300 ai-2 with Fedora 44, kernel 7.0.12, Mesa RADV 25.3.6 inside the kyuz0 Vulkan container, and IOMMU off.
• Claim boundary preserved: the pasted CSV row is 30.65 pp512 with n_gen=0, so this guide treats it as prompt-processing capacity evidence, not as a tg128/generation headline.
• Vendor/buyer value: the row includes wall-power/GPU telemetry: Home Assistant wall power 29.7-162.4 W with a 114.14 W mean, GPU edge 40-75 C, and GPU socket power 14.098-104.021 W with a 70.92 W mean. This strengthens the Corsair evidence layer beyond Qwen speed rows.

• Qwen3-Coder direct speed-first row crossed 100 t/s: the exact Q4_K_S Qwen3-Coder 30B-A3B file measured 1423.05 pp512 / 100.99 tg128 r50 on the official llama.cpp b9851 Ubuntu Vulkan release binary with explicit -dev Vulkan0. This is direct llama-bench evidence, not MTP/server speculation. The older b9179 strict-clean 98.51 t/s row remains preserved as historical context.

• Balanced Qwen3-Coder latest-build control improved: UD-Q4_K_XL measured 1416.79 pp512 / 99.55 tg128 r5 on b9851. Treat this as a short latest-build control unless a longer repeat is needed.

• Gemma direct control stayed secondary: Gemma 4 26B-A4B UD-Q4_K_M measured 1326.52 pp512 / 55.45 tg128 r5 on b9851, so the existing Gemma 4 26B-A4B QAT MTP route remains the useful high-throughput Gemma path.

• Community map expanded to 11 systems/sources: papagenic contributed a Minix Elite ER939 Ai Ollama 0.30.10 report for qwen3.6:35b-a3b on Ubuntu 26.04, kernel 7.0.0-22, Mesa 26.1.3, BIOS UMA 1G, and IOMMU disabled.

• Claim boundary preserved: the row is useful buyer-path evidence for Minix/Ollama/Ubuntu 26.04, but it is not a direct llama-bench headline because backend/Vulkan ICD, script details, repeats, and warm/cold state are not yet confirmed.

• Public traction snapshot refreshed: the guide reached 192 GitHub stars, 9 forks, and 4 watchers in the 2026-07-02 GitHub API snapshot. This remains a small-niche demand signal; the main value is still reproducible public evidence that reduces buyer setup friction.

• Runtime watchlist refreshed: the llama.cpp b9851 sentinel is now measured; the later 2026-07-02 check separately measured llama.cpp b9859 and a user-local Ollama 0.31.1 sanity path.

• Public traction snapshot refreshed: the guide reached 160+ GitHub stars and 10 Strix Halo-class systems represented in the 2026-06-21 snapshot. This is a small-niche demand signal; the main vendor value remains reproducible public evidence that reduces buyer setup friction.
• ROCmFP4 / CHADROCK advanced lane added: ROCMFP4_CHADROCK.md tracks the rocmfp4-llama / tuned-GGUF route separately from the beginner Vulkan/RADV/Ollama path and from first-party direct llama-bench headline rows.
• ROCmFP4 Crown Halo smoke added: the jcbtc/qwen3.6-35b-a3b-crown-halo-mtp-dynamic artifact now has first-party Beelink load/API/MTP smoke evidence. It runs, but the high-speed community dynamic-MTP band is not yet reproduced on this HIP-only Beelink route.
• CHADROCK ACE/SABER stability profile added: the jcbtc/chadrock-35b-ace-saber-rocmfp4-mtp artifact loads and serves on the guide's Beelink/RADV path with the pinned ciru-ai/ROCmFPX runner. The exact 3946-token reference profile averaged 141.37 t/s over three gen512 repeats at 100% draft acceptance; 1K/8K/16K profiles measured 78.00/83.85/107.23 t/s. This is strong advanced-lane evidence and practical acceptance guidance, but still separate from direct llama-bench rows.
• Nemotron 3 Nano Omni b9747 smoke added: unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF MXFP4_MOE loads and runs directly on the official llama.cpp b9747 Vulkan binary at 1277.60 pp512 / 56.56 tg128. This is current NVIDIA Omni/FP4 support evidence, not a new speed headline.
• Current model watchlist refreshed: Kimi-K2.7-Code, GLM-5.2, MiniMax-M3, Nemotron 3 Nano Omni NVFP4/MXFP4, DeepSeek V4 Flash REAP, llama.cpp b9747, and Ollama 0.30.10 were triaged for practical guide value.

• GMKtec EVO-X2 NixOS evidence package linked: ciru-ai published a public artifact for GMKtec EVO-X2 / NixOS / IOMMU-on / NPU-aware Strix Halo benchmarking, including sanitized CSV/SQLite exports and a compact public metrics index.
• NPU sidecar evidence added: the artifact reports +3.29% main 64k iGPU workload latency with concurrent NPU load versus +68.96% with a comparable iGPU auxiliary load, plus FastFlowLM-NPU LFM2.5 1.2B at 32k around 1646 prompt tok/s and 38.18 decode tok/s.
• ROCmFP4 and quality-eval context added: selected Chadrock/Qwopus/Qwen3.6/Gemma/CrownV7 rows are linked as community tuned-route evidence with quality metrics. These are not first-party Beelink direct llama-bench headlines.

• Gemma 4 26B-A4B QAT MTP repeat documented: the matched-head llama-server route measured 102.69 t/s cold, 107.42 t/s with only T3 left among known local services, and 110.00 t/s as the best repeat on ac4cddeb0. This is server/speculative evidence, not a direct llama-bench replacement.
• Current setup routing improved: the README now sends new users to setup.sh first, then Ollama/Open WebUI for local chat, and only then to direct llama.cpp, MTP, Lemonade, or ROCm/vLLM paths when they need benchmark or server control.
• Community Beelink CachyOS ROCm/ZenDNN row added: devoidfury contributed a second Beelink owner stack with CachyOS, kernel 7.0.11, ROCm 7.2.4, local ZenDNN, and Qwen3.6 27B MTP UD-Q6_K_XL. ROCm roughly doubled pp5000 versus Vulkan on that setup while decode stayed around 8 t/s, making it backend-crossover evidence rather than a decode headline.

• Latest-control rows added: Qwen3-30B-A3B-Instruct-2507 stayed above 100 t/s at 100.38 t/s, LFM2.5 stayed in the 170 t/s class at 171.17 t/s, and Nemotron 3 Super stayed in the 18 t/s capacity class at 18.24 t/s.
• Gemma 4 QAT direct route added: Gemma 4 26B-A4B IT QAT UD-Q4_K_XL measured 74.80 t/s direct and became the practical current Google-model row, replacing the older non-QAT Gemma 4 row as the recommendation.
• Negative/control evidence preserved: Qwen3.6 27B NVFP4 loaded but was not a speed route on this Vulkan/RADV path, and the MTP smoke result remained too slow to recommend.

• No Vulkan/RADV sentinel regression found: Qwen3-30B-A3B-Instruct-2507 measured 103.18 t/s, the exact Qwen3-Coder Q4_K_S speed-first file reproduced the 98 t/s class at 98.02 t/s r50 and 98.49 t/s generation-only, and Qwen3-Coder UD-Q4_K_XL stayed in the 96-97 t/s class.
• Small-MoE and capacity rows held up: LFM2.5 measured 176.48 t/s, and Nemotron 3 Super measured 18.93 t/s on the b9544 control.

• New speed and capacity scouts added: LFM2.5 8B-A1B reached 170.02 t/s generation-only, Nemotron 3 Nano reached 75.97 t/s, and Nemotron 3 Super 120B-A12B reached 18.43 t/s direct on one 128GB Strix Halo system.
• Failure data kept: DeepSeek V4 Flash and other large-model routes stayed documented as blocked or impractical where model distribution, architecture support, or storage/runtime requirements prevented a clean local benchmark.

• Nimo AI Mini PC evidence added: boxwrench contributed a Ryzen AI MAX+ 395 / Radeon 8060S / 128GB Nimo bundle in issue #4 with system metadata, reproducibility notes, raw benchmark rows, thermal telemetry, and model-specific follow-ups.
• Community map now covers 8 systems: the guide now tracks Beelink first-party data plus three Corsair systems, two GMKtec sources, MS-S1-Max, and Nimo community evidence. This was later expanded to 11 systems/sources with a second Beelink owner stack, ciru-ai's GMKtec EVO-X2 NixOS/NPU artifact, and papagenic's Minix Elite ER939 Ai Ollama report.
• Large-model buyer context added: Nimo rows cover Qwen 3.5/3.6 35B, Qwen 122B-class serving, Qwen3-Coder-Next, StepFun Step-3.7-Flash, GPT-OSS/Gemma notes in the raw bundle, and DFlash negative/control evidence. These are community serving/eval rows, not first-party direct llama-bench headline claims.
• Gemma 4 QAT follow-up added: boxwrench added Gemma 4 12B, 26B-A4B, and 31B QAT Q4_0 rows with matched MTP assistant-head comparisons. The useful lesson is not a homepage headline; it is that matched QAT assistant heads can materially improve single-stream decode and acceptance. Atomic PR #26 later fixed the reported PARALLEL=2 Gemma 4 MTP crash, so fresh post-merge 2-slot community numbers are now the useful follow-up.
• Vendor/adoption value improved: COMMUNITY_NIMO.md summarizes what the Nimo bundle proves and what it does not prove, so vendors/reviewers can see how additional hardware reduces setup and buyer uncertainty without turning community data into endorsement language.

• First local direct 100 t/s row added: Qwen3-30B-A3B-Instruct-2507 IQ4_XS reached 100.04 t/s tg128 r50 and 1416.03 t/s pp512 on llama.cpp b9467 / Vulkan/RADV. This is a direct llama-bench result, but it is explicitly kept separate from the Qwen3-Coder 98.51 t/s headline and the Qwen3.6 MTP server route.
• Windows LM Studio evidence added: bennos1911 contributed a Minisforum MS-S1-Max Windows 11 / LM Studio 0.4.15 / Qwen3.6 Q4_K_M serving report with benchmark script, CSV output, and hardware telemetry. It is documented as Windows-path evidence, not as a same-shape Linux llama-bench comparison.
• Tuned GMKtec 100 t/s report added: Look_Over_There contributed a Reddit GMKtec EVO-X2 Qwen3-Coder Q4_K_S b9467 report where most short-context runs were around 99.90 t/s and the best observed run reached 100.0 t/s after about 10 runs. It is explicitly labeled as tuned thermal/power-policy evidence because the system had heatsink repaste, memory-pad reseating, lower reported temperatures, and GPU/CPU high-performance policy.
• Community signal updated: community results now include Beelink first-party data plus Corsair, GMKtec, and MS-S1-Max community evidence across Linux Vulkan/RADV, WSL2/HIP, Windows LM Studio, wall-power, RPC, USB4 tuning, and tuned thermal/power-policy reports.

• Upstream watch rechecked: ROCm production remains 7.2.4, vLLM has moved to 0.22.1, and the previous isolated Ollama 0.24.0 check did not change the then-installed Ollama 0.23.1 guidance at that time.
• No new headline from latest llama.cpp direct reruns: the 2026-06-01 de6f727aa Qwen3-Coder direct check measured 95.55 t/s tg128 with mmap=0, so the direct headline stays at 98.51 t/s on the b9179 strict-clean speed-first row.
• Qwen3.6 27B MTP control confirmed: the latest-build rerun measured 7.61 t/s without MTP and 14.69 t/s with MTP, so the official dense 27B Q8_0 route remains useful negative evidence rather than a speed candidate. Details live in PERFORMANCE_NOTES.md.
• Community hygiene improved: responsible-sharing guidance was added to CONTRIBUTING.md and SHARE.md, and Fail-Safe's .gitignore PR added macOS/Windows cache-file ignores for cleaner community contributions.

• MTP route crossed 100 t/s broad average: llama.cpp b9360 (6b4e4bd58) with Mesa/RADV 26.1.1 pushed Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn to 101.16 t/s best six-prompt average with draft-n=2, --poll 100, and -ub 1024. Three t16 repeats landed at 101.15 / 101.10 / 101.06 t/s.
• At that date, still not a direct 100 t/s claim: b9360 direct Qwen3-Coder Q4_K_S measured 97.23 tg128, and UD-Q4_K_XL measured 92.60 tg128. The Qwen3-Coder direct row remained b9179 Q4_K_S 98.51 t/s and balanced b9049/b9010 96-97 t/s until the later 2026-06-02 guide update added a separate Qwen3-30B-A3B-Instruct-2507 direct 100.04 t/s row.
• Added raw evidence under data/raw/2026-05-27/latest-llamacpp-b9360/ and updated BENCHMARKS.md, MTP_SPECULATIVE_DECODING.md, data/benchmarks.csv, data/mtp_speculative.csv, and data/headline_claims.csv.

• MTP route improved again: llama.cpp b9334 (192d8ae) with Mesa/RADV 26.1.1 raised the local Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 98.57 t/s with draft-n=3; the best b9334 prompt reached 116.75 t/s.
• At that point, still not a broad 100 t/s claim: repeated b9334 MTP runs landed around 97.76-98.57 t/s depending on thread/poll settings. This was close and valuable, but later got superseded by the b9360 -ub 1024 MTP rerun.
• Direct latest-stack check stayed below the headline: llama.cpp b9334 direct Qwen3-Coder Q4_K_S measured 96.27 tg128, and UD-Q4_K_XL measured 94.15 tg128. The existing direct headline remains the b9179 Q4_K_S 98.51 t/s strict-clean row and the balanced b9049/b9010 96-97 t/s UD row.
• Added raw evidence under data/raw/2026-05-26/latest-llamacpp-b9334/ and updated BENCHMARKS.md, MTP_SPECULATIVE_DECODING.md, data/benchmarks.csv, data/mtp_speculative.csv, and data/headline_claims.csv.

• GMKtec Qwen3-Coder full row: mottledMantis added the requested full pp512/tg128 Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 run on GMKtec EVO-X2: 1157.29 pp512 / 91.40 tg128.
• Command-shape caveat preserved: the full row used -b 512 -ub 512, flash_attn=0, and use_mmap=1, so it is documented as portability and flag-sensitivity evidence, not as an apples-to-apples replacement for the Beelink headline row.
• Added the raw CSV under data/raw/2026-05-19/community-gmktec-qwen-coder-issue17/ and updated COMMUNITY_RESULTS.md, CONTRIBUTORS.md, BENCHMARKS.md, SHARE.md, and data/community_results.csv.

• GMKtec MTP reproduction: mottledMantis reproduced the exact Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn route from the guide on a GMKtec EVO-X2 with Ubuntu 26.04, kernel 7.0.0-15, Mesa RADV 26.0.3, and llama.cpp b9235. draft-n=2 averaged 93.29 t/s over six prompts, slightly above the local Beelink b9235 92.30 t/s row. This strengthens the MTP route but still is not a broad 100 t/s claim.
• GMKtec Qwen3-Coder baseline: the same contributor added a Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 generation-only row at 92.11 t/s. This is lower than the Beelink/Corsair b9049 rows, but useful as a GMKtec/latest-stack baseline and as evidence that build/model/source/host-state details matter.
• Added raw community artifacts under data/raw/2026-05-19/community-gmktec-*, updated COMMUNITY_RESULTS.md, data/community_results.csv, and data/mtp_speculative.csv.

• Latest MTP master rerun at the time: llama.cpp b9235 (d14ce3dab) raised the Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 92.30 t/s with draft-n=3. The fastest prompt in this rerun was 109.21 t/s; the older b9187 sweep held the best single-prompt MTP result at 110.61 t/s until the later b9334 rerun.
• Official 27B MTP Q8_0 checked: ggml-org/Qwen3.6-27B-MTP-GGUF was tested on the same b9235 Vulkan/RADV stack. It reached 7.74 t/s without MTP and 14.59 t/s with the best MTP setting, making it a useful negative result rather than a speed/headline candidate.
• Added raw evidence under data/raw/2026-05-19/ and updated the MTP CSV/claim index.

• MTP speculative decoding documented: Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn reached 90.80 t/s average over six llama-server prompts and 110.61 t/s on the best prompt. This is a server/speculative result, not a replacement for the direct llama-bench headline and not a broad 100 t/s average.
• Qwen3-Coder ngram server route checked: Qwen3-Coder Q4_K_S with llama-server ngram speculation reached 95.21 t/s average, below the direct 98.51 t/s speed-first headline.
• Community wall-power cross-section added: Fail-Safe's Corsair AI Workstation 300 power data now covers Qwen3-Coder, Qwen3.6, gpt-oss-120b, and Qwen3-Coder-Next, with sustained generation rows from about 137-174 W and 1.6-3.4 J/token.
• Contribution flow tightened: CONTRIBUTING.md and the benchmark/power issue templates now ask for the metadata needed to turn community reports into structured evidence.
• Added raw evidence under data/raw/2026-05-17/ and data/raw/2026-05-10/community-power-issue6/.

• Qwen3-Next 80B improved: llama.cpp b9172 with Vulkan/RADV confirmed 59.06 t/s tg128 and 751.70 pp512, replacing the old 54.92 t/s b8933 row as the historical May 80B Qwen-family result.
• Qwen3-Coder speed-first peak: b9179 plus Q4_K_S confirmed 98.51 t/s r50 after fixing the tuned/power-profiles-daemon conflict and pausing benchmark noise. This is a speed-first quant row, not the balanced UD default.
• No main headline speedup from b9172: Qwen3-Coder UD, Qwen3.6, and gpt-oss-120b did not beat the current b9049/b9010 balanced headline rows.
• Ollama 0.24.0 isolated check: Qwen3.6 measured 49.05 t/s warm generation, effectively identical to the same-prompt Ollama 0.23.1 control at 49.09 t/s.
• ROCm nuance confirmed again: Lemonade ROCm b1259 won Qwen3-Next pp512 (800.38 vs Vulkan 751.70), but Vulkan/RADV won tg128 (59.06 vs ROCm 49.57). The guide now avoids "RADV wins everything" wording and keeps the beginner rule focused on generation-heavy GGUF chat/coding.
• Added raw evidence under data/raw/2026-05-16/.

• llama.cpp b9049 rerun: Qwen3-Coder 30B-A3B measured 96.76 t/s generation in the max-performance guide-flags confirmation; Qwen3.6 35B-A3B measured 62.56 t/s on the balanced UD row and 81.30 t/s on the speed-first Q4_0 row.
• Ollama upgraded to 0.23.1: Qwen3.6 API warm average remained 50.51 t/s, matching the 0.21.2 baseline.
• gpt-oss-120b local check: ggml-org MXFP4 split GGUF loaded locally and measured 55.57 t/s tg128, 726.99 t/s pp512, and prompt processing through 65K tokens via llama.cpp b9049 Vulkan/RADV.
• HIP/Vulkan workload split added: local spot check shows HIP winning pp16384 and Vulkan winning tg128 on both Qwen3.6 and Qwen3-Coder rows; see BACKEND_CROSSOVER.md.
• ROCm/vLLM bugwatch added: current upstream ROCm/vLLM release and issue status moved to ROCM_VLLM_BUGWATCH.md.
• Headline range tightened: current direct llama.cpp headline range became 63-97 t/s for the balanced rows. The previous b9010 Qwen3-Coder peak of 97.24 t/s remains in the data as historical evidence, and Qwen3.6 added an 81.30 t/s speed-first quant row.
• Added clean raw evidence under data/raw/2026-05-07/latest-stack-rerun/clean-b9049-rerun/.

• Qwen3-Coder 30B-A3B benchmark updated: controlled b9010 Vulkan RADV rerun averaged 97.24 t/s generation and 1346 pp512 across two separate -r 20 runs.
• Qwen3.6 UD rerun: controlled b9010 Vulkan RADV rerun averaged 63.06 t/s generation and 1109 pp512 across two separate -r 20 runs. The old "UD costs 13%" warning was not reproduced on the current stack.
• Ollama Qwen3.6 rerun: controlled API test averaged 50.5 t/s warm generation across 10 runs, replacing the older 45-46 t/s easy-path claim. Current Qwen3.6 Ollama overhead is about 20-21%, not ~30%.
• Multi-user Qwen3.6 serving: llama-server continuous batching reached 162 t/s aggregate at -np 8 with ~0.31 s TTFT, then plateaued at 166 t/s at -np 16.
• Multi-user Qwen3-Coder serving: llama-server continuous batching reached 173 t/s aggregate at -np 8; -np 16 regressed to 130 t/s aggregate.
• Local long-context prompt scaling: Qwen3.6 processed 64K prompts at 740 t/s and Qwen3-Next 80B processed 64K prompts at 544 t/s on Vulkan RADV.
• Filled-KV decode: Qwen3.6 generated 41.4 t/s after a 64K f16 prompt; q4_0 KV raised decode to 51.3 t/s but increased total request time from 73.5 s to 90.0 s because prompt ingest slowed.
• 128K filled-KV decode: Qwen3.6 generated 32.2 t/s after 128K and Qwen3-Next 80B generated 29.1 t/s after 128K, both without truncation.
• Real-corpus 64K check: using this guide's own documentation as the prompt, Qwen3.6 decoded at 40.8 t/s after ~65K tokens and Qwen3-Next 80B at 37.8 t/s after ~64K tokens. Prompt ingest was slower than synthetic prompts, but decode-after-fill barely changed.
• ROCm HIP spot check: current local HIP b8460 path is usable with the HSA override, but remains behind Vulkan for short-context tg: Qwen3.6 52.7 t/s and Qwen3-Coder 73.7 t/s.
• At that point, the headline range moved from 65-87 t/s to 65-97 t/s. This was later tightened to 63-97 t/s for balanced rows in the 2026-05-07 rerun, extended to 63-98.5 t/s by the 2026-05-16 Qwen3-Coder speed-first Q4_K_S row, extended with a separate 100.04 t/s Qwen3-30B-A3B-Instruct-2507 IQ4_XS direct row on 2026-06-02, and later lifted to 101.0 t/s for Qwen3-Coder Q4_K_S on official b9851. The previous 87.11 t/s result remains in data/benchmarks.csv as historical-local data.
• Added raw benchmark output under data/raw/2026-05-03/ so the new headline can be audited.

• Beelink price audit: The official GTR9 Pro page now lists the 128GB+2TB variant at $4,399 with a $4,699 compare-at price. Earlier snapshots and checkout prices were substantially lower, so the guide no longer uses the old lower Beelink figure as a headline claim.
• Removed stale "Strix Halo is much cheaper than DGX Spark" wording. At the current Beelink official price, the gap to DGX Spark is only about $300, while lower-priced Strix Halo systems still exist.
• Updated cost, hardware comparison, buying guide, and Mac Studio FAQ language so price-sensitive claims are clearly date-bound.

• AMDVLK ICD hijacking discovered: All "pp regression" findings (b8460 vs b8933, Mesa 26.0.2 vs 26.0.5) were caused by AMDVLK's /etc/vulkan/icd.d/amd_icd64.json silently overriding RADV. No actual regression exists. Corrected on #22375. All benchmarks re-verified on actual RADV
• Qwen3.6-35B-A3B benchmark: 64 t/s tg, 1064 pp512 via Vulkan RADV. Drop-in replacement for Qwen3.5 with better coding/reasoning quality, identical speed. The old UD-Q4_K_M penalty note is superseded by the May 2026 controlled rerun.
• Qwen3-Next 80B-A3B benchmark: 55 t/s tg, 657 pp512 via Vulkan RADV (b8933). 80B MoE (3B active) with 256K context window. The separate Qwen3-Coder-Next 80B-A3B MoE row measured 38 t/s (architecture label corrected September 5)
• Gemma 4 26B-A4B benchmark: 48.5 t/s tg, 1142 pp512 via Vulkan RADV (b8933). Included a KV-cache warning; September 2026 correction: the external ~3.5× KL ratio compares Gemma 26B with Gemma 31B, not Qwen or downstream task-quality loss.
• Llama 4 Scout 109B benchmark: 18.3 t/s tg, 331 pp512 via Vulkan RADV (b8933). 109B parameter model running on a mini PC; this artifact exceeds 24GB VRAM without offload
• Merged PR #1: vulkan-tools install check in setup.sh (thanks @ignasivt)
• Updated April price snapshot for Beelink, Corsair, and GMKtec; superseded by the May 1 price audit above
• Added linux-firmware-20251125 source attribution and downgrade instructions
• Added Ubuntu 26.04 LTS note (Wayland-only, testing in progress)
• Ollama upgraded to 0.21.2: FA now enabled by default. Original Qwen3.6 via Ollama result was 45.5 t/s; superseded by the May 2026 controlled 50.5 t/s rerun.
• Ollama ROCm confirmed working on gfx1151 with HSA_OVERRIDE_GFX_VERSION=11.5.1 (Ollama 0.20.4). Benchmarked: 42.4 t/s tg vs Vulkan's 46.6 t/s (-9%). Vulkan still recommended for speed

Performance discoveries:

• llama.cpp b8298 to b8460 = +25% tg and +24% pp on MoE models (52 to 65 t/s tg, 868 to 1080 pp512)
  • Key PRs: #19625 (FA refactor), #20551 (graphics queue), #20334 (GDN shader)
  • +25% breaks down as ~14% generic (both backends got this) + ~11% Vulkan-specific
  • Dense models show <2% change (already at bandwidth ceiling)
• RADV now beats AMDVLK on both pp AND tg with latest build (old AMDVLK tg advantage gone)
• Exceeded theoretical tg ceiling: measured 65 t/s vs calculated max of ~57 t/s. The standard formula (bandwidth / active_model_size) underestimates MoE performance because it ignores caching and memory access optimizations in newer llama.cpp builds. The real ceiling is a moving target.
• RADV now beats ROCm on both pp (1080 vs 1047) and tg (65 vs 55) on same b8460 build
• ROCm works on kernel 6.19.4 with HSA_OVERRIDE_GFX_VERSION=11.5.1 + HSA_ENABLE_SDMA=0
• ROCm b8460 got +14% tg from generic improvements (47.87 to 54.67)
• Batch/ubatch sweep: default 512 is optimal, no tuning headroom left

New benchmarks:

• Llama 3.1 70B (4.8 t/s, 94% of theoretical ceiling, doesn't fit on RTX 4090)
• Qwen3-Coder-30B UD-Q4_K_XL (87 t/s tg via RADV, superseded by the May 2026 97 t/s controlled rerun)
• Qwen3-0.6B (266 t/s tg, 13,112 pp512)
• Extended context scaling (pp flat from 512 to 8K, only 3% drop)

Beginner content:

• Ollama vs llama.cpp FAQ with browser analogy and llama-server setup
• Model recommendation guide (10 use cases)
• Cost comparison (local vs cloud with break-even analysis)
• Buying guide (7 systems with a historical March 2026 price snapshot and Beelink board/NIC revision guidance)
• Glossary (20+ terms for beginners)
• FAQ (8 common questions)
• Use cases (Claude Code, Cursor, RAG, image gen, TTS)
• Windows vs Linux comparison

Infrastructure:

• One-command setup script (setup.sh)
• Auto-update script for llama.cpp (update-and-build.sh)
• CONTRIBUTING.md and initial GitHub issue templates; expanded in later May updates for benchmarks, power reports, model requests, bugs, suggestions, and impersonation/security reports
• Historical launch release v1.0.0; current release history is preserved on the GitHub Releases page.
• 19 topics for discoverability
• GitHub stars + last-commit badges

Fixes:

• At that time, prices were verified against current retail (March 2026 snapshot)
• Historical DGX Spark wording called the comparison apples-to-apples; September correction: exact artifact/backend/cache/workload matching was not established (see Hardware Comparison)
• Fixed 12 outdated "ROCm broken on 6.19.x" references
• A low BIOS UMA reserve is mandatory, not just speed-neutral: use 512MB if available, but 2GB is fine when that is the vendor minimum
• Vulkan Driver Comparison updated with b8460 data
• RADV_PERFTEST env vars (cswave32, nogttspill) tested and found to be -10% slower. Don't use.
• Posted findings on llama.cpp Vulkan discussion

• Complete rewrite with live benchmarks on current system
• Added: Kernel 6.19.x ROCm fix (HSA_OVERRIDE_GFX_VERSION=11.5.1)
• Added: Mesa 26.0.2 results (+4-5% tg improvement over 26.0.1)
• Added: qwen3-coder:30b-a3b-q8_0 benchmarks (51.4 t/s -- fastest model in that initial run)
• Added: Long context performance data from lhl (Vulkan vs ROCm at 32K)
• Added: rocWMMA status update (upstream broken, lhl's tuned branch works)
• Added: vLLM setup and known issues
• Added: RDMA clustering section
• Added: Kernel/ROCm compatibility matrix
• Added: linux-firmware-20251125 warning
• Added: LLVM compiler regression workaround
• Added: Qwen3.5 ROCm hang bug (ROCm #6027)
• Added: Backend decision guide
• Added: Testing checklist
• Added: Collapsible troubleshooting sections
• Updated: ROCm HIP works on kernel 6.19.4 with HSA override (even +6% faster pp than 6.18.14)
• Updated: All benchmark numbers re-measured
• Updated: Replaced nano instructions with tee for copy-paste ready commands
• Corrected: rocWMMA is no longer blanket "don't use" -- lhl's tuned branch is best for long context
• Historical benchmark correction: iommu=pt did not improve the measured memory-read result; later buyer guidance separated the faster amd_iommu=off desktop profile from NPU and mobile-suspend requirements

• Basic setup guide based on pablo-ross' GMKtec guide
• Ollama Vulkan configuration
• ROCm container setup

These are the highest-value tests to add next, because they answer practical buyer/setup questions that current evidence only partially covers:

• Lemonade/FastFlowLM NPU on Linux: the May 16 preflight showed amdxdna and /dev/accel/accel0 (NPU firmware version not recorded), but XRT/FastFlowLM are not installed. Next step is a separate NPU lane. Prerequisites per the FastFlowLM Linux guide (checked 2026-09-25): an IOMMU-on boot profile (the measured desktop host boots with amd_iommu=off), the amdxdna driver from kernel 7.0 or later or amdxdna-dkms, NPU firmware 1.1.0.0 or later (check with flm validate), and XRT from ppa:lemonade-team/stable (libxrt-npu2). Then reboot, run flm validate, and test small Qwen/Gemma rows for speed and power.
• Same-machine Windows native Vulkan/Ollama/LM Studio vs Linux: Windows LM Studio and WSL2/HIP now have community baselines, but the most useful beginner answer is still a same-machine native Windows app result versus native Linux Vulkan/RADV with the same model and benchmark shape.
• More GMKtec/Bosgame/Framework native Linux reproductions: The first GMKtec native result matched within about 2%; more vendors turn the guide from one-machine evidence into a platform map.
• DeepSeek V4 Flash current route: the pinned 90.86GB ordinary UD-IQ2_XXS GGUF now loads and generates directly on official b10034 at 13.27 tg128 and passes a basic deterministic check. The smaller 46.98GiB REAP route still needs its separate ds4 runtime; future work should compare quality/runtime tradeoffs rather than repeat the resolved ordinary-GGUF load test.
• Tokens per watt with wall-power data: Fail-Safe supplied valuable Corsair wall-power telemetry, and this guide now has Beelink amdgpu PPT telemetry. A Beelink wall-meter run would make the efficiency story publishable.
• NPU/iGPU telemetry tooling: xdna-top and similar tools could make NPU-sidecar and iGPU contention claims easier to verify, but should be documented as instrumentation until they produce measured model rows.
• Lucebox / DFlash / PFlash: highest-upside experimental route for 27B long-prompt + generation workloads, but the old preflight lacked a developer toolchain; an isolated HIP toolchain now built the v0.4.1 controls, but this does not qualify Lucebox/DFlash; older rocWMMA design notes are historical after the 2026 upstream removal.
• vLLM/AWQ/DFlash throughput: keep this experimental until it has a reproducible OpenAI-compatible server row that competes with llama-server/Ollama for a real use case. Plain AWQ smoke works, but it is not the fastest default.
• Strix Halo successors: AMD announced the Ryzen AI Max PRO 400 Series (formerly Gorgon Halo) on 2026-05-20. The first PRO 495 systems with 192GB opened for order between 2026-09-28 and 2026-09-30, with shipping from mid-October to November (dated table in the buyer comparison, checked 2026-09-30). It is a comparison target, not current setup advice; 192GB configurations are not qualified here and none is measured. Later Medusa Halo / Ryzen AI Max 500 remains a future target (rumored only, not confirmed by AMD).

This project is licensed under the MIT License. See LICENSE.

Found this guide useful? Give it a star on GitHub -- it helps other Strix Halo owners find it. Found something wrong? Open an issue.