Mixture-of-Experts (MoE) inference on CPU is bounded primarily by DRAM bandwidth per generated token. Standard 4.50-bpw scalar quantization (Q4_0) leaves significant memory-bandwidth headroom unused, while conventional 2–3 bit scalar K-quants (Q2_K, Q3_K) require multi-shift bit unpacking and fall back to Q4_0 whenever expert matrix dimensions are not multiples of 256 (such as 1856 × 2688 in Nemotron-3.5-30B-A3B).

llama-modchip v2.5.3 implements two block-64 quantization formats engineered for SIMD register lookup—Fractal-64 (GGML_TYPE_Q2_0 = 42, 30 bytes / 64 weights = 3.75 bpw) and LUT-VQ3-64 (GGML_TYPE_VQ3_0 = 43, 24 bytes / 64 weights = 3.00 bpw)—paired with cache-aware KV management (HARP-Cache), /dev/shm Slot-0 prefix snapshots, and RouteCrack + Stockfish-PVS v2 speculative decoding:

1. Domain-Aware LUT-VQ3-64 + Fractal-64 Hybrid Quantization: On uncompressed BF16 MoE weights (blk.1.ffn_up_exps.weight), Fractal-64 (3.75 bpw) reaches 17.75 dB SQNR (99.15% Cosine Similarity, +1.45 dB over Q3_K), while LUT-VQ3-64 (3.00 bpw) reaches 13.57 dB SQNR (97.78% Cosine Similarity, +3.42 dB over Q2_K) via 128 3D Lloyd-Max centroids decoded in registers with AVX-512 VBMI VPERMI2B. Keeping boundary layers and residual output projections (ffn_down_exps, MLA attention) in Fractal-64 while encoding interior expansion projections in LUT-VQ3-64 cuts up_exps footprint by 33.3% vs. Q4_0 with a +0.4818 (+6.92%) WikiText-2 Perplexity delta vs. True BF16 (7.4400 vs. 6.9582 on 12,288 wiki.test.raw tokens; +0.0790 / +1.07% vs. Fractal-64 7.3610), 0.1495 / 0.0944 nats/tok Logit KLD vs. True BF16 (85.2% Top-1, 98.0% Top-5), and 100.0 / 100.0 on benchmarks/eval_quality_and_tools.py.
2. DeepSeek-V4-Flash-0731 284B (13.17B Active) Inside 64 GB RAM: Four 3-shard 0731 checkpoints execute resident inside 64 GB DDR5 RAM: the donor-free safety-aligned default DeepSeek-V4-Flash-0731-MaxQ-Fractal (56.93 GiB disk / 57.54–58.13 GiB anonymous-heap VmRSS at -c 32768 -ub 512, pure single-pass 0731 @ 7872f01b BF16/FP8 across all 1,328 tensors: 174 dense bottlenecks [173 × Q6_K + 1 × Q8_0] + 317 Fractal-64 MLA/Indexer tensors + 129 @antirez 1.5M-token imatrix routed-MoE tensors, AtomicChat wiki-alt.txt ctx=5632 51-chunk PPL = 7.2977 ± 0.0504 [12-chunk 7.3180 ± 0.1039], WikiText-2 PPL = 11.6139 ± 0.5845, 89.6 / 100 Gate [70.0/80 Functional (30/30 Tools) + 19.56/20 Speed/RAM at 8.80 tok/s Gate / 8.66 tok/s pure N=1]), its opt-in uncensored twin DeepSeek-V4-Flash-0731-Abliterated-Fractal (56.93 GiB, --preset dsv4-ablit, identical single-pass recipe, WikiText-2 PPL = 11.3944 ± 0.5650, 70.0 / 80 Functional Gate, unconfounded 0.0959 / 0.0512 nats/tok KLD & 90.7% Top-1 / 99.4% Top-5 vs MaxQ), compact safety-aligned DeepSeek-V4-Flash-0731-VQ3 (56.40 GiB, Unsloth UD-IQ1_S @ fbbb5b93 gate/up donor, wiki-alt.txt 12c PPL = 7.6579 ± 0.1077, WikiText-2 PPL = 12.2345 ± 0.5976, 0.8409 / 0.4397 nats/tok KLD vs MaxQ, 9.05 tok/s N=1, 41.11 tok/s cold prefill, 19.4–25.3 tok/s agentic bursts), and cross-platform DeepSeek-V4-Flash-0731-Fractal (56.57 GiB, WikiText-2 PPL = 11.5288 ± 0.5563).
3. Nemotron-3.5-30B-A3B (3.0B Active) in 16–24 GiB RAM: Safety-aligned NVIDIA-Nemotron-3.5-30B-A3B-VQ3-64 (13.30 GiB, WikiText-2 PPL = 7.4400 ± 0.2626 vs. True BF16 6.9582 ± 0.2415) and Fractal-64 (14.36 GiB, PPL = 7.3610 ± 0.2602, 0.1254 / 0.0929 nats KLD vs. BF16) save -4.44 to -5.67 GiB RAM vs. Q4_0 at 22.44 tok/s N=1 (24.18–60.40 tok/s with Stockfish-PVS), alongside an opt-in uncensored Abliterated-VQ3-64 (13.30 GiB, PPL = 7.7721 ± 0.2835, 0.0308 / 0.0312 nats KLD & 92.4% Top-1 / 100.0% Top-5 vs. VQ3-64, --preset nemotron-ablit).
4. Sub-20ms /dev/shm Slot-0 Restore + Cross-Platform Native Support: Restores the 40-tool Claude Code prefix (2,776 tokens) from /dev/shm in 10.4 ms (0.16s TTFT) on Linux, runs on Apple Silicon Metal (57.65 tok/s M1 Max, 78.60 tok/s M3 Max, 27.0 tok/s M5 Max 284B), and compiles natively on Windows 11 (MinGW-w64 UCRT64, verified up to 1M context).

Measured via quantize_row_*_ref $\to$ dequantize_row_* on the uncompressed 59.0 GiB BF16 master checkpoint (blk.1.ffn_up_exps.weight, $N = 4,194,304$ weights, kurtosis $\kappa = 3.2–4.8$):

Measured on 16-Core AMD Zen 5 (--cpu-range 0-15 --cpu-strict 1, Transparent HugePages enabled; full methodology and shipped-hybrid vs. full-VQ3 ablation breakdown in benchmarks/QUALITY_GATE_VQ3_64.md).

Scalar uniform quantizers quantize each coordinate $w_i \in [-M, +M]$ independently across the 3D Cartesian bounding cube $[-M, +M]^3$ of volume $8M^3$. Heavy-tailed MoE weights ($\kappa = 3.2–4.8$) have a radial joint density $p(\mathbf{w}) \propto \exp(-|\mathbf{w}|_p^\alpha / \sigma^\alpha)$ concentrated inside the inscribed $L_2$ sphere ($\text{Vol} \approx 4.189 M^3$, leaving $47.64%$ of the cube empty) or $L_1$ cross-polytope ($\text{Vol} \approx 1.333 M^3$, leaving $83.33%$ empty).

LUT-VQ3-64 places 128 3D Lloyd-Max centroids ($7\text{ bits} / 3\text{ weights} = 2.333\text{ bpw}$, antipodally symmetric $\mathbf{c}_{127-k} = -\mathbf{c}_k$) directly on the populated spherical/cross-polytope shell inside a 24-byte block_vq3_0 struct (patches/vq3_codebook.h):

• 2 bytes (d): FP16 super-block scale (scaled by GGML_VQ3_G = 8.6373016238e-3f).
• 22 bytes (qs[22], 176 bits):
  • qs[0..19] bits 0..6 (140 bits): 20 7-bit 3D indices covering 60 weights at stride 22 (j, j+22, j+44).
  • qs[20..21] bits 0..5 (12 bits): 2 6-bit 2D tail indices covering 4 weights (20, 42 and 21, 43).
  • s24 (24 bits packed in bit 7 of qs[0..21] and bit 6 of qs[20..21]): Eight 3-bit unsigned sub-scales ($m_b = 2 + u_b \in {2..9}$) across the eight 8-weight sub-blocks (24 bytes / 64 weights = 3.00 bpw).

The 128-entry 3D codebook occupies $128 \times 3 = 384\text{ bytes}$, fitting inside 6 AVX-512 ZMM registers (cb0_lo/hi, cb1_lo/hi, cb2_lo/hi). AVX-512 VBMI VPERMI2B (_mm512_permutex2var_epi8) resolves 64 byte lookups per instruction directly in registers with zero L1/L2 cache reads. In the shipped Hugging Face checkpoints, interior pre-activation expansion tensors (ffn_up_exps in Nemotron, interior ffn_gate_shexp & ffn_up_shexp in DeepSeek-V4) use LUT-VQ3-64 (3.00 bpw), while boundary layers, ffn_down_* residual projections, and MLA attention tensors remain in Fractal-64 (GGML_TYPE_Q2_0 = 42, 3.75 bpw, 30 B / 64 W: 2B FP16 d + 4B 8×4-bit sub-scales {4..19} + 24B 3-bit indices into {-14,-8,-4,-1,+1,+4,+8,+14}).

Exact bit-widths (bpw), base checkpoints, and tensor allocations for each published GGUF:

Note on MoE Expert Routing (Unpruned Top-6 vs. Runtime Tail Pruning): By default (Unpruned Top-6), DeepSeek-V4-Flash evaluates all 6 routed experts + 1 shared expert (Top-6) per layer with zero runtime expert dropping relative to the loaded GGUF (1.00000 logit cosine similarity vs. unpruned Top-6 execution on the same quantized checkpoint; weight quantization fidelity itself is measured separately via BF16 SQNR and continuous WikiText-2 PPL / Logit KLD in benchmarks/QUALITY_GATE_VQ3_64.md). Runtime $L_2$-quadrature Top-5 / Top-4 pruning is isolated per user via 0700 DSV4_MOE_CFG_PATH ($XDG_RUNTIME_DIR/modchip-<uid>/dsv4_moe_cfg.bin) and resets to Top-6 on server startup.

Why DeepSeek-V4-Flash-0731 (and not V4.1-Flash-1017) for 64 GB Resident CPU Inference: DeepSeek-V4.1-Flash-1017 (DeepseekV41ForCausalLM, 510.3 GB source across 384 routed experts/layer) adds 188.83 GiB of native FP8 Engram lookup tables on top of 151.77 GiB of Q2 model weights (340.60 GiB total even at 2-bit quantization), requiring 128 GB+ RAM with NVMe SSD table streaming or 512 GB RAM for resident execution. DeepSeek-V4-Flash-0731 (284.33B total / 13.17B active, zero Engram tables) is the largest DeepSeek-V4 Flash checkpoint that runs 100% resident inside 64 GB RAM with zero disk streaming.

git clone https://github.com/synchopate/llama-modchip.git
cd llama-modchip
./modchip chat

Compatibility Note (GGML_TYPE_Q2_0 = 42 & GGML_TYPE_VQ3_0 = 43): Fractal-64 (30 B / 64 W) repurposes upstream GGML_TYPE_Q2_0 = 42 (18 B / 32 W) in-place and LUT-VQ3-64 (24 B / 64 W) appends custom type 43. Unpatched llama.cpp, Ollama, vLLM, and LM Studio will fail with unknown tensor type 43, a tensor bounds mismatch, or unknown model architecture: 'deepseek4'.

• Portable GLIBC_2.30 Linux x86_64 Binaries (GitHub Release v2.5.3): Built inside Ubuntu 20.04 (GLIBC_2.30 max symbol, -static-libstdc++, zero libssl/libcurl dependency; requires OpenMP libgomp1 / libgomp — Ubuntu 20.04+, Debian 11+, Fedora, RHEL/Rocky/Alma 9+) with SHA-256 verification in ./modchip download-bin and ./install.sh --prebuilt:
  • llama-server-modchip-linux-x86_64-avx512: AMD Zen 4/5 (Ryzen 7000/8000/9000, EPYC Genoa/Turin, Threadripper 7000) & Intel Ice Lake / Sapphire Rapids (avx512f + avx512vbmi; -DGGML_AVX512_BF16=OFF so Ice Lake / Tiger Lake never hit vdpbf16ps SIGILL).
  • llama-server-modchip-linux-x86_64-avx2: Pre-AVX512 x86_64 CPUs (x86-64-v3: Intel Haswell/Broadwell/Skylake/Alder/Raptor Lake, AMD Zen 1/2/3 — compiled with -mno-avx512f -DGGML_AVX_VNNI=OFF, 0 %zmm instructions). The unsuffixed llama-server-modchip-linux-x86_64 asset is an identical copy of this x86-64-v3 binary.

# Auto-select -avx512 or -avx2 binary for your Linux x86_64 CPU and verify SHA256:
BIN_ISA=$(grep -Eq "avx512_?vbmi" /proc/cpuinfo && echo "avx512" || echo "avx2")
curl -fLO "https://github.com/synchopate/llama-modchip/releases/download/v2.5.3/llama-server-modchip-linux-x86_64-${BIN_ISA}"
curl -fLO "https://github.com/synchopate/llama-modchip/releases/download/v2.5.3/SHA256SUMS"
sha256sum -c SHA256SUMS --ignore-missing
chmod +x "llama-server-modchip-linux-x86_64-${BIN_ISA}"

# 1A. Default 30B Nemotron-3.5-A3B VQ3-64 + Fractal-64 (13.30 GiB, Aligned — pass -c 32768 -ub 512 on 16 GB hosts):
hf download Synchopate/NVIDIA-Nemotron-3.5-30B-A3B-VQ3-64-CPU-GGUF \
  NVIDIA-Nemotron-3.5-30B-A3B-VQ3-64.gguf --local-dir .
./llama-server-modchip-linux-x86_64-${BIN_ISA} -m NVIDIA-Nemotron-3.5-30B-A3B-VQ3-64.gguf -c 262144 -t 16 -b 2048 -ub 512 --port 8080

# 1B. 30B Nemotron-3.5-A3B Fractal-64 (14.36 GiB, Aligned — recommended on AVX2-only CPUs & Apple Silicon):
hf download Synchopate/NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64-CPU-GGUF \
  NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64.gguf --local-dir .

# 2A. Default 284B DeepSeek-V4-Flash-0731-MaxQ-Fractal (56.93 GiB, Safety-Aligned Single-Pass Q6_K + Fractal-64):
hf download Synchopate/DeepSeek-V4-Flash-0731-MaxQ-Fractal-64GB-CPU-GGUF \
  --include "DeepSeek-V4-Flash-0731-MaxQ-Fractal-*.gguf" --local-dir ./dsv4-maxq
# Safe 64 GB Desktop default (-c 32768 -ub 512 -> ~57.5–58.1 GiB anonymous-heap VmRSS):
./llama-server-modchip-linux-x86_64-${BIN_ISA} -m ./dsv4-maxq/DeepSeek-V4-Flash-0731-MaxQ-Fractal-00001-of-00003.gguf \
  -c 32768 -t 16 -b 2048 -ub 512 -ctk q8_0 -ctv q8_0 -fa on --port 8080
# Dedicated Headless 64 GB Server (with zram/swap) or 96 GB+ Workstation (full 256K MLA context):
# ./llama-server-modchip-linux-x86_64-${BIN_ISA} -m ./dsv4-maxq/DeepSeek-V4-Flash-0731-MaxQ-Fractal-00001-of-00003.gguf \
#   -c 262144 -t 16 -b 2048 -ub 512 -ctk q8_0 -ctv q8_0 -fa on --port 8080

# 2B. Compact Safety-Aligned 284B DeepSeek-V4-Flash VQ3 (56.40 GiB) or Cross-Platform Fractal-64 (56.57 GiB):
hf download Synchopate/DeepSeek-V4-Flash-0731-VQ3-64GB-CPU-GGUF \
  --include "DeepSeek-V4-Flash-0731-VQ3-*.gguf" --local-dir ./dsv4-vq3
hf download Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF \
  --include "DeepSeek-V4-Flash-0731-Fractal-*.gguf" --local-dir ./dsv4

# 2C. Opt-In Abliterated / Uncensored Variants (284B dsv4-ablit & 30B nemotron-ablit — requires confirmation):
# ./install.sh --preset dsv4-ablit        # Synchopate/DeepSeek-V4-Flash-0731-Abliterated-Fractal-64GB-CPU-GGUF (56.93 GiB)
# ./install.sh --preset nemotron-ablit    # Synchopate/NVIDIA-Nemotron-3.5-30B-A3B-Abliterated-VQ3-64-CPU-GGUF (13.30 GiB)

64 GB Host RAM Headroom (-c 32768 -ub 512 Desktop vs. -c 262144 Headless): Because --load-mode none --repack stores repacked CPU weights in anonymous heap (57.54–59.13 GiB VmRSS), the Linux kernel cannot demand-page those pages back from disk under desktop RAM pressure without swap/zram. On a 64 GB Linux box (~62.5 GiB usable RAM), keep -c 32768 -ub 512 (start_server.sh enforces -c 32768 -ub 512 and disk-backed slot snapshots automatically on <= 68 GB hosts unless MODCHIP_CTX_SIZE is set) and enable zram or close heavy GUI browsers. Use -c 262144 (256K tokens) on dedicated headless 64 GB nodes with zram/swap or 96 GB+ workstations. Measured context scaling up to 3,038 tokens (30.7–38.4 tok/s prefill, 9.16 tok/s decode) is reported in docs/DEEPSEEK_V4_OPTIMIZATIONS.md §6.

• Standard GGUFs (Q4_0, MXFP4, Q3_K_M, Q8_0) with Automatic GPU+CPU Offload: Run windows/AVVIA_MODCHIP.bat (windows/README.md). It detects physical CPU cores, free RAM, and GPU VRAM (nvidia-smi), downloads official llama.cpp CUDA 12.4 / Vulkan binaries, computes --n-cpu-moe, and starts http://127.0.0.1:8080.
• Patched DeepSeek-V4 284B & Nemotron-30B (Fractal-64 type 42 & LUT-VQ3-64 type 43): Run powershell -ExecutionPolicy Bypass -File windows\build_patched_engine.ps1 -Target dsv4 (or -Target nemotron) to compile the native MinGW-w64 UCRT64 AVX-512 engine, then launch via windows/AVVIA_DSV4.bat (details in docs/WINDOWS_PORT.md).

curl -sL https://raw.githubusercontent.com/synchopate/llama-modchip/main/scripts/check_cpu_readiness.py | python3

Submit your terminal scorecard via a GitHub Benchmark Issue or GitHub Discussions to add your CPU to the leaderboard below.

Each model family has a dedicated technical guide with mathematical derivations, operator-level latency profiles, and quality gate benchmarks:

Microarchitectural details and tuning flags for each processor and GPU hybrid are documented in docs/HARDWARE_OPTIMIZATIONS.md:

Operator-level profiling and mathematical derivations in docs/DEEPSEEK_V4_OPTIMIZATIONS.md (probabilities and KL-divergence measured relative to unpatched Top-6 execution on the 56.57 GiB Fractal checkpoint):

Full ablation tables and 100/100 Quality Gate reports in docs/NEMOTRON_OPTIMIZATIONS.md and benchmarks/QUALITY_GATE_VQ3_64.md:

• Prefill Throughput (pp512): 169.4 ± 29.6 tok/s (Apple Metal accelerated)
• Decode Throughput (tg128): 27.0 ± 0.4 tok/s in llama-bench
• Real Served Generation (llama-server): 20.7 – 23.6 tok/s across coding and prose (code_edit: 20.7 tok/s, code_new: 23.3 tok/s, free_text: 23.6 tok/s) with 0 swapouts (iogpu.wired_limit_mb=57344)
• Architecture & Placement: Apple M5 Max (18-Core CPU + 40-Core Metal GPU), 64 GB LPDDR5X. 5 pure-IQ1_S MoE layers offloaded to CPU (blk.(10|12|14|16|18).ffn_.*_exps.=CPU), remainder on Metal.
• Driver & Patches: patches/apple-silicon/0001 (Fractal-64 Q2_0 NEON/Metal), 0002 (Metal tensor run mapping), 0003 (Metal register unpack). Built via apple-silicon/build-dsv4-engine.sh and launched with apple-silicon/run-dsv4.sh. Contributed by Bobby Grayson (@notactuallytreyanastasio). Full report: apple-silicon/results/dsv4-m5max.md.

Run ./modchip bench (or windows\BENCHMARK_PC.bat on Windows) and Submit Your Scorecard (GitHub Issue) (pure-CPU entries sorted by Refactor decode):

llama-modchip/
├── modchip                                    # Unified CLI (setup | start | chat | bench | eval | quantize | web | claude | stop)
├── install.sh                                 # Automated Installer (Linux)
├── README.md / ARCHITECTURE.md / FAQ.md       # Overview, Mathematical Architecture & Engineering FAQ
├── RELEASE_NOTES.md                           # v2.5.3 Release Notes
├── docs/
│   ├── NEMOTRON_OPTIMIZATIONS.md              # Nemotron-3.5-30B-A3B & Nemotron-H-8B optimizations
│   ├── DEEPSEEK_V4_OPTIMIZATIONS.md           # DeepSeek-V4-Flash 284B profiling, SVD & L2-quadrature MoE
│   ├── HARDWARE_OPTIMIZATIONS.md              # Chip-by-chip optimizations (Zen 5, Threadripper, Vulkan, CUDA, Metal, ARM64)
│   └── WINDOWS_PORT.md                        # Native Windows 11 patched-engine build & Ryzen 7 9800X3D 1M-context report
├── vq3/                                       # LUT-VQ3-64 3D Lloyd-Max codebook, self-contained C encoder & kernel tests
├── patches/
│   ├── apple-silicon/                         # Apple Silicon engine patches (Fractal-64 Q2_0 NEON/Metal, register unpack)
│   ├── deepseek-v4-flash-fractal-256k.patch   # Base engine patch (DeepSeek-V4-Flash 284B + Fractal-64 Q2_0 = 42)
│   ├── dsv4-64gb-cpu.patch                    # Apple Silicon base engine patch (pinned to llama.cpp d2f83055)
│   ├── dsv4-ipc-hardening.patch               # Per-user 0700 IPC & /dev/shm_<uid> hardening patch (Linux/macOS/Windows)
│   ├── lut-vq3-64-avx512-vbmi.patch           # Incremental LUT-VQ3-64 (VQ3_0 = 43) AVX-512 VBMI + AVX2 + scalar patch
│   ├── vq3_codebook.h                         # 128-entry 3D + 64-entry 2D LUT-VQ3-64 codebook header
│   └── zen5-nemotron-256k-optimizations.patch # Standalone ggml/llama core patch for Nemotron-3.5
├── windows/                                   # Windows 10/11 Patched Engine Builder (build_patched_engine.ps1, AVVIA_DSV4.bat) & Hybrid Pack
├── apple-silicon/                             # Apple Silicon M1–M5 setup scripts (build-dsv4-engine.sh, run-dsv4.sh, M5 Max report)
├── fedora/                                    # Fedora 44 / AMD Ryzen 7 8745HS (GEEKOM A8) all-in-one script
├── speculative/                               # RouteCrack + Stockfish-PVS v2 (rc_stockfish.h, ngram-map.cpp)
├── benchmarks/                                # eval_quality_and_tools.py (100-pt Quality Gate), QUALITY_GATE_VQ3_64.md & results/
├── gateway/sse_gateway.py                     # Agentic SSE Proxy + Compact Tool Table + Dark-Mode Web Chat HUD (:8787)
└── scripts/                                   # build_zen5_llama.sh, build_{nemotron,dsv4}_vq3.py, build_fractal_moe_gguf.sh, chat.py

• Salvatore Sanfilippo (@antirez) for single-pass DeepSeek-V4 routed-MoE quantization research and the 1.5M-token DeepSeek-V4-Flash-chat-v2-routed-moe-ds4-1p5m.dat (antirez/deepseek-v4-gguf @ f71f23d552d6) importance matrix used in the donor-free single-pass MaxQ-Fractal (v2.5.3) and Abliterated-Fractal checkpoints.
• Unsloth (unsloth/DeepSeek-V4-Flash-0731-GGUF) for the UD-IQ1_S (@ fbbb5b93b01893c8dc28612b1b2be5118e001f44) checkpoint used as the upstream donor (70 IQ1_S routed-MoE gate_exps/up_exps tensors in blk.0..34) when transcoding the compact DeepSeek-V4-Flash-0731-Fractal (56.57 GiB) and DeepSeek-V4-Flash-0731-VQ3 (56.40 GiB) variants.
• AtomicChat (AtomicChat/dsv4-eval-artifacts) for publishing the open wiki-alt.txt (-c 5632 -b 5632) evaluation corpus and cross-vendor DeepSeek-V4-Flash-0731 perplexity reference logs.
• DeepSeek AI (deepseek-ai) and NVIDIA for the foundational DeepSeek-V4-Flash (MIT License) and Nemotron-3.5-30B-A3B (NVIDIA Open Model License) architectures, and ggml-org/llama.cpp for the upstream C/C++ inference runtime.
• Prem Research (prem-research) and HangGlidersRule for the DeepSeek-V4-Flash-0731-abliterated and Darkstar-Nemotron-3.5-Lightning-30B-A3B-Abliterated-BF16 checkpoints used for the opt-in dsv4-ablit and nemotron-ablit presets.