Testing Nvidia's new NVFP4-local quant of Qwen 3.8 Flash Next
An incredible model specifically designed for single DGX Spark or RTX 6000 Pro setups.
October 16, 2026
Nvidia released what they've called an "NVFP4-local" quant of my current favorite Qwen 3.8 Flash Next. Unlike their previous official NVFP4 quant this one is designed for a single DGX Spark or a single RTX 6000 Pro. With a single Spark you'd have a key/value store of around 850-900k tokens, and on a single RTX 6000 Pro at least 650k-750k tokens.
I decided to modify their recipe and benchmark it on a TP=2 dual Spark setup as part of my continuing LLM tuning saga. Two Sparks allows for a K/V store of >5M tokens and lots of headroom to spare on each node. As usual full results in GitHub.
The good if not incredible news, NVFP4-local was essentially as smart than nvidia's official NVFP4 quant, and smarter than the faster quants I had been using. Nvidia did quantization-aware distilleation instead of post-training rounded. This is probably the smartest model that can run on 96GB of RAM.
| Test | n | A nvidia NVFP4 | B RadixArk BF16-dense | C RadixArk FP8-dense (prod) | FP4-Local | C-only / F4-only (p) |
|---|---|---|---|---|---|---|
| HumanEval+ base / plus, thinking off | 164 | 97.0 / 95.1 | 95.7 / 93.9 | 95.1 / 93.3 | 97.6 / 94.5 | 3 / 5 (p=0.73) |
| MBPP+ base / plus, thinking off | 378 | 93.9 / 79.9 | 92.3 / 79.4 | 93.1 / 79.9 | 93.4 / 79.9 | 5 / 5 (p=1.0) |
| GSM8K, thinking off | 1319 | 96.5 | 96.4 | 96.4 | 96.4 | 14 / 15 (p=1.0) |
| HumanEval+ base / plus, thinking on, 16K budget | 164 | — | 95.1 / 92.7 | 95.1 / 92.7 | 95.7 / 92.7 | 4 / 4 (p=1.0) |
| LiveCodeBench (80 problems), thinking on, 16K budget | 80 | — | 43.8 | 41.2 | 46.2 | 0 / 4 (p=0.125) |
| Long context 64K + 128K (retrieve / two-hop / reverse) | 144 | — | 96.5% (two-hop 43/48) | 95.1% (41/48) | 97.2% (44/48) | 0 / 3 (p=0.25) |
| Corpus NLL, nats/token (190,191 identical tokens) | 0.1315 | 0.1310 | 0.1340 | 0.1333 |
Now the downside, its a bit slower than the other quants. NVFP4-local is as the name describes designed for single GPU setups. Running it with TP=2 works but we do give up a little bit of speed based on the old quant and configuration: the b12x kernel does not support expert parallelization. We're using a nightly vLLM build, so maybe this will come eventually. I benchmarked it across all levels of speculative coding (my previous config "C" has been running on K=2)
| ctx | task | config C | FP4-Local K=4 | K=3 | K=2 |
|---|---|---|---|---|---|
| thinking off | |||||
| 1K | code | 66.3 | 69.0 (3.39, 20.3) | 66.2 (3.01, 22.0) | 64.0 (2.60, 24.6) |
| 1K | code_ts | 68.0 | 72.0 (3.51, 20.5) | 69.9 (3.18, 22.0) | 64.0 (2.59, 24.8) |
| 1K | edit | 70.1 | 77.4 (3.96, 19.6) | 73.1 (3.42, 21.4) | 66.8 (2.73, 24.5) |
| 128K | code | 65.2 | 61.4 (3.14, 19.6) | 66.0 (3.08, 21.4) | 61.7 (2.56, 24.1) |
| 128K | code_ts | 65.1 | 63.5 (3.22, 19.7) | 64.8 (2.99, 21.7) | 62.9 (2.61, 24.1) |
| 128K | edit | 67.6 | 68.5 (3.51, 19.5) | 62.0 (2.92, 21.3) | 60.4 (2.56, 23.6) |
| mean | 67.0 | 68.7 | 67.0 | 63.3 | |
| thinking on (template default) | |||||
| 1K | code | 56.4 | 47.7 (2.39, 20.0) | 43.8 (2.00, 21.9) | 50.0 (2.05, 24.3) |
| 1K | code_ts | 49.2 | 44.3 (2.23, 19.9) | 47.0 (2.16, 21.7) | 46.5 (1.90, 24.4) |
| 1K | edit | 54.8 | 49.7 (2.51, 19.8) | 54.3 (2.47, 22.0) | 57.4 (2.33, 24.6) |
| 128K | code | 51.3 | 42.9 (2.19, 19.6) | 48.3 (2.27, 21.3) | 48.5 (2.03, 23.9) |
| 128K | code_ts | 48.2 | 41.5 (2.13, 19.5) | 43.4 (2.05, 21.2) | 43.6 (1.82, 23.9) |
| 128K | edit | 52.5 | 48.8 (2.51, 19.4) | 52.0 (2.45, 21.3) | 51.4 (2.18, 23.6) |
| mean | 52.1 | 45.8 | 48.1 | 49.5 |
Thinking-on speeds drop across the board, and it gets worse with concurrency, about 20% slower at full capacity. I think its acceptable loss given some of the other benefits, like the increased headroom and slightly improved quality, so this will be the new quant and configuration I run locally. The recipe nvidia shipped with the NVFP4-local model also greatly improves the configuration for warm-cache prefix hits. Agent turnarounds are a lot faster with pi.dev across the 4 agents I typically have working.
This is THE option for a single Spark setup. On an RTX 6000 Pro the model will absolutely scream as long as you don't need a larger K/V store. Once EP works with the b12x kernel its going to be the no brainer 2x Spark model too.