The end of October DGX Spark benchmark roundup
Qwen 3.8 Flash Next vs GLM 5.3 Flash vs Mimo 2.6 Flash
10/24/2026
I've run a bunch of benchmarks and optimizations on these three great models on a 2x DGX Spark setup. I wanted to do one roundup post now that I've run time-to-solve against all three and with both 0.6 and 1.0 temperatures. As always, full results (including raw decode/prefill speeds) live in https://github.com/nkavassalis/spark-tuning including how we ran these tests. (We used our best MTP settings for the time to solve tests)
Time to solve HumanEval+ (one problem at a time, wall clock per request, 16K budget)
| Model | Setting | HE+ solved / 41 | s per problem | s per solved | mean tokens | LCB solved / 10 | s per problem | mean tokens |
|---|---|---|---|---|---|---|---|---|
| Qwen FP4-Local (K=2) | thinking off (greedy) | 38 | 4.8 | 5.2 | 304 | 3 | 173 | 10,508 |
low |
39 | 13.4 | 14.1 | 854 | 1 | 291 | 15,340 | |
medium, T=0.6 |
36 | 14.7 | 16.7 | 945 | 1 | 301.5 | 15,726 | |
medium, T=1.0 |
38 | 15.8 | 17.0 | 1,010 | 1 | 332.5 | 15,789 | |
xhigh |
37 | 40.4 | 44.8 | 2,168 | 1 | 292 | 15,313 | |
| GLM FP4-Local (K=3) | high |
36 | 18.5 | 21.1 | 728 | 2 | 323 | 10,680 |
max |
35 | 99.3 | 116.4 | 3,875 | 1 | 438 | 15,043 | |
low, T=0.6 |
35 | 4.2 | 4.9 | 166 | 1 | 266 | 8,758 | |
low, T=1.0 |
35 | 15.3 | 17.9 | 541 | 2 | 272 | 8,503 | |
high, T=0.6 |
38 | 16.2 | 17.5 | 703 | 2 | 344 | 11,788 | |
high, T=1.0 |
37 | 23.7 | 26.3 | 874 | 2 | 378 | 12,618 | |
| MiMo (DFlash 7) | thinking off, T=0.6 | 34 | 7.2 | 8.7 | 356 | 1 | 110 | 4,069 |
| thinking off, T=1.0 | 35 | 9.5 | 11.1 | 411 | 0 | 160 | 4,327 | |
| thinking on, T=0.6 | 37 | 21.8 | 24.1 | 904 | 1 | 414 | 14,199 | |
| thinking on, T=1.0 | 37 | 37.4 | 41.4 | 1,205 | 2 | 467 | 13,134 |
And the rest of the quality tests
| Model | Setting | HumanEval+ (164) | MBPP+ (378) | GSM8K (1319) | LiveCodeBench (80) | Long context 64K/128K (144) |
|---|---|---|---|---|---|---|
| Qwen (FP8-dense) | thinking off | 93.3 | 79.9 | 96.4 | — | 95.1% |
thinking on, xhigh |
92.7 | — | — | 41.2 | ||
| Qwen FP4-Local | thinking off | 94.5 | 79.9 | 96.4 | — | 97.2% |
thinking on, low |
94.5 | — | — | 47.5 | ||
thinking on, medium, T=0.6 |
95.7 | — | — | 43.8 | ||
thinking on, medium, T=1.0 (server default)** |
95.1 | — | — | 37.5 | ||
thinking on, xhigh |
92.7 | — | — | 46.2 | ||
| MiMo | thinking off (server rep-pen 1.05) | 88.4 | 74.3 | 96.4 | — | 90.3% |
| thinking off, T=0.6 | 84.8 | 75.4 | — | — | ||
| thinking off, T=1.0 (server default) | 86.0 | 70.6 | — | — | ||
| thinking on | 89.0 | — | — | 46.2 | ||
| thinking on, T=0.6 | 92.1 | — | — | 48.8 | ||
| thinking on, T=1.0 | 92.1 | — | — | 45.0 | ||
| GLM FP4-Local | low |
91.5 | 81.7 | 96.5 | — | 99.3% |
high |
92.1 | — | — | 55.0 | ||
low, T=0.6 / T=1.0 |
92.1 / 91.5 | 83.6 / 83.1 | — | — | ||
high, T=0.6 / T=1.0 |
90.2 / 91.5 | — | — | 57.5 / 53.8 |
Qwen 3.8 Flash Next FP4-local across two nodes with the n-gram in memory is still my weapon of choice, but looking forward to what comes next!