More FP4-local testing with GLM-5.3
Trying Nvidia's next local optimized model
10/17/2026
In addition to the new QAD quant of Qwen I looked at yesterday, Nvidia has released a GLM-5.3 NVFP4 quant designed for 2x Sparks as part of their new local optimized collection.
As a quick recap, Nvidia is using quantization-aware distillation on these. This one is a little beefer than Qwen, requiring two Sparks or two RTX 6000 Pros, with 1.8 and 2M token K/V respectively. They're using a new lossless compression on the NVFP4 scale factors, which requires the Blackwell specific b12x kernel. We give up expert parallelization but they tend to run faster than regular NVFP4 quants otherwise.
The bad first, as expected GLM-5.3 continues to be a bit poky on Sparks. Qwen 3.8 Flash Next's architecture is clever, and its raw inference speed on lower memory bandwidth setups is incredible. Now on a pair of RTX 6000 Pros this GLM quant is going to absolutely scream. Nvidia's recipe uses K=5 for 2x Sparks and K=3 for 2x RTX 6000s. In my testing K=3 was always faster on Sparks, with the 4th and 5th token rarely being accepted. For editing, K=5 and K=3 is basically equal, but for reasoning at max effort K=5 is about 20% slower.
Single-user decode, tok/s (500 tokens, median of 3, 1K and 128K context):
| ctx | task | GLM FP4-Local, effort low / max | GLM E224, low / max | Qwen FP4-Local (live), thinking off / on | Qwen config C, off / on |
|---|---|---|---|---|---|
| 1K | code | 38.9 / 38.5 | 29.1 / 28.1 | 64.0 / 50.0 | 66.3 / 56.4 |
| 1K | code_ts | 41.9 / 33.3 | 31.0 / 26.8 | 64.0 / 46.5 | 68.0 / 49.2 |
| 1K | edit | 45.1 / 37.9 | 31.6 / 29.7 | 66.8 / 57.4 | 70.1 / 54.8 |
| 128K | code | 40.6 / 35.1 | 29.5 / 27.1 | 61.7 / 48.5 | 65.2 / 51.3 |
| 128K | code_ts | 42.6 / 33.3 | 31.4 / 26.6 | 62.9 / 43.6 | 65.1 / 48.2 |
| 128K | edit | 37.0 / 42.4 | 27.1 / 29.0 | 60.4 / 51.4 | 67.6 / 52.5 |
| mean | 41.0 / 36.8 | 30.0 / 27.9 | 63.3 / 49.5 | 67.0 / 52.1 |
Draft length (K) sweep: nvidia's K=5 is the slowest; K=3 is ours
| ctx | task | K=5 (recipe) | K=4 | K=3 | K=2 |
|---|---|---|---|---|---|
| effort low | |||||
| 1K | code | 35.5 (3.14, 11.3) | 38.7 (3.10, 12.5) | 38.9 (2.79, 14.0) | 38.7 (2.42, 16.0) |
| 1K | code_ts | 41.6 (3.81, 10.9) | 42.2 (3.42, 12.4) | 41.9 (3.02, 13.9) | 40.1 (2.57, 15.6) |
| 1K | edit | 44.4 (4.09, 10.8) | 45.6 (3.81, 12.0) | 45.1 (3.37, 13.4) | 41.5 (2.71, 15.3) |
| 128K | code | 34.0 (3.04, 11.2) | 40.2 (3.26, 12.3) | 40.6 (2.94, 13.8) | 40.2 (2.55, 15.8) |
| 128K | code_ts | 41.3 (3.81, 10.8) | 41.1 (3.39, 12.1) | 42.6 (3.12, 13.7) | 38.9 (2.50, 15.6) |
| 128K | edit | 31.0 (2.84, 10.9) | 35.2 (2.90, 12.1) | 37.0 (2.70, 13.7) | 37.1 (2.41, 15.4) |
| mean | 38.0 | 40.5 | 41.0 | 39.4 | |
| template default (effort max) | |||||
| 1K | code | 31.3 (2.85, 11.0) | 35.3 (2.84, 12.4) | 38.5 (2.77, 13.9) | 35.4 (2.25, 15.7) |
| 1K | code_ts | 28.1 (2.53, 11.1) | 30.3 (2.45, 12.4) | 33.3 (2.41, 13.8) | 34.5 (2.20, 15.7) |
| 1K | edit | 30.8 (2.80, 11.0) | 36.1 (2.94, 12.3) | 37.9 (2.74, 13.8) | 36.7 (2.36, 15.5) |
| 128K | code | 34.7 (3.12, 11.1) | 34.3 (2.77, 12.4) | 35.1 (2.53, 13.8) | 38.7 (2.46, 15.7) |
| 128K | code_ts | 29.4 (2.67, 11.0) | 31.1 (2.53, 12.3) | 33.3 (2.40, 13.9) | 32.8 (2.11, 15.6) |
| 128K | edit | 34.9 (3.18, 11.0) | 42.0 (3.49, 12.0) | 42.4 (3.10, 13.7) | 37.9 (2.47, 15.3) |
| mean | 31.5 | 34.9 | 36.8 | 36.0 |
Aggregate under concurrency, tok/s (recipe bench, thinking off / effort low):
| Streams | GLM FP4-Local | GLM E224 | Qwen FP4-Local (live) | Qwen config C |
|---|---|---|---|---|
| 1 | 35.3 | 26.6 | 56.5 | 58.3 |
| 2 | 48.4 | 43.2 | 86.9 | 96.1 |
| 3 | 51.5 | 59.5 | 115.5 | 137.6 |
| 4 | 75.3 | 70.6 | 137.9 | 174.5 |
| 5 | 73.0 | 70.7 | 155.7 | 197.7 |
| 6 | 93.7 | 86.7 | 186.9 | 228.2 |
Cold prefill, tok/s (unique prompt; targets 2K / 8K / 32K / 64K):
| GLM FP4-Local | GLM E224 | Qwen FP4-Local | Qwen config C |
|---|---|---|---|
| 1,657 / 2,049 / 2,170 / 2,164 | 1,479 / 1,860 / 1,950 / 1,987 | 2,517 / 2,723 / 2,728 / 2,672 | 1,109 / 2,344 / 3,031 / 2,989 |
It's the fastest GLM 5.3 quant yet for this setup. Also GLM tokens tend to be a little longer than Qwen tokens so this isn't truly apples to oranges. Raw token rates aren't really a measure of impact either, more on that shortly.
The great, it's extremely smart. Smarter than the pruned GLM-5.3 quants I tested earlier this week. Nvidia's QAD is really delivering with these models. The default max reasoning is a bit pokey, but on the more reasonable "high". GLM 5.3 no longer supports a true "thinking off" mode, however on "low" we consistently did not see any reasoning produced, despite the presence of empty <thinking> blocks through all of these tests. Update 10/24/2026 In further GLM testing we did still occasionally get thinking on low.
Quality (same harness, data and sandbox as Qwen and the other GLMs; run on the worker)
| Test | n | Qwen C | Qwen FP4-Local | GLM E192 | GLM E224 | GLM FP4-Local | vs Qwen C: C-only / GLM-only (p) | vs FP4-Local (p) | vs E192 (p) |
|---|---|---|---|---|---|---|---|---|---|
| HumanEval+ base / plus, thinking off | 164 | 95.1 / 93.3 | 97.6 / 94.5 | 95.1 / 92.1 | 95.1 / 92.1 | 94.5 / 91.5 | 9 / 6 (0.61) | 10 / 5 (0.30) | 7 / 6 (1.0) |
| MBPP+ base / plus, thinking off | 378 | 93.1 / 79.9 | 93.4 / 79.9 | 94.7 / 79.9 | 94.4 / 82.0 | 94.7 / 81.7 | 10 / 17 (0.25) | 9 / 16 (0.23) | 8 / 15 (0.21) |
| GSM8K, thinking off | 1319 | 96.4 | 96.4 | 96.3 | — | 96.5 | 22 / 24 (0.88) | 21 / 22 (1.0) | 17 / 20 (0.74) |
| HumanEval+ base / plus, thinking, 16K | 164 | 95.1 / 92.7 | 95.7 / 92.7 | 97.0 / 92.7 | — | 97.0 / 92.1 | 7 / 6 (1.0) | 5 / 4 (1.0) | 6 / 5 (1.0) |
| LiveCodeBench (80), thinking, 16K | 80 | 41.2 | 46.2 | 47.5 | — | 55.0 | 1 / 12 (0.003) | 2 / 9 (0.065) | 4 / 10 (0.18) |
| Long context 64K + 128K | 144 | 95.1% | 97.2% | 97.9% | 100% | 99.3% (two-hop 47/48) | 1 / 7 (0.07) | 1 / 4 (0.38) | 0 / 2 (0.50) |
| Corpus NLL, nats/token (176,964 GLM tokens) | 0.591 | 0.426 | 0.261 | ||||||
| Corpus bits/byte (tokenizer-independent) | 0.048 | 0.048 | 0.196 | 0.141 | 0.087 |
Time to solve (one problem at a time, wall clock per request, 16K budget)
| Model | Setting | HE+ solved / 41 | s per problem | s per solved | mean tokens | LCB solved / 10 | s per problem | mean tokens |
|---|---|---|---|---|---|---|---|---|
| Qwen FP4-Local (K=2) | low |
39 | 13.4 | 14.1 | 854 | 1 | 291 | 15,340 |
medium |
36 | 14.7 | 16.7 | 945 | 1 | 301.5 | 15,726 | |
xhigh |
37 | 40.4 | 44.8 | 2,168 | 1 | 292 | 15,313 | |
| GLM FP4-Local (K=3) | high |
36 | 18.5 | 21.1 | 728 | 2 | 323 | 10,680 |
max |
35 | 99.3 | 116.4 | 3,875 | 1 | 438 | 15,043 |
Very smart, and it shows at how quickly it solves these problems. Qwen still out performs generally at any of its thinking levels, especially the new FP4-local quant. GLM doesn't have a "medium" to compare against. Full notes here.
Nothing has yet to dethrone Qwen for my personal AI needs, but I am super excited for what comes next in these QAD FP4 quants!
Bonus: benchmarking is a good way to stay warm before you want to turn the furnace on for the season.
