Hybrid reasoning models interleave attention with recurrent mixers such as Mamba-2 and Gated DeltaNet, whose state is cached between decoding steps. Prior work showed that storing this state in FP16 or INT8 with round-to-nearest (RTN) makes Mamba-2 and GDN hybrids more verbose, and that stochastic rounding (SR) fixes it. We study the other 16-bit format that serving engines expose, bf16, on six open hybrid reasoning models and 590 competition-math problems. With bf16 RTN state, five of six models reason longer (geometric-mean length 1.07x to 1.27x; also under greedy decoding in the two models tested) and hit the 32K token cap more often. On Qwen3.5-9B, cap hits rise from 118 to 223 and accuracy falls from 85.6% to 70.8%. On LiveCodeBench, both models tested also reason longer and lose 12.6 and 13.4 points of pass@1. Rerunning the capped items with a 128K budget halves the accuracy gap but does not close it (93.6% vs. 86.6%): bf16 runs that never finish are twice as common, and 85% of them are exact repetition loops. On Nemotron-Nano-9B-v2, by contrast, 30 of 31 extra caps finish at 128K, so the cost there is mostly extra tokens (exploratory). Same-width bf16 SR removes the length increase in four of five affected models, restores Qwen3.5-9B to 87.6% and brings code length and pass@1 near FP32 levels, so the failure is mostly rounding bias, not missing resolution. With an integer-hash random generator, SR on Qwen3.5-9B matches FP32 on cap hits and accuracy (113 vs. 118; 85.6% for both) at the memory and decoding speed of bf16 RTN. In the Nemotron-Nano-v2 lineage, SR on 25% of heads (an intermediate-decay band) removes the effect, while random heads and the top teacher-forced-KL octiles do not; in Nemotron-3 and Qwen3.5-9B the effect is spread across heads. We recommend that evaluations of hybrid models report the state dtype, cap rate and output length alongside accuracy.
Figures & tables
Figure 1: Effective per-step decay of a quiet leaky integrator h←round(e−rh+ϵ) stored in bf16, relative to the true rate, over 20,000 trajectories. Input noise ϵ has the given standard deviation relative to the initial state. RTN freezes slow decay and distorts it up to about 4u ; SR is unbiased at every rate. The shaded band is the decay range of the Nemotron-Nano-9B-v2 heads whose SR removes the length effect (§ 8 ); the toy explains its upper edge but not its lower edge.
Figure 2: Share of the 590 items finished within n output tokens; items that hit the 32K cap never finish, so each curve ends at one minus the cap rate. Titles give the bf16/FP32 length ratio (Table 1 ).
Model (mixer)
Length ratio
Capped
Accuracy (%)
McNemar
Both finished
( p )
FP32 → bf16
FP32 → bf16
p
n
FP32 / bf16
Sampling
Nemotron-Nano-9B-v2 (M2)
× 1.197 (7e-19)
1 → 31
89.3 → 88.8
0.74
558
92.7 / 93.5
Nemotron-Nano-12B-v2 (M2)
× 1.188 (1e-18)
0 → 17
90.0 → 90.5
0.70
573
92.1 / 93.2
Nemotron-3-Nano-30B-A3B (M2)
× 1.081 (3e-6)
36 → 81
92.5 → 87.3
3e-7
509
98.4 / 98.6
Qwen3.5-9B (GDN)
× 1.265 (7e-16)
118 → 223
85.6 → 70.8
2e-14
322
99.7 / 99.7
Table 1: FP32 vs. bf16 RTN recurrent state ( n=590 , 32K cap). Length ratio is bf16/FP32. “Both finished” is accuracy on items that neither arm capped.
Qwen3.5-9B ( n=268 )
Nemotron-Nano-9B-v2 ( n=32 )
FP32
bf16 RTN
FP32
bf16 RTN
Unfinished at 128K
44
93
0
1
of which exact repetition loops
42
79
0
1
bf16-only 32K caps that finish at 128K
108 / 150
30 / 31
Accuracy among finished reruns
97.3%
97.7%
37.5%
25.8%
Median output length on rerun items
19,951
41,949
16,314
37,401
Table 2: Items capped at 32K in either arm, rerun at 128K with the same sampling settings. Qwen3.5-9B is pre-registered; Nemotron-Nano-9B-v2 (32 items) is exploratory. “Relaxed” accuracy covers all 590 items, using the 128K result for rerun items and the 32K result otherwise.
Model
Flog [95% CI]
Length ratio
Capped
Accuracy (%)
vs. FP32
SR/FP32
FP32 / RTN / SR
FP32 / RTN / SR
McNemar p
Nemotron-Nano-9B-v2
1.05 [0.88, 1.27]
× 0.991
1 / 31 / 0
89.3 / 88.8 / 89.7
0.88
Nemotron-Nano-12B-v2
0.98 [0.82, 1.16]
× 1.004
0 / 17 / 0
90.0 / 90.5 / 90.3
0.86
Nemotron-3-Nano-30B-A3B
0.38 [ − 0.37, 1.00]
× 1.049
36 / 81 / 41
92.5 / 87.3 / 92.2
0.81
Qwen3.5-9B
0.87 [0.58, 1.26]
× 1.031
118 / 223 / 111
85.6 / 70.8 / 87.6
0.24
hash generator
0.97 [0.70, 1.37]
× 1.007
118 / 223 / 113
85.6 / 70.8 / 85.6
1.00
Table 3: bf16 SR on all heads, same storage as bf16 RTN. Flog is the share of the bf16 log-length excess removed. Length ratio is relative to FP32. “Hash generator”: an integer hash instead of Triton’s Philox.
Model
Length ratio
Flog [95% CI]
Capped (of 350)
Pass@1 (%)
bf16/FP32 [95% CI]
FP32 / RTN / SR
FP32 / RTN / SR
Nemotron-Nano-9B-v2
× 1.37 [1.29, 1.46]
0.95 [0.77, 1.15]
36 / 162 / 30
58.3 / 45.7 / 59.4
Qwen3.5-9B
× 1.09 [1.05, 1.13]
0.73 [0.27, 1.33]
169 / 233 / 179
46.3 / 32.9 / 45.4
Table 4: LiveCodeBench v6 increment, 175 problems × 2 samples, 32K cap. Length ratio is bf16 RTN/FP32 with a 95% bootstrap CI over problems; Flog as in Table 3 . Pass@1 is exploratory.
SR applied to
Heads
Flog [95% CI]
Capped
KL share
none (bf16 RTN)
0
0
31
–
all heads
3456
1.05 [0.88, 1.27]
0
94% †
q1+q2 ( r∈[0.57u,4.1u) )
864
1.03 [0.86, 1.23]
1
52% ‡
q0+q1 (top-2 KL octiles)
864
0.59 [0.44, 0.76]
9
86% ‡
q0+q3 (size-matched control)
864
− 0.05 [ − 0.28, 0.14]
34
–
random (211 in q1+q2 )
864
0.06 [ − 0.14, 0.24]
34
–
Table 5: Nemotron-Nano-9B-v2: SR on subsets of heads, RTN elsewhere. Octiles q0 (slowest) to q7 of the weights-only decay rate, 432 heads each. “KL share” is the share of the bf16 teacher-forced KL at positions beyond 4K tokens that is removed by keeping the octile in FP32 ( † with SR on all heads instead; ‡ sum of the two single-octile shares, not measured jointly). KL shares keep an octile in FP32, whereas the closed-loop arms apply SR to it.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Test
Criterion
Result
Verdict
Head localization, Nemotron-Nano-9B-v2 (§ 8 )
SR on q1+q2
Flog>0.5 , CI lower >0.25 ; caps ≤10
1.03 [0.86, 1.23]; caps 1
pass
SR on q0
Flog<0.33
−0.12
pass
SR on q0+q3 (size-matched)
Flog<0.33
−0.05
pass
q2 vs. q0 (equal size)
Flog difference CI lower >0
0.55 [0.34, 0.80]
pass
SR on q0+q1 (top-2 KL octiles)
Flog(q1+q2)−Flog(q0+q1) CI lower >0 ; fail if Flog(q0+q1)≥0.8
0.44 [0.26, 0.64]; Flog 0.59 (predicted 0.29)
pass
Appendix
Table 6: Pre-registered tests and outcomes.
Hugging Face repository
Parallelism
Running cap FP32 / bf16
T / top- p / top- k
nvidia/NVIDIA-Nemotron-Nano-9B-v2
DP8
44 / 88
0.6 / 0.95 / –
nvidia/NVIDIA-Nemotron-Nano-12B-v2
DP8
38 / 76
0.6 / 0.95 / –
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
DP8
43 / 86
1.0 / 1.0 / –
Qwen/Qwen3.5-9B
DP6 / DP8 ∗
122 / 240
0.6 / 0.95 / 20
Qwen/Qwen3-Next-80B-A3B-Thinking
TP4 × DP2
223 / 436
0.6 / 0.95 / 20
tiiuae/Falcon-H1R-7B
DP8
47 / 94
0.6 / 0.95 / –
Appendix
Table 7: Serving settings. “Running cap” is the engine’s per-rank maximum of concurrent requests, read from the server logs.