Temperature scaling is typically applied as a post-hoc transformation of model probabilities. In posterior-induced flow language models, this creates two mismatches. First, posterior tempering implicitly changes both the endpoint distribution and the distribution of intermediate noisy states, while the denoiser remains trained on the original states. Second, the tempered posterior weights are combined with transport dynamics derived for the untempered flow. These mismatches are reflected empirically: lowering temperature improves GSM8K performance much less reliably in flow LMs than in discrete diffusion LMs. We introduce Bayes-Consistent Tempering (BCT), which makes temperature explicit in both the distributions and dynamics of the generative flow. Its learned realization trains the denoiser on tempered intermediate states and tempers the endpoint distribution using the corresponding tempered dynamics. We also derive training-free variants for conventionally trained checkpoints. Across hyperspherical flow LMs with two denoiser architectures and a von Mises--Fisher flow LM, BCT improves GSM8K generation across broad temperature ranges and achieves the strongest tempering performance. The gains also extend to Sudoku, demonstrating benefits on tightly constrained generation. At very low temperatures, the learned realization is also markedly more robust than the training-free variants, supporting temperature as a property of the generative flow across training and inference rather than a post-hoc probability adjustment. Code and model weights are available at https://daioba.github.io/bct .
Figures & tables
Figure 1: Conceptual overview of Bayes-Consistent Tempering (BCT). Posterior tempering implicitly changes both the endpoint distribution and the conditional bridge, while standard sampling retains the original conditional velocities and denoiser training states. BCT restores consistency between the bridge and sampling dynamics, and its learned realization additionally aligns training with the tempered bridge.
Figure 2: GSM8K accuracy against the sampling temperature on the two backbones (NFE =1024 ). Lines are the mean over three generation seeds and bands ± one sample standard deviation. Curves without a prefix use the released checkpoint; Cont. (dashed) is the matched continuation. At T=1 every BCT correction is the identity. Values in Table 1 .
TFT
Cont.
S -FLM
T
BCT-L
Standard
BCT-E
BCT-P
Standard
BCT-E
BCT-P
S-arch
1
12.6 ± 1.1
12.0 ± 0.5
12.0 ± 0.5
12.0 ± 0.5
12.5 ± 0.5
12.5 ± 0.5
12.5 ± 0.5
0.8
16.3 ± 0.2
14.1 ± 0.5
15.0 ± 0.5
15.9 ± 0.2
14.7 ± 0.7
15.0 ± 1.1
16.7 ± 0.2
0.6
20.4 ± 0.4
15.3 ± 0.6
16.5 ± 0.4
20.2 ± 0.3
15.6 ± 0.2
16.9 ± 0.3
19.7 ± 0.5
0.5
21.5 ± 0.3
16.8 ± 0.2
17.4 ± 0.4
21.7 ± 0.6
16.3 ± 0.5
17.7 ± 0.3
21.2 ± 0.6
Table 1: GSM8K accuracy (%), mean ± sample standard deviation over three generation seeds (1319 problems, NFE =1024 ).
Figure 3: Pass@ K on GSM8K (S-arch, NFE =1024 ), estimated from eight generations per problem. Bands are 95% bootstrap intervals over problems; the panels share the vertical axis.
Figure 4: Sudoku-Hard exact solve rate (NFE =180 ): mean and ± one sample standard deviation over three generation seeds; base-checkpoint results are in Table 5 .
Figure 5: GSM8K accuracy of BCT on the vMF flow LM (NFE =1024 ): mean and ± one sample standard deviation over five generation seeds. BCT-E falls below the plotted range for T≤0.3 ; more detailed results are in Table 7 .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Held-out selection of λ for BCT-L (634 held-out TinyGSM problems, TFT, seed 1). Thin lines: accuracy at each selection temperature; thick line: their mean, the selection score. The dotted line marks the selected λ . λ=0 applies no Bayes correction and λ=1 the full correction. Values in Table 2 .
S-arch
DiT
DiT (500 upd.)
λ
T=0.4
0.2
0.1
Score
T=0.4
0.2
0.1
Score
Score
0
279
285
282
44.48
271
278
287
43.95
42.27
0.25
279
292
290
45.27
278
295
316
46.74
48.74
0.5
283
304
303
46.79
281
299
295
46.00
48.32
0.75
284
305
295
46.48
282
295
257
43.85
45.79
1
286
300
289
46.00
279
279
181
38.85
43.22
Appendix
Table 2: Held-out λ sweep: correct answers out of 634 at each selection temperature and the selection score (mean accuracy over the three temperatures, %); bold: selected value. The last column is a DiT TFT checkpoint after 500 updates, not used for selection.
Cont.
TFT
standard tempering
BCT-L
T
K=1
2
4
8
K=1
2
4
8
1
12.3
19.6
28.3
37.5
12.7
20.2
29.0
37.7
0.8
14.4
22.5
32.1
42.0
16.5
25.1
34.7
44.5
0.6
15.8
24.3
34.1
44.5
20.2
29.8
40.3
51.3
0.5
16.9
25.8
36.0
46.8
21.8
31.3
41.2
51.1
Appendix
Table 3: Pass@ K (%) of Figure 3 : unbiased estimator over n=8 generations per problem (generation seeds 1,11,…,71 ), S-arch, 1319 GSM8K problems. Pass@1 is the mean single-generation accuracy over the eight draws and agrees with the three-seed mean of Table 1 to within 0.6 points.
Figure 7: Standard temperature scaling on the TinyGSM baselines released with S -FLM (NFE =1024 , seed 1). Gray: S -FLM (Section 4.2 ). Values in Table 4 .
T
MDLM
Duo
S -FLM (S-arch)
S -FLM (DiT)
1
15.0 (198)
18.6 (245)
13.0 (172)
11.7 (154)
0.8
22.5 (297)
23.0 (304)
15.0 (198)
13.9 (183)
0.6
28.5 (376)
28.3 (373)
15.8 (208)
16.3 (215)
0.5
29.3 (386)
29.5 (389)
16.5 (218)
17.3 (228)
0.4
30.2 (398)
32.1 (423)
17.5 (231)
17.0 (224)
0.3
32.6 (430)
34.0 (448)
18.0 (238)
16.9 (223)
Appendix
Table 4: Accuracy (%) and correct answers out of 1319 for Figure 7 . S -FLM uses the exact velocity.
Base checkpoint
Cont.
TFT
T
Standard
BCT-E
BCT-P
Standard
BCT-E
BCT-P
BCT-L
1
42.9 ± 1.5
42.9 ± 1.6
42.9 ± 1.5
41.7 ± 1.2
41.7 ± 1.2
41.7 ± 1.2
41.7 ± 1.1
0.8
42.6 ± 1.6
43.7 ± 1.4
43.6 ± 1.0
41.6 ± 1.2
42.6 ± 1.2
42.4 ± 1.0
42.7 ± 0.7
0.6
42.3 ± 1.3
44.1 ± 1.2
44.2 ± 1.1
41.4 ± 1.1
43.1 ± 1.0
42.7 ± 1.2
44.3 ± 1.2
0.5
42.2 ± 1.3
44.4 ± 1.3
44.4 ± 1.2
41.3 ± 1.0
43.3 ± 1.2
43.0 ± 0.8
45.1 ± 1.3
0.4
42.1 ± 1.2
44.8 ± 1.4
44.8 ± 1.1
41.2 ± 1.0
43.5 ± 1.2
43.2 ± 0.5
45.6 ± 1.3
Appendix
Table 5: Sudoku-Hard exact solve rate (%), mean ± standard deviation over generation seeds 1, 11 and 21 (2,000 test puzzles, NFE =180 ). BCT-L uses the held-out λ=1 . The seed standard deviation is dominated by a shift shared by all samplers.
λ
T=0.4
T=0.2
T=0.1
Mean
0
41.8
40.8
40.1
40.9
0.25
42.8
43.5
43.3
43.2
0.5
44.0
44.5
44.9
44.5
0.75
44.9
45.4
46.9
45.7
1
45.6
46.3
47.5
46.5
Appendix
Table 6: Held-out selection of λ for BCT-L on Sudoku-Hard: exact solve rate (%) of TFT on 2,000 held-out puzzles (generation seed 1, NFE =180 ). Mean: the selection score. Bold: selected. No test puzzle was used.
Released
Cont.
TFT
T
Standard
BCT-E
BCT-P
Standard
BCT-E
BCT-P
BCT-L
1
23.55 ± 0.79
23.55 ± 0.79
23.55 ± 0.79
23.59 ± 0.71
23.59 ± 0.71
23.59 ± 0.71
21.55 ± 0.86
0.8
27.79 ± 0.83
29.19 ± 1.43
29.39 ± 0.48
27.99 ± 0.27
29.80 ± 0.75
29.28 ± 0.74
28.19 ± 0.97
0.6
30.34 ± 0.63
31.71 ± 0.64
31.63 ± 0.77
30.61 ± 0.77
31.54 ± 0.78
32.43 ± 0.45
32.05 ± 0.47
0.5
30.80 ± 0.80
31.40 ± 0.62
32.36 ± 1.03
31.80 ± 0.77
31.21 ± 0.47
33.28 ± 0.39
33.54 ± 0.63
0.4
31.19 ± 0.99
28.26 ± 0.52
33.39 ± 0.90
32.57 ± 0.83
26.37 ± 0.32
33.93 ± 0.58
34.33 ± 0.97
Appendix
Table 7: BCT on SFCD: GSM8K accuracy (%), mean ± sample standard deviation over generation seeds 42–46 (1319 problems, NFE =1024 ). Released: the SFCD TinyGSM checkpoint with its released EMA weights. Cont. and TFT: 20,000 updates from it, evaluated with an adaptation-local EMA. BCT-L uses the held-out λ=1 . At T=1 every correction is the identity, so the three columns of a model coincide.