Existing latent reasoners typically run for a preset latent horizon during inference. Intuitively, a longer horizon should provide more opportunity for reasoning. However, in this work, we find that extending latent Chain-of-Thought (CoT) to the full preset horizon may in fact degrade reasoning performance. We observe that latent CoT evolves through distinct functional stages, with earlier computation forming frontier reasoning states, while later computation mainly reorganizes existing candidates and can weaken gold candidate support. These findings motivate a rethink of fixed-horizon latent inference. To address this, we characterize the reasoning benefits of latent CoT by realized and remaining constructive progress. Empirically, we find that higher realized constructive progress corresponds to higher current reasoning accuracy, while more remaining progress corresponds to larger future gains from continued latent CoT. To this end, we introduce a lightweight method that estimates the remaining reasoning progress and adaptively adjusts the latent horizon during inference. Across representative latent CoT reasoners and multiple arithmetic benchmarks, our method improves reasoning accuracy by up to 7.2 percentage points while reducing latent computation by up to 29.0%, without retraining the LLM.
Figures & tables
Figure 1: The full latent horizon can bury better reasoning outcomes. We evaluate six trained latent CoT reasoners across GPT-2 ( Radford et al., 2019 ) and Llama-1B ( Grattafiori et al., 2024 ) backbones, covering Coconut, CODI, and SIM-CoT variants. For each problem, we retrospectively stop latent reasoning at the first depth that produces a correct answer, if such a depth exists. Gray bars show accuracy at the preset full horizon, while colored bars show accuracy under this retrospective stopping point.
Figure 2: An example of latent reasoning progress. We use the symbolic Coconut model from Zhu et al. (2026b) and roll it out for six latent steps to trace the reasoning progress. We report the top-6 candidates at each step, with highlighted nodes indicating high support.
Figure 3: Latent progression and revision. (a) Gold candidate formation; (b) Gold candidate support. For trajectories whose answer is formed at the third latent step, (c) Correction and reversal rates, and (d) Accuracy over revision vs. the best accuracy reached along the trajectory.
GSM8K
SVAMP
MultiArith
Latent step
Method
Full- K
Ours
Gain
Full- K
Ours
Gain
Full- K
Ours
Gain
saving (%)
GPT-2
CODI
42.1
43.4
+1.3
41.0
43.1
+2.1
94.4
95.0
+0.6
20.6
SIM-CoT
39.2
41.5
+2.3
40.0
43.5
+3.5
91.7
92.2
+0.5
29.0
Coconut
27.1
27.7
+0.6
31.4
33.7
+2.3
71.7
78.9
+7.2
11.0
SIM-CoT Coconut
44.7
44.7
+0.0
40.6
42.2
+1.6
92.8
93.3
+0.5
25.1
Table 1: Adaptive latent CoT inference. Full- K denotes the preset latent horizon while ours uses adaptive latent inference. Latent step saving (%) reports the average step reduction.
Model / Budget policy
Accuracy (%)
GSM8K
SVAMP
MultiArith
(a) CODI family
4 steps
5 steps
6 steps
4 steps
5 steps
6 steps
4 steps
5 steps
6 steps
GPT-2
CODI
Fixed depth
36.6
41.2
42.1
41.5
41.0
41.0
92.8
93.3
94.4
Adaptive
41.5
43.2
43.4
42.2
42.2
43.1
92.8
93.3
95.0
SIM-CoT
Fixed depth
35.3
39.1
39.2
41.3
40.6
40.0
89.4
91.1
91.7
Table 2: Comparison of latent CoT performance under adaptive and fixed-preset inference. We report task accuracy (%) under maximum latent budgets of 4, 5, and 6 steps.
Figure 4: Constructive progress completion across latent CoT . Bars show how constructive progress completion is distributed. The top strips summarize completion status at latent step t=3 , contrasting completed and still progressing. Triangles mark the first latent step where at least 75% of examples have completed constructive progress. Completion marks the first latent step where the latent CoT realizes all constructive progress.
Backbone
Method
GSM8K
SVAMP
MultiArith
Lower
Higher
Lower
Higher
Lower
Higher
GPT-2
CODI
29.3
45.0
16.6
70.8
81.0
96.7
SIM-CoT
10.8
53.3
13.2
73.4
25.3
94.1
Llama-1B
CODI
24.9
69.8
54.4
77.0
95.1
99.0
SIM-CoT
34.7
67.2
45.3
80.3
94.5
98.4
GPT-2
Coconut
11.4
32.8
21.1
45.3
31.4
38.7
Table 3: Reasoning performance across constructive progress. At latent depth t=3 , we compare task accuracy (%) across different levels of realized constructive progress, as defined in Section 4.2 .
Accuracy (%)
Upper bound
Compute
Dataset
Backbone
Method
Ours
ALR
DEER
Dynamic K-Head
Ours with true Ω
Per-item oracle
Avg. steps
GSM8K
GPT-2
CODI
41.1
38.4 − 2.7
37.3 − 3.8
38.4 − 2.7
42.6
47.0
4.37
SIM-CoT
39.2
35.1 − 4.1
34.3 − 4.9
34.3 − 4.9
41.8
44.7
4.28
Llama-1B
CODI
54.5
53.1 − 1.4
54.4 − 0.1
52.5 − 2.0
55.6
58.8
4.57
SIM-CoT
54.7
52.3 − 2.4
52.8 − 1.9
52.9 − 1.8
55.5
58.2
4.72
SVAMP
GPT-2
SIM-CoT
40.2
39.9 − 0.3
39.6 − 0.6
39.8 − 0.4
41.5
42.9
4.68
Table 6: Comparison with transferred adaptive latent methods. All methods run on the same frozen reasoners with matched average latent computation, and none stops before our adaptation step. The upper bounds replace our estimate with the true remaining progress, or give each item its best stopping step under the same total computation. Subscripts indicate the difference relative to ours.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
(a) Regression target
Remaining progress
Future gain
Current value
Shuffled
Signal free
Accuracy
47.2
46.9 − 0.3
45.3 − 1.9
45.1 − 2.1
45.1 − 2.1
Appendix
Table 11: Remaining-progress targets under matched latent computation. Each comparison keeps the trajectory representation, regressor, offline trajectories, and adaptive controller fixed, and differs only in the regression target. Panel (a) compares regression targets under the default design, with average latent computation matched at 4.49 steps. Panels (b) – (d) compare remaining progress with the current reasoning value under alternative designs. We report final task accuracy on GSM8K averaged over the four CODI-family reasoners, with subscripts indicating the difference relative to remaining progress.