Organizations: Academy of Sciences, Beijing, China · Department of Automation, Tsinghua University, Beijing, China · Department of Computer Science, Peking University, Beijing, China
The test-time compute strategy, such as Chain-of-Thought (CoT), has significantly enhanced the ability of large language models to solve complex tasks like logical reasoning. However, empirical studies indicate that simply increasing the compute budget can sometimes lead to a collapse in test-time performance when employing typical task decomposition strategies such as CoT. This work hypothesizes that reasoning failures with larger compute budgets stem from static planning methods, which hardly perceive the intrinsic boundaries of LLM reasoning. We term it as the Limited Reasoning Space hypothesis and perform theoretical analysis through the lens of a non-autonomous stochastic dynamical system. This insight suggests that there is an optimal range for compute budgets; over-planning can lead to redundant feedback and may even impair reasoning capabilities. To exploit the compute-scaling benefits and suppress over-planning, this work proposes Halo, a model predictive control framework for LLM planning. Halo is designed for long-horizon tasks with reason-based planning and crafts an entropy-driven dual controller, which adopts a Measure-then-Plan strategy to achieve controllable reasoning. Experimental results demonstrate that Halo outperforms static baselines on complex long-horizon tasks by dynamically regulating planning at the reasoning boundary.
Figures & tables
Figure 1: The Universality of the performance degradation across Model Scales. We evaluate seven state-of-the-art LLMs (from 7B to o1-mini) on the Omni-MATH benchmark. (Left) Width-Driven: As the number of parallel reasoning paths ( k ) increases in CoT-SC, performance initially peaks due to ensemble benefits but eventually degrades as noise dominates the majority vote. (Right) Depth-Driven: Similarly, in Tree-of-Thoughts (ToT), extending search depth ( d ) beyond the limited reasoning space leads to catastrophic error propagation.
Figure 2: The Perils of Over-Reasoning: A Case Study. While moderate reasoning yields the correct answer (Left), extending the chain beyond the limited reasoning space introduces noise (Step 4). This noise is amplified through recursive feedback, leading to hallucination (Right). Our framework, Halo (Bottom Center), detects such drift and resets the state to maintain logical stability.
Figure 3: The Halo Framework: Closing the Loop on Reasoning Dynamics. Top: Standard Chain-of-Thought operates in an open-loop regime , where stochastic errors accumulate unchecked until the trajectory suffers from severe degradation. Bottom: Halo introduces a Model Predictive Control (MPC) loop comprising three stages: (1) The Observer (Sec. 3.1 ) estimates the instantaneous drift rate λ^t via mean attention entropy H ; (2) The Controller (Sec. 3.2 ) tracks the accumulated uncertainty Ωt against a Tolerance Threshold Ψ ; (3) The Actuator (Sec. 3.3 ) executes a Trajectory Rectification via semantic compression and history reset, projecting the system back to stable space and resetting the uncertainty score Ωt←0 .
Family
Model
Params
Context
Open-Dense
LLaMA-3.1-8B-Instruct
8B
128k
LLaMA-3.1-70B-Instruct
70B
128k
Gemma-2-27B-IT
27B
8k
Mistral-7B-Instruct-v0.3
7B
32k
Open-SOTA
Qwen2.5-72B-Instruct
72B
32k
Qwen2.5-Math-72B
72B
32k
Table 1: Model Specifications. We expand the evaluation suite to 10 models, covering the SOTA open-source landscape (Qwen2.5, LLaMA-3.1, Gemma-2) to verify Halo’s architectural universality.
Method
Mathematical & Symbolic Reasoning
Long-Context Stability & Retrieval
GSM8K (Tier 1)
MATH (Tier 1)
OMNI (Tier 2)
RULER (Tier 2)
LongBench
InfBench
LRA-L2
SR ↑
RTO
SR ↑
RTO
SR ↑
RTO
SR ↑
RTO
SR ↑
RTO
SR ↑
RTO
SR ↑
RTO
Standard CoT
82.4
1.00
10.8
1.00
12.5
1.00
18.2
1.00
35.6
1.00
28.9
1.00
15.3
1.00
CoT-SC ( k =10)
86.1
2.15
14.2
2.15
15.8
2.15
21.4
2.15
39.2
2.15
32.5
2.15
18.7
2.15
AdaCoT
88.5
1.45
19.3
1.45
21.5
1.45
25.1
1.45
42.8
1.45
36.7
1.45
22.1
1.45
ToT ( b =5, d =3)
87.4
3.50
19.1
3.50
21.3
3.50
25.7
3.50
41.5
3.50
35.2
3.50
21.5
3.50
Table 2: Holistic Multi-Metric Performance (LLaMA-3-70B). Success Rate (SR, %) and Relative Token Overhead (RTO, × ). Best results are bolded , and second-best results are underlined . Halo achieves Pareto-optimal performance, significantly outperforming baselines on Tier 2 tasks (Omni-MATH, RULER) where reasoning collapse typically occurs.
Backbone
CoT
ToT
AdaCoT
Halo
Gain
Small & Efficient Models
LLaMA-3.1-8B
42.5
48.1
49.3
56.8
+15%
Mistral-v0.3
44.2
49.5
51.0
57.4
+13%
Gemma-2-27B
58.1
62.4
63.8
69.2
+8.5%
MoE Architectures
Mixtral-8x7B
55.4
59.2
60.5
66.7
+10%
Table 3: Cross-Model Robustness on Omni-MATH. Halo demonstrates consistent gains across varying scales. Rel. Gain denotes improvement over AdaCoT. Notably, Halo pushes the SOTA Qwen2.5-Math to 91.3%, breaking the static inference ceiling.
Figure 4: Mechanistic Insights into Reasoning Stability on Omni-MATH. (a) Trajectory Projection: t-SNE visualization of hidden states st . Standard CoT (Red) diverges into the high-entropy regime, while Halo (Blue) maintains logical anchoring via periodic trajectory rectification (marked by ⋆ ). (b) Phase Transition: Reasoning accuracy exhibits a sharp collapse as cumulative uncertainty breaches the threshold Ψ . Halo preemptively acts to avoid this regime.
Figure 5: Distributional Alignment of Reasoning Boundaries. We compare the step index of baseline failures (Red) and Halo interventions (Blue) across seven reasoning domains. The tight clustering of data points indicates that Halo consistently identifies the critical reasoning horizon with low variance.
Figure 6: Stability Phase Transition Analysis. The heatmap illustrates the Success Rate (SR) as a function of Reasoning Length N . The dashed line represents the theoretical stability boundary N∗ . The stochastic fluctuations in the red Divergence Regime indicate the collapse of semantic consistency as the accumulated uncertainty dominates the trajectory.
Sensitivity ( α )
Tolerance Threshold ( Ψ )
Ψ=2.0
Ψ=4.0
Ψ=5.0
Ψ=6.0
Ψ=8.0
0.5
51.2
52.8
53.1
52.5
50.4
0.85 (Ours)
53.5
56.5
56.8
56.2
51.9
1.0
52.8
55.9
56.4
55.7
51.2
1.2
50.1
54.2
54.8
53.9
50.8
Table 4: Sensitivity Analysis on Omni-MATH (LLaMA-3-8B). Success Rate (SR, %) with varying Tolerance Thresholds ( Ψ ) and Sensitivity ( α ). The performance remains robust across a wide range of configurations compared to the AdaCoT baseline (49.3%).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Physical Meaning in LLM Reasoning
Running Example (Inequality Task)
St
Hidden State / Semantic Embedding at step t . It captures the current logical snapshot in the latent space M .
At Step 14, the vector S14 encodes the intermediate thought: "The inequality simplifies to x2−4x−32>0 . The roots are roughly 2±6.4 , so x≈8.4 and x≈−4.4 ."
ξt
Stochastic Noise Vector . Represents random fluctuations in token sampling (e.g., top- p ) or "distractor" associations triggered by the context.
A random noise vector ξ15 triggers a low-probability association with "complex analysis," proposing a distractor thought: "Wait, logarithms have complex domains… maybe x=2+3i ?"
Jt
Local Jacobian Matrix ( ∇StG ). Measures how strongly the model amplifies a small distraction in the next step.
The model is uncertain about the integer constraint. The Jacobian J15 is expansive ( ∥J15∥>1 ), causing the minor "complex number" thought to dominate the attention in the next token generation.
H(At)
Attention Entropy . A proxy for "Attention Dispersion." High entropy implies the model is losing focus and attending uniformly to irrelevant history.
At Step 15, H15 spikes to 3.2 (vs. baseline 0.8). The attention map is diffuse, looking at both the initial "integer" constraint and the new "complex root" noise simultaneously.
λ^t
Instantaneous Drift Rate . Estimated via Eq. 8 ( λ^t=β+αHt ). A positive value indicates the onset of a chaotic regime.
The calculated drift λ^15 becomes positive (+0.45), signaling that the reasoning trajectory is diverging from the logical manifold (Real Numbers R ) into hallucination (Complex Plane C ).
Ωt
Accumulated Uncertainty Score . The integral of drift ( ∑λ^k ). Represents the total risk of collapse accumulated so far.
By Step 15, Ω15 reaches 5.2 , breaching the safety threshold. The system recognizes that the chain has become too "fuzzy" to continue reliably.
Appendix
Table 5: Nomenclature and Physical Interpretation. The Running Example traces the trajectory of an LLM solving a logarithmic inequality, illustrating how Halo detects and corrects a "Complex Number Hallucination" at Step 15.
Aggregation Method
Correlation ( r )
Lead Time (Tokens)
Final Layer Only
0.62
0.8
Attention Head with Max Variance
0.71
1.2
First 1/2 Layers Average
0.54
0.5
Layer-Averaged (Halo)
0.88
3.2
Appendix
Table 6: Predictive Power of Different Entropy Aggregations. We report the Pearson Correlation ( r ) with semantic drift and the Detection Lead Time (in tokens) before logic collapse on Omni-MATH.
Dataset
Tier
Domain
Samples
Lavg
GSM8K
1
Math
1,319
∼150
MATH (Lvl 1-2)
1
Math
2,000
∼210
Omni-MATH
2
Symbolic
4,428
∼850
RULER (32k)
2
Context
1,000
∼31,500
Appendix
Table 7: Statistical Characteristics of Evaluation Benchmarks. Lavg denotes average token length of the ground truth solution/context.
Figure 7: Universality of Phase Transition across Model Scales. (a) Mistral-7B exhibits an earlier and sharper performance cliff at N∗≈12 , indicating higher sensitivity to stochastic drift. (b) Mixtral-8x7B extends the stable region to N∗≈22 but shows increased variance in the transition zone due to MoE routing dynamics. The consistent existence of the “Performance Cliff” across different backbones validates that the critical reasoning horizon N∗ is an intrinsic property determined by model capacity.
Table 8: Sensitivity Analysis of Halo Hyperparameters. Success Rate (%) on Omni-MATH using LLaMA-3-8B. The results highlight the trade-off between over-correction (low Ψ ) and under-correction (high Ψ ). Bold indicates the optimal configuration.
Scaling test-time compute through extended chains of thought has become a dominant paradigm for improving large language model reasoning. However, existing research implicitly assumes that longer thinking always yields better results. This assumption remains largely unexamined. We systematically investigate how the marginal utility of additional reasoning tokens changes as compute budgets increase. We find that marginal returns diminish substantially at higher budgets and that models exhibit ``overthinking'', where extended reasoning is associated with abandoning previously correct answers. Furthermore, we show that optimal thinking length varies across problem difficulty, suggesting that uniform compute allocation is suboptimal. Our cost-aware evaluation framework reveals that stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy.
Shu Zhou, Rui Ling, Junan Chen +3
Nanjing University · Baidu · Nanjing University of Finance & Economics
Large language models (LLMs) excel at complex reasoning, yet their efficiency is limited by the surging cognitive overhead of long thought traces. In this paper, we propose LightThinker, a method that enables LLMs to dynamically compress intermediate thoughts into compact semantic representations. However, static compression often struggles with complex reasoning where the irreversible loss of intermediate details can lead to logical bottlenecks. To address this, we evolve the framework into LightThinker++, introducing Explicit Adaptive Memory Management. This paradigm shifts to behavioral-level management by incorporating explicit memory primitives, supported by a specialized trajectory synthesis pipeline to train purposeful memory scheduling. Extensive experiments demonstrate the framework's versatility across three dimensions. (1) LightThinker reduces peak token usage by 70% and inference time by 26% with minimal accuracy loss. (2) In standard reasoning, LightThinker++ slashes peak token usage by 69.9% while yielding a +2.42% accuracy gain under the same context budget for maximum performance. (3) Most notably, in long-horizon agentic tasks, it maintains a stable footprint beyond 80 rounds (a 60%-70% reduction), achieving an average performance gain of 14.8% across different complex scenarios. Overall, our work provides a scalable direction for sustaining deep LLM reasoning over extended horizons with minimal overhead.
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multiple-Choice Knapsack Problem (OS-MCKP). This perspective highlights a meta-cognitive requirement---anticipating problem difficulty and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. First, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost levels and decide whether a problem is worth attempting before generation. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Experiments across mathematical reasoning, multiple-choice question answering, and code generation show that ROI-Reasoning improves scores and generally reduces reference-order regret under tight budgets, with up to a 417% relative score improvement over the strongest same-backbone baseline in mathematics.
Muyang Zhao, Chunhao Tian, Qi Qi +1
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China