cs.AIFeb 22, 2026
SaveLimited Reasoning Space: The cage of long-horizon reasoning in LLMs
Organizations: Academy of Sciences, Beijing, China · Department of Automation, Tsinghua University, Beijing, China · Department of Computer Science, Peking University, Beijing, China
Abstract
The test-time compute strategy, such as Chain-of-Thought (CoT), has significantly enhanced the ability of large language models to solve complex tasks like logical reasoning. However, empirical studies indicate that simply increasing the compute budget can sometimes lead to a collapse in test-time performance when employing typical task decomposition strategies such as CoT. This work hypothesizes that reasoning failures with larger compute budgets stem from static planning methods, which hardly perceive the intrinsic boundaries of LLM reasoning. We term it as the Limited Reasoning Space hypothesis and perform theoretical analysis through the lens of a non-autonomous stochastic dynamical system. This insight suggests that there is an optimal range for compute budgets; over-planning can lead to redundant feedback and may even impair reasoning capabilities. To exploit the compute-scaling benefits and suppress over-planning, this work proposes Halo, a model predictive control framework for LLM planning. Halo is designed for long-horizon tasks with reason-based planning and crafts an entropy-driven dual controller, which adopts a Measure-then-Plan strategy to achieve controllable reasoning. Experimental results demonstrate that Halo outperforms static baselines on complex long-horizon tasks by dynamically regulating planning at the reasoning boundary.
Figures & tables
Figure 1: The Universality of the performance degradation across Model Scales. We evaluate seven state-of-the-art LLMs (from 7B to o1-mini) on the Omni-MATH benchmark. (Left) Width-Driven: As the number of parallel reasoning paths ( ) increases in CoT-SC, performance initially peaks due to ensemble benefits but eventually degrades as noise dominates the majority vote. (Right) Depth-Driven: Similarly, in Tree-of-Thoughts (ToT), extending search depth ( ) beyond the limited reasoning space leads to catastrophic error propagation.
Figure 2: The Perils of Over-Reasoning: A Case Study. While moderate reasoning yields the correct answer (Left), extending the chain beyond the limited reasoning space introduces noise (Step 4). This noise is amplified through recursive feedback, leading to hallucination (Right). Our framework, Halo (Bottom Center), detects such drift and resets the state to maintain logical stability.
Figure 3: The Halo Framework: Closing the Loop on Reasoning Dynamics. Top: Standard Chain-of-Thought operates in an open-loop regime , where stochastic errors accumulate unchecked until the trajectory suffers from severe degradation. Bottom: Halo introduces a Model Predictive Control (MPC) loop comprising three stages: (1) The Observer (Sec. 3.1 ) estimates the instantaneous drift rate via mean attention entropy ; (2) The Controller (Sec. 3.2 ) tracks the accumulated uncertainty against a Tolerance Threshold ; (3) The Actuator (Sec. 3.3 ) executes a Trajectory Rectification via semantic compression and history reset, projecting the system back to stable space and resetting the uncertainty score .
| Family | Model | Params | Context |
| Open-Dense | LLaMA-3.1-8B-Instruct | 8B | 128k |
| LLaMA-3.1-70B-Instruct | 70B | 128k | |
| Gemma-2-27B-IT | 27B | 8k | |
| Mistral-7B-Instruct-v0.3 | 7B | 32k | |
| Open-SOTA | Qwen2.5-72B-Instruct | 72B | 32k |
| Qwen2.5-Math-72B | 72B | 32k |
Table 1: Model Specifications. We expand the evaluation suite to 10 models, covering the SOTA open-source landscape (Qwen2.5, LLaMA-3.1, Gemma-2) to verify Halo’s architectural universality.
| Method | Mathematical & Symbolic Reasoning | Long-Context Stability & Retrieval | |||||||||||||
| GSM8K (Tier 1) | MATH (Tier 1) | OMNI (Tier 2) | RULER (Tier 2) | LongBench | InfBench | LRA-L2 | |||||||||
| SR | RTO | SR | RTO | SR | RTO | SR | RTO | SR | RTO | SR | RTO | SR | RTO | ||
| Standard CoT | 82.4 | 1.00 | 10.8 | 1.00 | 12.5 | 1.00 | 18.2 | 1.00 | 35.6 | 1.00 | 28.9 | 1.00 | 15.3 | 1.00 | |
| CoT-SC ( =10) | 86.1 | 2.15 | 14.2 | 2.15 | 15.8 | 2.15 | 21.4 | 2.15 | 39.2 | 2.15 | 32.5 | 2.15 | 18.7 | 2.15 | |
| AdaCoT | 88.5 | 1.45 | 19.3 | 1.45 | 21.5 | 1.45 | 25.1 | 1.45 | 42.8 | 1.45 | 36.7 | 1.45 | 22.1 | 1.45 | |
| ToT ( =5, =3) | 87.4 | 3.50 | 19.1 | 3.50 | 21.3 | 3.50 | 25.7 | 3.50 | 41.5 | 3.50 | 35.2 | 3.50 | 21.5 | 3.50 | |
Table 2: Holistic Multi-Metric Performance (LLaMA-3-70B). Success Rate (SR, %) and Relative Token Overhead (RTO, ). Best results are bolded , and second-best results are underlined . Halo achieves Pareto-optimal performance, significantly outperforming baselines on Tier 2 tasks (Omni-MATH, RULER) where reasoning collapse typically occurs.
| Backbone | CoT | ToT | AdaCoT | Halo | Gain |
| Small & Efficient Models | |||||
| LLaMA-3.1-8B | 42.5 | 48.1 | 49.3 | 56.8 | +15% |
| Mistral-v0.3 | 44.2 | 49.5 | 51.0 | 57.4 | +13% |
| Gemma-2-27B | 58.1 | 62.4 | 63.8 | 69.2 | +8.5% |
| MoE Architectures | |||||
| Mixtral-8x7B | 55.4 | 59.2 | 60.5 | 66.7 | +10% |
Table 3: Cross-Model Robustness on Omni-MATH. Halo demonstrates consistent gains across varying scales. Rel. Gain denotes improvement over AdaCoT. Notably, Halo pushes the SOTA Qwen2.5-Math to 91.3%, breaking the static inference ceiling.
Figure 4: Mechanistic Insights into Reasoning Stability on Omni-MATH. (a) Trajectory Projection: t-SNE visualization of hidden states . Standard CoT (Red) diverges into the high-entropy regime, while Halo (Blue) maintains logical anchoring via periodic trajectory rectification (marked by ). (b) Phase Transition: Reasoning accuracy exhibits a sharp collapse as cumulative uncertainty breaches the threshold . Halo preemptively acts to avoid this regime.
Figure 5: Distributional Alignment of Reasoning Boundaries. We compare the step index of baseline failures (Red) and Halo interventions (Blue) across seven reasoning domains. The tight clustering of data points indicates that Halo consistently identifies the critical reasoning horizon with low variance.
Figure 6: Stability Phase Transition Analysis. The heatmap illustrates the Success Rate (SR) as a function of Reasoning Length . The dashed line represents the theoretical stability boundary . The stochastic fluctuations in the red Divergence Regime indicate the collapse of semantic consistency as the accumulated uncertainty dominates the trajectory.
| Sensitivity ( ) | Tolerance Threshold ( ) | ||||
| 0.5 | 51.2 | 52.8 | 53.1 | 52.5 | 50.4 |
| 0.85 (Ours) | 53.5 | 56.5 | 56.8 | 56.2 | 51.9 |
| 1.0 | 52.8 | 55.9 | 56.4 | 55.7 | 51.2 |
| 1.2 | 50.1 | 54.2 | 54.8 | 53.9 | 50.8 |
Table 4: Sensitivity Analysis on Omni-MATH (LLaMA-3-8B). Success Rate (SR, %) with varying Tolerance Thresholds ( ) and Sensitivity ( ). The performance remains robust across a wide range of configurations compared to the AdaCoT baseline (49.3%).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Physical Meaning in LLM Reasoning | Running Example (Inequality Task) |
| Hidden State / Semantic Embedding at step . It captures the current logical snapshot in the latent space . | At Step 14, the vector encodes the intermediate thought: "The inequality simplifies to . The roots are roughly , so and ." | |
| Stochastic Noise Vector . Represents random fluctuations in token sampling (e.g., top- ) or "distractor" associations triggered by the context. | A random noise vector triggers a low-probability association with "complex analysis," proposing a distractor thought: "Wait, logarithms have complex domains… maybe ?" | |
| Local Jacobian Matrix ( ). Measures how strongly the model amplifies a small distraction in the next step. | The model is uncertain about the integer constraint. The Jacobian is expansive ( ), causing the minor "complex number" thought to dominate the attention in the next token generation. | |
| Attention Entropy . A proxy for "Attention Dispersion." High entropy implies the model is losing focus and attending uniformly to irrelevant history. | At Step 15, spikes to 3.2 (vs. baseline 0.8). The attention map is diffuse, looking at both the initial "integer" constraint and the new "complex root" noise simultaneously. | |
| Instantaneous Drift Rate . Estimated via Eq. 8 ( ). A positive value indicates the onset of a chaotic regime. | The calculated drift becomes positive (+0.45), signaling that the reasoning trajectory is diverging from the logical manifold (Real Numbers ) into hallucination (Complex Plane ). | |
| Accumulated Uncertainty Score . The integral of drift ( ). Represents the total risk of collapse accumulated so far. | By Step 15, reaches 5.2 , breaching the safety threshold. The system recognizes that the chain has become too "fuzzy" to continue reliably. |
Table 5: Nomenclature and Physical Interpretation. The Running Example traces the trajectory of an LLM solving a logarithmic inequality, illustrating how Halo detects and corrects a "Complex Number Hallucination" at Step 15.
| Aggregation Method | Correlation ( ) | Lead Time (Tokens) |
| Final Layer Only | 0.62 | 0.8 |
| Attention Head with Max Variance | 0.71 | 1.2 |
| First 1/2 Layers Average | 0.54 | 0.5 |
| Layer-Averaged (Halo) | 0.88 | 3.2 |
Table 6: Predictive Power of Different Entropy Aggregations. We report the Pearson Correlation ( ) with semantic drift and the Detection Lead Time (in tokens) before logic collapse on Omni-MATH.
| Dataset | Tier | Domain | Samples | |
| GSM8K | 1 | Math | 1,319 | |
| MATH (Lvl 1-2) | 1 | Math | 2,000 | |
| Omni-MATH | 2 | Symbolic | 4,428 | |
| RULER (32k) | 2 | Context | 1,000 |
Table 7: Statistical Characteristics of Evaluation Benchmarks. denotes average token length of the ground truth solution/context.
Figure 7: Universality of Phase Transition across Model Scales. (a) Mistral-7B exhibits an earlier and sharper performance cliff at , indicating higher sensitivity to stochastic drift. (b) Mixtral-8x7B extends the stable region to but shows increased variance in the transition zone due to MoE routing dynamics. The consistent existence of the “Performance Cliff” across different backbones validates that the critical reasoning horizon is an intrinsic property determined by model capacity.
Table 8: Sensitivity Analysis of Halo Hyperparameters. Success Rate (%) on Omni-MATH using LLaMA-3-8B. The results highlight the trade-off between over-correction (low ) and under-correction (high ). Bold indicates the optimal configuration.