Organizations: Plaksha University · Nanyang Technological University · Interview Street Technologies Pvt. Ltd. (HackerRank) · Indian Statistical Institute
We investigate whether variation in information density across chain-of-thought (CoT) reasoning steps predicts answer correctness and can serve as a controllable inference-time target. We prompt frozen pretrained models to solve problems via CoT and measure the density of each reasoning step with two independent metrics: LP variance (how unevenly information is distributed across steps, computed as the variance in mean per-token negative log-likelihood) and the Caveman compression ratio. We then test whether density correlates with, and can be steered to improve, answer correctness. Experiments across six model scales, two training paradigms, and 60 GSM8K problems yield four main findings. First, lower LP variance predicts correctness in supervised fine-tuned (SFT) models, but this relationship weakens with scale and reverses in reinforcement-learning (RL) distilled models. Training paradigm therefore appears more influential than model size in determining density behaviour. Second, synthetic experiments reveal an intrinsic complexity preference: models trained on uniformly distributed data learn complexity-favouring distributions 2.0 to 2.6 times more accurately than simplicity-favouring ones. Third, direct inference-time density steering reduces accuracy from 60.0% to 8.3%, indicating that natural density patterns are functionally important to reasoning. Fourth, pre-generation nudging combining hint injection with contrastive KV-cache steering improves accuracy by 13.3 percentage points, consistent across both metrics. Finally, we introduce Entropy-Gated Density Routing (EGDR), a self-supervised method using token-level Shannon entropy to gate proportional KVcache adjustments during generation, improving accuracy by 10.0 percentage points without any external guidance.
Figures & tables
Figure 1: Overview of the paper’s argument. (A) The complexity preference gap Δ grows 21.3% from GPT-2 XL (2019) to Mistral 7B (2023), driven by training quality rather than scale alone. (B) The density–correctness relationship depends on training paradigm: SFT models show uniform density in correct solutions (consistent with UID); RL-distilled models show structured non-uniformity. This determines the beneficial direction of any density-based intervention. (C) Naïve density steering collapses accuracy by 51.7 pp; principled pre-generation nudging (D: Hint+Cache) achieves +13.3 pp; EGDR achieves +10.0 pp without any external hint.
Comparison
Base Δ
Max Δ
Change
GPT-2 (124M → 1.5B)
0.3190
0.3378
+4.8%
Pythia (70M → 2.8B, ctrl.)
0.2935
0.3323
+13.2%
Cross-gen. (2019 → 2023)
0.3343
0.4054
+21.3%∗∗∗
Table 1: Complexity preference gap Δ across model families. The cross-generational effect ( +21.3% ) substantially exceeds the controlled within-family scaling effect ( +13.2% over 40× parameters in Pythia), implicating training methodology, not scale, as the primary driver. Full per-model results in Table 8 .
Corpus
KL (avg)
Bias
Spear. ρ
KL ratio
Uniform
0.0038
+0.094
0.60∗
–
Linear
0.0409
−0.50
0.964∗∗∗
1.0×
Inverse
0.0967
+0.77
0.21 n.s.
2.4×
Table 2: Synthetic experiment results (averaged across 2M–50M models). The Uniform KL (0.0038) replicates Ranga et al. (10) . The KL asymmetry ratio (Inverse/Linear =2.4× ) is consistent across all individual model sizes (range: 2.0–2.6 × ).
Model
Paradigm
Acc.
ΔLP
Regime
Qwen2.5-0.5B
SFT
26.7%
−0.029
Reversed
Qwen2.5-1.5B
SFT
51.7%
+0.020
UID
Qwen2.5-3B
SFT
60.0%
−0.003
Collapsed
Qwen2.5-7B
SFT
63.3%
−0.001
Collapsed
R1-Distill-1.5B
RL
66.7%
+0.013
Dual-density
R1-Distill-7B
RL
76.7%
+0.014
Dual-density
Table 3: Density–correctness relationship across model families. LP gap =Var(C)−Var(W) . Architecture, vocabulary, tokeniser, and inference procedure are identical between SFT and RL families at each scale; the reversal is attributable to training paradigm.
Table 5: Principled nudging results (Qwen2.5-1.5B-Instruct, N=60 ). ΔLP=Var(C)−Var(W) ; ΔCav=Cav(C)−Cav(W) (negative = correct solutions denser). Both signals agree in direction across all conditions.
Figure 2: EGDR applies a piecewise KV cache correction at every generated token based on the model’s own Shannon entropy Ht . High-entropy tokens (computationally dense moments, Ht>1.5Hˉ ) receive a positive correction +αd , reinforcing the reasoning trajectory. Low-entropy tokens (predictable filler, Ht<0.6Hˉ ) receive a small negative correction −0.3αd . This self-supervised gating requires no external input and never interrupts step-level generation. At α=0.30 , EGDR achieves +10.0 pp accuracy on GSM8K with the most statistically significant density footprint of any evaluated condition ( p=0.019 ).
α
Accuracy
vs. A
vs. D
0.05
58.3%
+6.6 pp
−6.7 pp
0.10
55.0%
+3.3 pp
−10.0 pp
0.15
56.7%
+5.0 pp
−8.3 pp
0.20
58.3%
+6.6 pp
−6.7 pp
0.30
61.7%
+10.0 pp
−3.3 pp
Table 6: EGDR coefficient grid search (Qwen2.5-1.5B-Instruct, N=60 ). Accuracy increases monotonically; absence of degradation at α=0.30 suggests the optimum lies above the tested range.
Cond.
Acc.
vs. A
ΔH
ΔCav
p
A: Baseline
51.7%
–
–
+0.006
–
B: Hint
46.7%
−5.0 pp
–
−0.008
–
C: Cache
53.3%
+1.6 pp
–
−0.027
–
D: Hint+Cache
65.0%
+13.3 pp
–
−0.007
–
F: EGDR
61.7%
+10.0 pp
−0.053
−0.033
0.019
G: Hint+EGDR
63.3%
+11.6 pp
−0.041
−0.024
0.021
Table 7: Complete results across all conditions. ΔH : mean entropy(correct) − mean entropy(wrong); negative = correct solutions more confident per token. EGDR (F) is the only condition reaching p<0.05 on ΔH .
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Model
Yr
Δ
Change
KS (next)
GPT-2 Small (124M)
2019
0.3190
–
p<0.01
GPT-2 Medium (355M)
2019
0.3378
+5.9%
n.s.
GPT-2 Large (774M)
2019
0.3343
+4.8%
n.s.
GPT-2 XL (1.5B)
2019
0.3343
+4.8%
–
Pythia 70M
2021
0.2935
–
p<0.01
Pythia 160M
2021
0.3063
+4.4%
p<0.01
Appendix
Table 8: Per-model complexity preference gap Δ . The Pythia two-stage plateau (rapid growth to 410M, near-stasis at 1B–2.8B) suggests a scaling law for complexity preference with a ceiling around 1B parameters in this architecture.
Large Language Models (LLMs) with extended reasoning capabilities often generate verbose and redundant reasoning traces, incurring unnecessary computational cost. While existing reinforcement learning approaches address this by optimizing final response length, they neglect the quality of intermediate reasoning steps, leaving models vulnerable to reward hacking. We argue that verbosity is not merely a length problem, but a symptom of poor intermediate reasoning quality. To investigate this, we conduct an empirical study tracking the per-token predictive entropy of large reasoning models across reasoning trajectories. We find that high-quality reasoning traces exhibit two consistent properties: low uncertainty convergence and fast uncertainty descent. These findings suggest that high-quality reasoning traces are informationally dense, that is, reasoning steps contribute to reaching a low uncertainty level relative to the total reasoning length. Motivated by this, we propose InfoDensity, a reward framework for RL training that captures both properties through a single suffix-max envelope of the entropy trajectory, weighted by a length scaling term that favors achieving equivalent quality more concisely. Experiments on mathematical and general reasoning benchmarks demonstrate that InfoDensity outperforms state-of-the-art baselines on the accuracy-efficiency trade-off.
Chengwei Wei, Jung-jae Kim, Longyin Zhang +2
Institute for Infocomm Research (I2R), A*STAR, Singapore · Centre for Frontier AI Research (CFAR), A*STAR, Singapore
Large Language Models often achieve strong performance by generating long intermediate chain-of-thought reasoning. However, it remains unclear when a model's final answer is actually determined during generation. If the answer is already fixed at an intermediate stage, subsequent reasoning tokens may constitute post-decision explanation, increasing inference cost and latency without improving correctness. We study the evolution of predicted answers over reasoning steps using forced answer completion, which elicits the model's intermediate predictions at partial reasoning prefixes. Focusing on Qwen3-4B and averaging results across all datasets considered, we find that predicted answers change in only 32% of queries. Moreover, once the final answer switch occurs, the model generates an average of 760 additional reasoning tokens per query, accounting for a substantial fraction of the total reasoning budget. Motivated by these findings, we investigate early stopping strategies that halt generation once the answer has stabilized. We show that simple heuristics, including probe-based stopping, can reduce reasoning token usage by 500 tokens per query while incurring only a 2% drop in accuracy. Together, our results indicate that a large portion of chain-of-thought generation is redundant and can be reduced with minimal impact on performance.
Ayan Datta, Zhixue Zhao, Bhuvanesh Verma +3
IIIT Hyderabad · University of Sheffield, UK · Goethe University, Frankfurt am Main, Germany
Large reasoning models achieve high accuracy through extended chain-of-thought but generate 5--8 more tokens than necessary, applying verbose reasoning uniformly regardless of problem difficulty. We propose Hint Tuning, a data-efficient approach that teaches models to calibrate reasoning depth. Our key insight: the corresponding instruct model serves as an ideal difficulty probe. By testing what the instruct model can solve with varying guidance, we automatically construct training data across three states: No-Hint (direct answer), Sparse-Hint (minimal prefix), and Full-Hint (complete reasoning). This converts the abstract challenge of difficulty labeling into a measurable consistency check between the instruct and reasoning models. With only 1K self-annotated samples, Hint Tuning achieves 24--66% token reduction (31.5% average) across mainstream reasoning models (Qwen3-Thinking, DeepSeek-R1-Distill) at multiple scales (4B--32B) while maintaining competitive accuracy on five benchmarks. Unlike methods requiring massive distillation datasets or expensive RL, we achieve superior efficiency through simple alignment with the instruct model's capabilities. Code and data are available at https://github.com/redai-infra/hint-tuning.
Siqi Fan, Minghao Li, Xiaoqian Ma +6
University of Electronic Science and Technology of China · Xiaohongshu Inc. · National University of Singapore