InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning

Chengwei Wei; Jung-jae Kim; Longyin Zhang; Shengkai Chen; Nancy F. Chen

cs.AIMar 18, 2026

InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning

Authors: Chengwei Wei, Jung-jae Kim, Longyin Zhang, Shengkai Chen, Nancy F. Chen

Organizations: Institute for Infocomm Research (I2R), A*STAR, Singapore · Centre for Frontier AI Research (CFAR), A*STAR, Singapore

Abstract

Large Language Models (LLMs) with extended reasoning capabilities often generate verbose and redundant reasoning traces, incurring unnecessary computational cost. While existing reinforcement learning approaches address this by optimizing final response length, they neglect the quality of intermediate reasoning steps, leaving models vulnerable to reward hacking. We argue that verbosity is not merely a length problem, but a symptom of poor intermediate reasoning quality. To investigate this, we conduct an empirical study tracking the per-token predictive entropy of large reasoning models across reasoning trajectories. We find that high-quality reasoning traces exhibit two consistent properties: low uncertainty convergence and fast uncertainty descent. These findings suggest that high-quality reasoning traces are informationally dense, that is, reasoning steps contribute to reaching a low uncertainty level relative to the total reasoning length. Motivated by this, we propose InfoDensity, a reward framework for RL training that captures both properties through a single suffix-max envelope of the entropy trajectory, weighted by a length scaling term that favors achieving equivalent quality more concisely. Experiments on mathematical and general reasoning benchmarks demonstrate that InfoDensity outperforms state-of-the-art baselines on the accuracy-efficiency trade-off.

Figures & tables

Dataset	#Questions	%Correct	Avg. Steps
GSM8K	400	48.3	3.9
MATH	117	43.6	4.7
OlympiadBench	1000	33.9	5.8
Omni-MATH	999	24.1	4.8

Table 1: Statistics of the ProcessBench subset used for step-level and trajectory-level analysis.

Figure 1: Step-level uncertainty reduction (

\Delta H_{t}

) distributions for correct (blue) and first-incorrect (red) steps in the ProcessBench subset across four models. Dotted vertical lines mark the per-class means.

Figure 2: Step-level entropy trajectory across four datasets: GSM8K, MATH, OlympiadBench, and Omni-MATH. Each curve shows the mean entropy over normalized reasoning steps, with shaded regions indicating

\pm 1

standard error.

Method	AMC23			AIME24			MATH500			GPQA-D			Overall
Method	Acc $\uparrow$	Len $\downarrow$	AES $\uparrow$	Acc $\uparrow$	Len $\downarrow$	AES $\uparrow$	Acc $\uparrow$	Len $\downarrow$	AES $\uparrow$	Acc $\uparrow$	Len $\downarrow$	AES $\uparrow$	Acc $\uparrow$	Len $\downarrow$	AES $\uparrow$
DeepSeek-R1-Distill-Qwen-1.5B
Original	50.0	9.6k	0.00	20.0	14.3k	0.00	69.6	6.2k	0.00	6.6	14.4k	0.00	36.5	11.1k	0.00
DEER	57.5	7.0k	0.72	23.3	11.8k	0.67	65.6	3.1k	0.21	5.6	12.5k	-0.63	38.0	8.6k	0.35
NoThink	62.5	2.4k	1.49	6.7	7.6k	-2.86	71.8	1.3k	0.89	9.1	1.5k	2.03	37.5	3.2k	0.79
LCPO	67.5	7.6k	1.25	30.0	12.3k	1.64	74.2	5.2k	0.36	8.6	13.5k	0.97	45.1	9.7k	0.83

Table 2: Evaluation results on four benchmarks across three models.

\uparrow

\downarrow

indicate higher/lower-is-better. The best AES per dataset and overall within each model block is bolded . The second best is underlined .

Base Model	After InfoDensity	Share	Mean length
Correct ( $n{=}387$ )	Preserved ( $\checkmark$ )	92.2%	$2471\to 1622$ ( $-34\%$ )
Correct ( $n{=}387$ )	Regression ( $\times$ )	7.8%	$4684\to 10323$ ( $+120\%$ )
Wrong ( $n{=}381$ )	Rescued ( $\checkmark$ )	27.6%	$15228\to 2700$ ( $-82\%$ )
Wrong ( $n{=}381$ )	Still wrong ( $\times$ )	72.4%	$14954\to 12159$ ( $-19\%$ )

Table 3: Effect of InfoDensity on DSR-1.5B, pooled across four benchmarks. “Mean length” shows the mean response tokens change under

16384

-token generation cap.

Figure 3: Length-scaling ablation on DSR-1.5B trained on DeepMath. Each trajectory traces the joint evolution of validation accuracy (

y

-axis) and average response length (

x

-axis) across training checkpoints, for four values of the length-scaling coefficient

\lambda

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Per-step distributions of

\Delta H_{t}

for correct (blue) and first-incorrect (red) steps in the ProcessBench subset, across the full 5-model panel: DeepSeek-R1-Distill-Qwen-1.5B, Llama-3.2-3B, Gemma-3-4B-pt, Qwen3-4B, and Qwen3-30B-A3B. Dotted vertical lines mark the per-class means; paired Cohen’s

d

values (printed in each panel) are uniformly small (

|d|\leq 0.12

Figure 5: ROC curves for

\Delta H_{t}

as a binary classifier of step correctness (positive class: correct steps; negative class: first-incorrect steps) in the ProcessBench subset across the full 5-model panel. All five curves cluster tightly around the chance diagonal (AUC

\in[0.489,0.531]

), confirming that step-level

\Delta H_{t}

is too noisy to serve as a robust error detector.

Figure 6: Step-level entropy trajectory of all five base models across four datasets: GSM8K, MATH, OlympiadBench, and Omni-MATH. Each curve shows the mean entropy over normalized reasoning steps, with shaded regions indicating

\pm 1

standard error.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 370 papers

Interpreting Reasoning of Large Language Models via Partial Information Decomposition
Sep 30, 2026Barproda Halder, Qiuyi Zhang, Sanghamitra DuttaLLM Interpretability Efficient Language Model Reasoning
University of Maryland, College Park · Elorian AI
ERR+: Sequential Entropy Resolution for Efficient and Decisive LLM Reasoning
Aug 28, 2026Xin Jiang, Minhao Wang, Wen Wu +4Reward Shaping RL for Language Model Reasoning
School of Computer Science and Technology, East China Normal University, Shanghai 200241, China · State Key Laboratory of Estuarine and Coastal Research, School of Computer Science and Technology, East China Normal University, Shanghai 200241, China
Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Apr 29, 2026Zhenyu Zhao, Sander Land, Daniel M. Bikel +1LLM Compression LLM Interpretability
Writer, Inc.

Sep 30, 2026J/K move · Enter open · S save

Interpreting Reasoning of Large Language Models via Partial Information Decomposition

Barproda Halder, Qiuyi Zhang, Sanghamitra Dutta

University of Maryland, College Park · Elorian AI

LLM Interpretability Efficient Language Model Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large reasoning models (LRMs) have achieved substantial improvements in solving complex mathematical problems, but often produce lengthy, repetitive, or erroneous reasoning trajectories. In this work, we introduce a new interpretability framework, SLIDER, to evaluate the quality of the reasoning process. SLIDER leverages an emerging body of work from information theory called Partial Information Decomposition to disentangle the information about the final answer between two consecutive reasoning steps into non-negative components: unique information (in preceding steps or current step), redundant information, and synergistic information. Building on this decomposition, we propose the Step-wise Repetitive Reasoning Index (Step-RRI), a theoretically grounded measure that assesses whether the answer-relevant information in the current step

S_i

is predominantly redundant with the past steps

S_{<i}

, relative to its unique and synergistic contributions. To evaluate the effectiveness of Step-RRI in detecting repetitiveness, we apply SLIDER to the redundancy class of the PRMBench dataset where Step-RRI improves step-level redundancy identification accuracy by over

10

points compared to embedding-similarity and information-gain baselines. Next, we define Trajectory-RRI, an aggregate measure of repetitiveness for an individual reasoning trajectory. To demonstrate its practical relevance, we show that average Trajectory-RRI strongly correlates with actual reasoning length across QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and GPT-4.1, motivating its use as a signal for improving reasoning efficiency. Finally, we introduce Trajectory-RRI-guided data selection for fine-tuning, demonstrating that selecting training data based on Trajectory-RRI can improve a fine-tuned model's reasoning efficiency while largely preserving its task performance.

Abstract

Efficient Language Model Reasoning

Explore similar work

Interpreting Reasoning of Large Language Models via Partial Information Decomposition

ERR+: Sequential Entropy Resolution for Efficient and Decisive LLM Reasoning

Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens