Learning to Reason Efficiently with Interventional Step Credit

Guanda Li; Xuefeng Bai; Yulong Chen; Zhiyong Wang; Kehai Chen; Min Zhang

cs.CLOct 19, 2026

Learning to Reason Efficiently with Interventional Step Credit

Authors: Guanda Li, Xuefeng Bai, Yulong Chen, Zhiyong Wang, Kehai Chen, Min Zhang

Organizations: Harbin Institute of Technology, Shenzhen · University of Aberdeen · University of Cambridge

Abstract

Large reasoning models often produce lengthy chain-of-thought (CoT) traces mixing useful intermediate steps with unnecessary ones. Recent methods explore step-level signals such as attention mass and step entropy to identify and suppress unnecessary steps. Despite their effectiveness, such signals reflect the model's behavior at a step rather than that step's contribution to the answer. To address this, we propose the interventional step credit (ISC), which takes a causal-intervention view and assesses step importance to the reasoning through targeted parameter-level interventions. Building on ISC, we further propose IGPO, which uses ISC to guide fine-grained advantage rescaling in reinforcement learning, reducing redundant steps while preserving the important ones. Experiments across two model scales and nine benchmarks show that IGPO achieves up to 62% average token reduction and better overall performance among compared methods.

Figures & tables

Figure 1: Answer accuracy on DS-7B under post-hoc CoT pruning. Deleting low-scoring steps identified by ISC better preserves accuracy than compared methods.

Figure 2: The overall framework of ISC and IGPO

Figure 3: Natural and post-unlearning answer flip rates for low- and high-ISC steps on the 1.5B and 7B models.

Figure 4: Accuracy on unrelated questions after unlearning low- and high-ISC steps in the 1.5/7B models.

Figure 5: Predictor performance on the validation set for labels derived from the 1.5B/7B models. [email protected] denotes balanced accuracy at a threshold of 0.5.

Methods	GSM8K			MATH500			AMC2023			Olympiad			AIME2024			AIME2025			Macro Avg
Methods	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES
1.5B Model
DS-1.5B	78.9	1018	0.00	81.7	4670	0.00	67.5	7356	0.00	48.6	9066	0.00	27.6	12327	0.00	22.9	12086	0.00	54.5	7754	0.00
$\heartsuit$ AutoThink	83.0	568	0.60	84.0	2195	0.62	67.0	5059	0.28	51.6	5053	0.63	34.6	9514	0.99	21.8	7944	0.10	57.0	5056	0.54
$\heartsuit$ AdaptThink	83.1	480	0.69	82.0	1782	0.63	67.0	3510	0.49	50.8	3708	0.73	31.0	6670	0.83	23.0	7255	0.41	56.1	3901	0.63
$\heartsuit$ DEER ‡	74.7	984	-0.23	67.8	2497	-0.38	60.0	5496	-0.30	32.0	5960	-1.37	23.3	9557	-0.55	10.0	9281	-2.58	44.6	5629	-0.90

Table 1: Acc., Tok. and AES across six mathematical reasoning benchmarks. Bold indicate the best results and underlined indicate the second-best results.

\heartsuit

\spadesuit

, and

\clubsuit

denote adaptive reasoning, outcome-supervised RL, and process-supervised RL methods, respectively. † Step Entropy counts thinking tokens only, so its Tok. and AES are excluded from highlighting. ‡ DEER uses greedy decoding.

Model	LiveCodeBench			GPQA			MMLU
Model	Acc.	Tok.	AES	Acc.	Tok.	AES	Acc.	Tok.	AES
1.5B Model
DS-1.5B	24.0	10286	0.00	27.9	9603	0.00	52.1	2650	0.00
Ours	28.6	6656	0.93	33.1	4889	1.05	52.9	1357	0.53
7B Model
DS-7B	48.1	8905	0.00	49.2	8153	0.00	71.6	2067	0.00

Table 2: OOD results on LiveCodeBench v6, GPQA Diamond, and MMLU. Best results are in bold .

Figure 6: Ablation and sensitivity analysis on DS-1.5B. (a) Intervention-method ablation: simple perturbation (Perturb.) versus unlearning (IGPO). (b) Sensitivity to the positive-label ratio

\rho

. Values inside token bars denote AES. Blue bars indicate IGPO with the default

\rho=40\%

Figure 7: Average validity and redundancy scores (

\times 100

) for 7B models, evaluated with ReasonEval-7B.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Stage	Setting	Value
ISC annotation	Updated modules	LoRA ( Hu et al., 2022 ) in down_proj
	High-ISC proportion $\rho$	Global $0.4$
	LoRA rank	$32$
	Learning rate and update count	$5\times 10^{-4}$ , 5 updates per target step
	Optimizer	AdamW
	NPO $\beta$ and retain weight $\lambda_{\mathrm{KL}}$	$\beta=0.1$ , $\lambda_{\mathrm{KL}}=1.0$

Appendix

Table 3: Training hyperparameters.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 371 papers

Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning
Sep 22, 2026Yuqing Zhou, Hong Wang, Manqing Mao +8RL for Language Model Reasoning Step-Level Credit Assignment in RL
George Mason University · Amazon, Inc
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
Feb 27, 2026Xintong Li, Sha Li, Rongmei Lin +10Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · Amazon
IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
Feb 22, 2026Yinhan He, Yaochen Zhu, Mingjia Shi +5Efficient Language Model Reasoning Language Model Post-Training
University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA

Sep 22, 2026J/K move · Enter open · S save

Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning

Yuqing Zhou, Hong Wang, Manqing Mao, Zhuoer Wang+7

George Mason University · Amazon, Inc

RL for Language Model Reasoning Step-Level Credit Assignment in RL Efficient Language Model Reasoning LLM Mathematical Reasoning

Open paper AskPDF arXiv

Large reasoning models can produce correct yet unnecessarily long reasoning traces. Existing methods improve reasoning efficiency with trajectory-level objectives or local token- and step-level signals, but rarely model inter-step semantic dependencies. This limits their ability to distinguish redundant steps from those that support later deductions, making it harder to shorten reasoning without sacrificing accuracy. We introduce RECAP (REdundancy-aware Credit Assignment via Propagation), which addresses this limitation by assigning credit where it is due based on both a step's downstream role in the reasoning structure and its contribution to solving the problem correctly. We define structural responsibility to capture the step's downstream role by measuring how strongly later reasoning depends on it, using credit propagated backward from the final-answer node through an outcome-independent, LLM-annotated semantic dependency graph. However, a step can have high structural responsibility yet steer the reasoning away from the correct solution. RECAP therefore introduces step efficacy to measure answer-directed progress through changes in gold-answer log-likelihood as each step is added. Together, these signals reshape rollout-level GRPO advantages into step-specific updates. RECAP requires neither a separately trained process reward model nor preconstructed concise trajectories. Across two 7B models and four mathematical reasoning benchmarks, RECAP improves the accuracy-efficiency trade-off. On Qwen2.5-Math-7B, it improves pass@1 by 2.0-3.7 percentage points while reducing reasoning tokens by 8%-31% relative to GRPO across all four benchmarks. Analysis suggests these savings reflect fewer reasoning operations and less dead-end reasoning, rather than more compact expression.

Abstract

Efficient Language Model Reasoning

Explore similar work

Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning