Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning

Tao Jing; Chen Kang; Feifei Zhai; Pengyuan Liu; Dong Yu

cs.CLOct 25, 2026

Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning

Authors: Tao Jing, Chen Kang, Feifei Zhai, Pengyuan Liu, Dong Yu

Abstract

Large reasoning models are costly to serve because they are large and because they generate long chains of thought (CoT). Weight pruning and CoT compression address these two costs separately, but they interfere in that pruning adds redundant reasoning on problems the model still solves and incomplete reasoning on those it no longer solves, and on harder problems makes the model less compressible by subsequent length-preference optimization. We trace this to a blind spot on each side: pruning criteria do not distinguish necessary from redundant reasoning, and preference-based CoT compression receives no training signal on problems a pruned model always or never solves. We propose Contrastive Set-Difference (CSD), which identifies neurons enriched in redundant reasoning by contrasting their activations on long and short correct solutions of the same problem and prunes them preferentially, and we extend preference pairs to the problems a pruned model always or never solves. Pruning 20% of DeepSeek-R1-Distill-Qwen-7B, the combination uses 8--22% fewer thinking tokens on four math benchmarks than pruning with reasoning-aware calibration (RAC) followed by the same compression, with accuracy within 0.5 points.

Figures & tables

Pairs	Problems	Rejected $y_{r}$
Self	$\tilde{\mathcal{C}}\neq\emptyset,\ \tilde{\mathcal{F}}\neq\emptyset$	$y\in\tilde{\mathcal{F}},\ \|y\|\geq\|y_{c}\|$
Dense	$\tilde{\mathcal{C}}=\emptyset\neq\mathcal{C}$	$y\in\tilde{\mathcal{F}},\ \|y\|\geq\|y_{c}\|$
Length	no self pair, $\|\tilde{\mathcal{C}}\|\geq 2$	$y\in\tilde{\mathcal{C}},\ \|y\|\geq\gamma\|y_{c}\|$

Table 1: Preference pairs: self, dense-anchored (Dense), and length-anchor (Length). Length-anchor pairs use problems without self pairs on which the pruned model has at least two correct solutions. The chosen response

y_{c}

is the pruned model’s shortest correct solution (

k

shortest for self pairs), except for dense-anchored pairs, where it is the dense model’s shortest correct solution. Self pairs target both redundant and incomplete reasoning, dense-anchored pairs target incomplete reasoning, and length-anchor pairs target redundant reasoning.

		MATH-500		GSM8K		AIME24		AIME25
Model	Params	Acc	Think	Acc	Think	Acc	Think	Acc	Think
Dense
Dense	7.62B	92.9	3274	87.6	386	55.8	12781	43.1	14318
TokenSqueeze-7B	7.62B	89.8	1757	85.2	128	54.0	9378	38.3	10205
Dense + PLPO	7.62B	90.7	1810	84.6	141	54.2	8357	41.5	9142
Pruned, before compression

Table 2: Main results. Acc is pass@1 (%) and Think the mean number of thinking tokens, with samples truncated before </think> counted at the budget. Pruned models remove 20% of decoder linear parameters, all from MLP neurons.

Benchmark	$\Delta$ Acc	$\Delta$ Think	$\Delta$ Geo		$\Delta$ Acc	$\Delta$ Think	$\Delta$ Geo
	Before PLPO				After PLPO
MATH-500	$-0.4$	$-2.9\%$	$-4.4\%$	( $-3.0\sigma$ )	$-0.5$	$-11.1\%$	$-14.5\%$	( $-7.3\sigma$ )
GSM8K	$0.0$	$-8.0\%$	$-7.7\%$	( $-6.6\sigma$ )	$-0.4$	$-21.7\%$	$-14.3\%$	( $-13.3\sigma$ )
AIME24+25	$+0.3$	$-0.2\%$	$-0.1\%$	( $-0.05\sigma$ )	$+0.1$	$-9.6\%$	$-11.0\%$	( $-3.6\sigma$ )

Table 3: CSD relative to RAC-S before and after PLPO.

\Delta

Acc is the difference in pass@1 in points and

\Delta

Think the relative change in mean thinking length, both computed from Table 2 .

\Delta

Geo is the geometric mean over problems of the ratio of per-problem mean thinking lengths, minus one, computed from the per-problem results; in parentheses, the mean per-problem log ratio divided by its standard error over problems, which covers evaluation sampling only, not variation between training runs. AIME pools the two years with equal weight per problem.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 371 papers

CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
Apr 19, 2026Yangsong Lan, Hongliang Dai, Piji LiLLM Pruning LLM Compression
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.
OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
Jul 13, 2026Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2Efficient Language Model Reasoning Sequential Decision Making
MIT · Columbia Business School
DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning
Oct 13, 2026Minghao Luo, Liang ChenEfficient Language Model Reasoning CoT Compression
The Chinese University of Hong Kong · EPFL

Apr 19, 2026J/K move · Enter open · S save

CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning

Yangsong Lan, Hongliang Dai, Piji Li

College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.

LLM Pruning LLM Compression LLM Inference Acceleration CoT Compression

Open paper AskPDF arXiv

Long Chain-of-Thought (CoT) reasoning is pivotal for the success of recent reasoning models but suffers from high computational overhead and latency. While prior works attempt to compress CoT via external compressor, they often fail to align with the model's internal reasoning dynamics, resulting in the loss of critical logical steps. This paper presents Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning (CRISP), a framework that compresses CoT by exploiting the model's intrinsic saliency. Our analysis reveals a distinct phenomenon: the reasoning termination token [object Object] acts as an information anchor, where its attention pattern effectively demarcates essential reasoning from redundancy. Based on this finding, we design a policy that utilizes these intrinsic attention signals to guide atomic compression operations. In contrast to coarse-grained pruning strategies, CRISP strategically distills the reasoning chain to maximize information density while preserving logical coherence. Empirical results across various backbone models and mathematical datasets demonstrate that CRISP achieves a 50-60% reduction in token count without compromising accuracy, effectively mitigating the efficiency bottleneck of long-context reasoning. We open-source our implementation to facilitate further research in efficient reasoning.

Abstract

Efficient Language Model Reasoning

Explore similar work

CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning