Large reasoning models are costly to serve because they are large and because they generate long chains of thought (CoT). Weight pruning and CoT compression address these two costs separately, but they interfere in that pruning adds redundant reasoning on problems the model still solves and incomplete reasoning on those it no longer solves, and on harder problems makes the model less compressible by subsequent length-preference optimization. We trace this to a blind spot on each side: pruning criteria do not distinguish necessary from redundant reasoning, and preference-based CoT compression receives no training signal on problems a pruned model always or never solves. We propose Contrastive Set-Difference (CSD), which identifies neurons enriched in redundant reasoning by contrasting their activations on long and short correct solutions of the same problem and prunes them preferentially, and we extend preference pairs to the problems a pruned model always or never solves. Pruning 20% of DeepSeek-R1-Distill-Qwen-7B, the combination uses 8--22% fewer thinking tokens on four math benchmarks than pruning with reasoning-aware calibration (RAC) followed by the same compression, with accuracy within 0.5 points.
Figures & tables
Pairs
Problems
Rejected yr
Self
C~=∅,F~=∅
y∈F~,∣y∣≥∣yc∣
Dense
C~=∅=C
y∈F~,∣y∣≥∣yc∣
Length
no self pair, ∣C~∣≥2
y∈C~,∣y∣≥γ∣yc∣
Table 1: Preference pairs: self, dense-anchored (Dense), and length-anchor (Length). Length-anchor pairs use problems without self pairs on which the pruned model has at least two correct solutions. The chosen response yc is the pruned model’s shortest correct solution ( k shortest for self pairs), except for dense-anchored pairs, where it is the dense model’s shortest correct solution. Self pairs target both redundant and incomplete reasoning, dense-anchored pairs target incomplete reasoning, and length-anchor pairs target redundant reasoning.
MATH-500
GSM8K
AIME24
AIME25
Model
Params
Acc
Think
Acc
Think
Acc
Think
Acc
Think
Dense
Dense
7.62B
92.9
3274
87.6
386
55.8
12781
43.1
14318
TokenSqueeze-7B
7.62B
89.8
1757
85.2
128
54.0
9378
38.3
10205
Dense + PLPO
7.62B
90.7
1810
84.6
141
54.2
8357
41.5
9142
Pruned, before compression
Table 2: Main results. Acc is pass@1 (%) and Think the mean number of thinking tokens, with samples truncated before </think> counted at the budget. Pruned models remove 20% of decoder linear parameters, all from MLP neurons.
Before PLPO
After PLPO
Benchmark
Δ Acc
Δ Think
Δ Geo
Δ Acc
Δ Think
Δ Geo
MATH-500
−0.4
−2.9%
−4.4%
( −3.0σ )
−0.5
−11.1%
−14.5%
( −7.3σ )
GSM8K
0.0
−8.0%
−7.7%
( −6.6σ )
−0.4
−21.7%
−14.3%
( −13.3σ )
AIME24+25
+0.3
−0.2%
−0.1%
( −0.05σ )
+0.1
−9.6%
−11.0%
( −3.6σ )
Table 3: CSD relative to RAC-S before and after PLPO. Δ Acc is the difference in pass@1 in points and Δ Think the relative change in mean thinking length, both computed from Table 2 . Δ Geo is the geometric mean over problems of the ratio of per-problem mean thinking lengths, minus one, computed from the per-problem results; in parentheses, the mean per-problem log ratio divided by its standard error over problems, which covers evaluation sampling only, not variation between training runs. AIME pools the two years with equal weight per problem.
Long Chain-of-Thought (CoT) reasoning is pivotal for the success of recent reasoning models but suffers from high computational overhead and latency. While prior works attempt to compress CoT via external compressor, they often fail to align with the model's internal reasoning dynamics, resulting in the loss of critical logical steps. This paper presents Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning (CRISP), a framework that compresses CoT by exploiting the model's intrinsic saliency. Our analysis reveals a distinct phenomenon: the reasoning termination token [object Object] acts as an information anchor, where its attention pattern effectively demarcates essential reasoning from redundancy. Based on this finding, we design a policy that utilizes these intrinsic attention signals to guide atomic compression operations. In contrast to coarse-grained pruning strategies, CRISP strategically distills the reasoning chain to maximize information density while preserving logical coherence. Empirical results across various backbone models and mathematical datasets demonstrate that CRISP achieves a 50-60% reduction in token count without compromising accuracy, effectively mitigating the efficiency bottleneck of long-context reasoning. We open-source our implementation to facilitate further research in efficient reasoning.
Yangsong Lan, Hongliang Dai, Piji Li
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.
Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting. However, these models often exhibit "computational overthinking," generating redundant reasoning steps that increase latency and cost without improving accuracy. Recent studies suggest that CoT trajectories can be significantly pruned, yet existing methods often rely on forcing a static thinking budget, heuristic filtering, sub-optimal early exit via classification, or expensive re-training. In this paper, we introduce OS-Pruner, a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem. Given a reasoning prefix, OS-Pruner learns whether further reasoning is worth its token cost by optimizing an explicit utility that trades off final-answer accuracy against generated length. Our novel formulation enables the model to dynamically assess the sufficient point of termination for a reasoning chain. OS-Pruner is designed to be lightweight during both training and inference, and to provide users with fine-grained control over the reasoning-effort vs. accuracy trade-off. On diverse reasoning benchmarks and base models, OS-Pruner achieves 20-60% reduction in generation length with minimal accuracy sacrifice.
Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2
Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.