Think Less, Locally: Local Preference Learning for Reasoning Compression

Muyao Yuan; Liyan Xu; Weizhan Zhang; Muyan Jiao; Jiangnan Li; Jinchao Zhang; Jie Zhou

cs.AINov 3, 2026

Think Less, Locally: Local Preference Learning for Reasoning Compression

Authors: Muyao Yuan, Liyan Xu, Weizhan Zhang, Muyan Jiao, Jiangnan Li, Jinchao Zhang, Jie Zhou

Organizations: MOEKLINNS, School of Computer Science and Technology, Xi’an Jiaotong University · WeChat AI, Tencent Inc.

Abstract

Large reasoning models achieve strong performance on challenging tasks, but their long chain-of-thought often incurs substantial inference costs by continuing to deliberate after sufficient information has been obtained. Existing reasoning compression methods primarily operate on reasoning trajectories, paying less attention to the local decision of whether further reasoning is necessary given the current state. We propose LoCo, a local preference learning method for reasoning compression. LoCo detects candidate overthinking states from the model's own traces, freezes the corresponding reasoning prefixes, regenerates shorter verified continuations from the same states, and trains on the resulting state-conditioned preference pairs with DPO. This formulation teaches the model when to continue reasoning and when to conclude, rather than simply encouraging shorter trajectories overall. We construct training data entirely from self-generated mathematical traces and evaluate LoCo on five mathematical and three code generation benchmarks. LoCo reduces completion length by 41.7%--45.4% on mathematics and 23.4%--36.3% on code, while maintaining or improving accuracy and achieving the best AUC across all code benchmarks. LoCo consistently improves the accuracy-efficiency trade-off on mathematics and transfers effectively to code generation despite math-only training, demonstrating the cross-domain generality of local preference learning for reasoning compression. The code and model will be publicly released at https://github.com/MuyaoYuan/LoCo.

Figures & tables

Figure 1: Overview of the LoCo pipeline. Starting from a self-generated reasoning trace, LoCo detects local overthinking states, regenerates shorter correct continuations from those states, and converts them into local preference pairs for training.

Method	MATH-500			AMC23			Olympiad
Method	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)
Base	91.88 $\pm$ 1.20	4015 $\pm$ 177	82.64 $\pm$ 1.12	90.31 $\pm$ 2.48	6285 $\pm$ 516	76.95 $\pm$ 2.07	61.33 $\pm$ 0.41	8972 $\pm$ 130	51.28 $\pm$ 0.37
DAST	91.38 $\pm$ 0.79	3720 $\pm$ 119	83.95 $\pm$ 0.66	88.12 $\pm$ 4.17	5981 $\pm$ 304	76.89 $\pm$ 3.22	59.91 $\pm$ 0.73	9155 $\pm$ 313	51.86 $\pm$ 0.59
DAST	(-0.50)	( $\downarrow$ 7.4%)	(+1.31)	(-2.19)	( $\downarrow$ 4.8%)	(-0.06)	(-1.42)	( $\uparrow$ 2.0%)	(+0.58)
TokenSqueeze	91.18 $\pm$ 0.89	2442 $\pm$ 52	85.92 $\pm$ 0.73	88.44 $\pm$ 4.62	4350 $\pm$ 582	79.42 $\pm$ 4.01	60.98 $\pm$ 0.95	6149 $\pm$ 105	54.29 $\pm$ 0.78
TokenSqueeze	(-0.70)	( $\downarrow$ 39.2%)	(+3.28)	(-1.87)	( $\downarrow$ 30.8%)	(+2.47)	(-0.35)	( $\downarrow$ 31.5%)	(+3.01)

Table 1: Mathematical reasoning results across five benchmarks of increasing difficulty. The table reports absolute performance and base-relative changes under a shared evaluation protocol. LoCo achieves the best overall trade-off, combining the strongest compression with consistently comparable or improved accuracy across difficulty levels.

Method	MBPP			LiveCodeBench			CodeContests
Method	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)
Base	84.05 $\pm$ 0.91	3403 $\pm$ 234	78.42 $\pm$ 0.83	41.77 $\pm$ 1.10	12319 $\pm$ 352	32.86 $\pm$ 0.87	18.48 $\pm$ 2.17	14900 $\pm$ 223	13.87 $\pm$ 1.50
DAST	81.13 $\pm$ 1.87	4097 $\pm$ 444	76.32 $\pm$ 1.68	39.82 $\pm$ 2.50	14158 $\pm$ 476	31.88 $\pm$ 1.66	16.06 $\pm$ 2.42	17757 $\pm$ 623	12.61 $\pm$ 1.79
DAST	(-2.92)	( $\uparrow$ 20.4%)	(-2.10)	(-1.95)	( $\uparrow$ 14.9%)	(-0.98)	(-2.42)	( $\uparrow$ 19.2%)	(-1.26)
TokenSqueeze	81.37 $\pm$ 2.18	2196 $\pm$ 131	77.40 $\pm$ 1.90	41.69 $\pm$ 2.22	9670 $\pm$ 200	35.03 $\pm$ 1.38	19.09 $\pm$ 0.97	11677 $\pm$ 305	15.54 $\pm$ 0.81
TokenSqueeze	(-2.68)	( $\downarrow$ 35.5%)	(-1.02)	(-0.08)	( $\downarrow$ 21.5%)	(+2.17)	(+0.61)	( $\downarrow$ 21.6%)	(+1.67)

Table 2: Code generation results on three benchmarks of increasing difficulty, evaluated after training on a math-only query pool. The table reports absolute performance and base-relative changes under a shared evaluation protocol. LoCo retains the strongest overall advantage, showing that the learned compression preference generalizes beyond the training domain.

Panel of Figure 2 — Figure 2: Accuracy versus token budget on AIME24 and CodeContests. LoCo achieves a better accuracy and efficiency trade-off than TokenSqueeze on both benchmarks.

Preference source
Method	AMC23			AIME24			AIME25
Method	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)	Acc (%)	Token	AUC (%)
Base	90.31 $\pm$ 2.48	6285 $\pm$ 516	76.95 $\pm$ 2.07	50.83 $\pm$ 6.61	13493 $\pm$ 1367	40.33 $\pm$ 5.00	37.92 $\pm$ 5.62	15134 $\pm$ 920	30.29 $\pm$ 3.82
Global	90.94 $\pm$ 2.65	5957 $\pm$ 562	77.66 $\pm$ 2.42	54.17 $\pm$ 5.56	12002 $\pm$ 870	42.57 $\pm$ 4.23	40.42 $\pm$ 5.47	12740 $\pm$ 1079	32.46 $\pm$ 3.90
Global	(+0.63)	( $\downarrow$ 5.2%)	(+0.71)	(+3.34)	( $\downarrow$ 11.1%)	(+2.24)	(+2.50)	( $\downarrow$ 15.8%)	(+2.17)
GPT-5.4 Cont.	90.62 $\pm$ 2.59	4083 $\pm$ 261	81.46 $\pm$ 2.17	52.92 $\pm$ 6.53	9204 $\pm$ 823	44.11 $\pm$ 5.54	39.58 $\pm$ 6.53	9690 $\pm$ 709	33.56 $\pm$ 4.78

Table 3: Ablation results on three representative mathematical reasoning benchmarks under matched training settings. The table compares alternative preference sources and cutpoint choices against the full LoCo design. LoCo yields the strongest overall compression–accuracy trade-off, indicating that both local preference construction and detector-selected cutpoints are important.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 388 papers

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
Apr 10, 2026Yi Sui, Chaozhuo Li, Dawei SongCoT Compression Efficient Language Model Reasoning
Beijing Institute of Technology, China · Beijing University of Posts and Telecommunications, China · The Open University, UK
ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
May 8, 2026Tingcheng Bian, Yuzhe Zhang, Jing Jin +5Reward Shaping RL for Language Model Reasoning
1Baidu Inc. · 2Shenzhen University · 3Peking University +2
Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
Aug 21, 2026Simeng Zhang, Yilong Chen, Wenyuan Zhang +4Efficient Language Model Reasoning CoT Compression
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Tencent Inc. +1

Apr 10, 2026J/K move · Enter open · S save

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning

Yi Sui, Chaozhuo Li, Dawei Song

Beijing Institute of Technology, China · Beijing University of Posts and Telecommunications, China · The Open University, UK

CoT Compression Efficient Language Model Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) achieve strong performance on complex tasks by leveraging long Chain-of-Thought (CoT), but often suffer from overthinking, leading to excessive reasoning steps and high inference latency. Existing CoT compression methods struggle to balance accuracy and efficiency, and lack fine-grained, step-level adaptation to redundancy and reasoning bias. Therefore, we propose State-Aware Reasoning Compression with Knowledge Guidance (STACK), a framework that performs step-wise CoT compression by explicitly modeling stage-specific redundancy sources and integrating with a retrieval-augmented guidance. STACK constructs online long-short contrastive samples and dynamically switches between knowledge-guided compression for uncertain or biased reasoning state and self-prompted compression for overly long but confident state, complemented by an answer-convergence-based early stopping mechanism to suppress redundant verification. We further propose a reward-difference-driven training strategy by combining Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO), enabling models to learn state-conditioned compression strategies. Experiments on three mathematical reasoning benchmarks show that STACK achieves a superior accuracy-efficiency balance, reducing average response length by 59.9% while improving accuracy by 4.8 points over existing methods.

Abstract

Efficient Language Model Reasoning

Explore similar work

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression

Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning