CoT Compression

CoT: Chain-of-Thought

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 37

All topics

BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking
Feb 5, 2026Bowen Yu, Sheng Zhang, Binhao Wang +8CoT Distillation Mathematical Reasoning
1City University of Hong Kong · 2Mohamed Bin Zayed University of Artificial Intelligence
ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure
Feb 1, 2026Jie Deng, Shining Liang, Jun Li +2Efficient Language Model Reasoning LLM Inference Acceleration
University of Chinese Academy of Sciences, Beijing, China · 2Microsoft, Beijing, China · 3Tongji University, Shanghai, China
CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization
Nov 7, 2025Ziqian Bi, Yinzhi Wang, Tianyang Wang +6Efficient Language Model Reasoning LLM Inference Acceleration
1Purdue University, USA · 2Baruch College, City University of New York, USA · 3The Ohio State University, USA +6
Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression
Oct 2, 2025Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad +3RL for Language Model Reasoning LLM Inference Acceleration
UNC Chapel Hill · The University of Texas at Austin · Microsoft Research
SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
Sep 17, 2025Kerui Huang, Shuhan Liu, Xing Hu +3Efficient Language Model Reasoning CoT Compression
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China · State Key Laboratory for Novel Software Technology, Nanjing University, China
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
Aug 4, 2025Taihang Zhen, Jialiang Hong, Kai Chen +12RL for Language Model Reasoning CoT Compression
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China · China Mobile (Suzhou) Software Technology Co., Ltd., Suzhou 215000, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China +2
Adaptive GoGI-Skip: Coupling Goal-Gradient Importance with Dynamic Uncertainty for Efficient Reasoning
May 13, 2025Ren ZhuangGradient-Based Attribution LLM Pruning
School of Information Science and Technology, Hangzhou Normal University

Feb 5, 2026, cs.LGJ/K move · Enter open · S save

BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

Bowen Yu, Sheng Zhang, Binhao Wang, Yi Wen+7

1City University of Hong Kong · 2Mohamed Bin Zayed University of Artificial Intelligence

CoT Distillation Mathematical Reasoning Teacher-Student Learning CoT Compression

Open paper AskPDF arXiv

Chain-of-Thought (CoT) reasoning has significantly improved LLMs' mathematical problem-solving capabilities, but distilling such capabilities into smaller models remains challenging due to the capacity mismatch between verbose teachers and compact students. Directly copying teachers' lengthy reasoning chains causes capacity overload, resulting in truncated outputs or repetitive failure. Existing remedies each sacrifice a critical property of CoT: implicit reasoning methods (e.g., compressing reasoning into hidden states) trade away interpretability and verifiability, while heuristic compression strategies (e.g., random step pruning) destroy logical integrity. To address this, we propose BRIDGE, a curriculum framework that first establishes structural understanding via masked reconstruction, then uses GRPO-based reinforcement learning to guide students in self-discovering the optimal balance between accuracy and brevity, and finally internalizes complex reasoning through teacher-guided rewriting on failure cases. On GSM8K, BRIDGE enables Qwen2.5-3B to achieve 11.29% accuracy improvement and 27.4% token reduction over the original model, outperforming instruction-tuned variants and distillation baselines. Zero-shot transfer experiments on SVAMP and MATH-500 further confirm the generalization of internalized reasoning. Our code and model checkpoints are publicly available at https://github.com/Applied-Machine-Learning-Lab/SDM2026_BRIDGE and https://huggingface.co/bowen0815/BRIDGE.

Momentum

Related topics

Latest papers 37