Overthinking in Language Models

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 29

All topics

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
Sep 30, 2026Chengzhu Bao, Xianglong Yan, Tianao Zhang +3LLM Quantization Overthinking in Language Models
Shanghai Jiao Tong University
When Reasoning Goes Astray: Attention Dynamics of Uncontrolled Reasoning
Sep 30, 2026Yuanhe Zhang, Ziwei Wang, Jie Ren +6Language Model Steering Overthinking in Language Models
Beijing University of Posts and Telecommunications · Wuhan University · Nanyang Technological University +1
Efficient Reasoning via Constrained Optimization in Latent Space
Sep 28, 2026Zhinan Hou, XingChen Li, Keyou YouOverthinking in Language Models Efficient Language Model Reasoning
Tsinghua University
LLM Parkinsonism: Executive-Control Failure, Token-Inefficient Persistence, and an Uncertainty-Aware Global Executive Control Architecture for Autonomous Language-Model Agents
Sep 25, 2026Dongsheng Xiao, Zeyuan Wang, Xuzhe Xia +2Overthinking in Language Models LLM Agents
Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia · School of Cyber Security, Northwestern Polytechnical University, Xi'an, Shaanxi, China · Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, British Columbia, Canada +2
When2Think: Learning When and How Much to Reason
Sep 17, 2026Jaejun Shim, HyunJin Kim, Young Jin Kim +1Overthinking in Language Models RL for Language Model Reasoning
Sungkyunkwan University · Microsoft
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
Aug 5, 2026Qiyuan Zhu, Dezhi Li, Pengyu Cheng +8Overthinking in Language Models KV Caching
1HKUST · 2XJTU · 3TSE +1
Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces
Aug 4, 2026Francesca Carlon, Vincent Ginis, Andres AlgabaLLM Evaluation Overthinking in Language Models
Data Analytics Lab, Vrije Universiteit Brussel, Pleinlaan 5, 1050 Brussels, Belgium · imec-SMIT, Vrije Universiteit Brussel, Pleinlaan 9, 1050 Brussels, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA
Same Task, Different Work: Prompt-Induced Waste in Coding Agents
Aug 2, 2026Sarel Weinberger, Amir HozezLLM Inference Efficiency Overthinking in Language Models
PointFive
Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning
Jul 31, 2026Xinyan Guan, Jiali Zeng, Chunlei Xin +5Overthinking in Language Models RL for Language Model Reasoning
1Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · 3Weixin AI, Tencent Inc, China
QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimization
Jul 23, 2026Siwei Chen, Siqi Chen, Xupeng Miao +1Overthinking in Language Models RL for Language Model Reasoning
School of Computer Science, Peking University · Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems · Department of Electronic Engineering, Tsinghua University +1
EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization
Jul 22, 2026Xinbang Dai, Zheyu Xin, Huikang Hu +7Overthinking in Language Models Self-Training for Language Models
Southeast University · Noah’s Ark Lab
Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
Jul 20, 2026Sheldon Yu, Tong Yu, Xunyi Jiang +6Overthinking in Language Models Causal Interventions in Language Models
UC San Diego · Adobe Research · University of New South Wales
UPAIR: Diagnosing Reasoning States via Uncertainty-Progress Alignment for Selective Intervention
Jul 19, 2026Cheng Yan, Zhijun Fan, Guangyang Ye +4Adaptive Inference Overthinking in Language Models
University of Science and Technology of China · AWS Agentic AI
Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought
Jul 13, 2026Daeyeop Lee, Hwanjo YuLLM Inference Efficiency Overthinking in Language Models
1KT Corporation · 2Pohang University of Science and Technology
OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
Jul 13, 2026Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2Overthinking in Language Models Sequential Decision Making
MIT · Columbia Business School
Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets
Jul 9, 2026Jack Hopkins, Dipika Khullar, Fabien RogerOverthinking in Language Models Privacy Leakage in Language Models
Anthropic Fellows Program · MATS · Anthropic.
Length Penalties Make Chain-of-Thought Less Monitorable
Jul 8, 2026Bryce LittleCoT Faithfulness Overthinking in Language Models
Overthink-Triggered Slowdown Attacks on LVLM-Based Robotic Systems
Jul 1, 2026Qiang Han, Jie Wu, Bo ChenAdversarial Attacks on VLMs Overthinking in Language Models
Department of Computer Science, Michigan Technological University
CAT: Confidence-Adaptive Thinking for Efficient Reasoning of Large Reasoning Models
Jul 1, 2026Qizhi Jiang, Shuo Wang, Pei Ke +2Overthinking in Language Models Efficient Language Model Inference
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province
Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking
Jul 1, 2026Chia-Hsuan Lee, Sihui Dai, Mingyang Zhou +4LLM Self-Correction Overthinking in Language Models
Capital One
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Jun 16, 2026Zihao Wei, Wenjie Shi, Liang Pang +8Overthinking in Language Models RL for Language Model Reasoning
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling
Jun 5, 2026Tengyao Tu, Yulin Li, Hui-Ling Zhen +6Overthinking in Language Models Efficient Language Model Reasoning
Harbin Institute of Technology, Shenzhen · Zhongguancun Academy · Huawei Noah’s Ark Lab +2
Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
Jun 1, 2026Simone Caldarella, Davide Talon, Rahaf Aljundi +2LLM Evaluation Overthinking in Language Models
University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe
Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
May 29, 2026Sanae Lotfi, Polina Kirichenko, Steven Li +1Overthinking in Language Models CoT Reasoning
1FAIR at Meta · 2Meta AI
Better, Faster: Harnessing Self-Improvement in Large Reasoning Models
May 24, 2026Qihuang Zhong, Liang Ding, Juhua Liu +3Overthinking in Language Models RL for Language Model Reasoning
1Nanyang Technological University, Singapore · 2Alibaba Group · School of Computer Science, Wuhan University, China +1
Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models
May 13, 2026Shuqiang Wang, Wei Cao, Jiaqi Weng +4Overthinking in Language Models Adversarial Attacks on LLMs
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Alibaba Group
Hint Tuning: Less Data Makes Better Reasoners
May 9, 2026Siqi Fan, Minghao Li, Xiaoqian Ma +6Overthinking in Language Models LLM Fine-Tuning
University of Electronic Science and Technology of China · Xiaohongshu Inc. · National University of Singapore
Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training
May 8, 2026Chen Wang, Hexuan Deng, Yining Zhang +5Overthinking in Language Models Reinforcement Learning with Verifiable Rewards
College of Software, Nankai University · 2Zhongguancun Academy · 3Harbin Institute of Technology +4
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language Models LLM Interpretability
State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

Chengzhu Bao, Xianglong Yan, Tianao Zhang, Jiaqi Chen+2

Shanghai Jiao Tong University

LLM Quantization Overthinking in Language Models Inference-Time Optimization CoT Reasoning Post-Training Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LLMs). However, recent studies reveal that when applied to reasoning models, PTQ not only degrades reasoning performance but also exacerbates overthinking, leading to longer reasoning trajectories. These issues may offset the efficiency gains expected from lower-precision inference. Existing approaches mainly rely on complex optimization procedures. More recent lightweight inference strategies instead use predefined overthinking markers, limiting their adaptability across quantized models. To address these issues, we propose Reasoning Analysis and Token-level Inference Optimization (RATIO), a framework that identifies model-specific overthinking tokens and assigns each a tailored penalty. RATIO first introduces Quantization-aware Reasoning Behavior Analysis (QRBA) to identify overthinking tokens by analyzing discrepancies between full-precision and quantized models. It then adopts Token-Specific Penalty Determination (TSPD), which leverages full-precision guidance to derive token-specific penalties without additional training. Extensive experiments show that RATIO achieves a better accuracy-efficiency trade-off than existing token-level interventions. Specifically, RATIO achieves up to 9.8 points accuracy improvement and reduces chain-of-thought (CoT) length by up to 51.3% compared with quantized baselines. The code will be available at https://github.com/steven-bao1/RATIO.

Momentum

Related topics

Latest papers 29