Efficient Language Model Reasoning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 188% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 166

Mar 9, 2026cs.CL

Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning

Large language models (LLMs) can achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet they often generate unnecessarily long reasoning paths that incur high inference cost. Self-consistency-based approaches push accuracy higher still, but they require sampling and aggregating multiple reasoning trajectories, leading to substantial computational overhead. In this paper, we introduce a confidence-aware selective sampling framework that, at inference time, analyzes a single reasoning trajectory to adaptively determine whether to rely on that trajectory alone or trigger multi-path sampling. The framework uses trajectory-level numeric features and sentence-level linguistic features extracted from reasoning states to guide selective multi-path reasoning. We train it on MedQA and evaluate it in-domain on MedQA and under calibration-only transfer on MathQA, MedMCQA, and MMLU, without further fine-tuning. Experimental results show that the proposed framework maintains comparable performance to full and efficient multi-path reasoning baselines, with accuracy changes of

-0.41 \pm 0.58

and

-0.31 \pm 0.58

percentage points, respectively, while reducing token usage by

71.7 \pm 5.0%

and

36.6 \pm 9.1%

. These findings demonstrate that reasoning trajectories contain rich signals for uncertainty estimation, enabling a simple, transferable mechanism to balance accuracy and efficiency in LLM reasoning.

Juming Xiong, Kevin Guo, Congning Ni +7

1Vanderbilt University · 2Vanderbilt University Medical Center · 3Intuit AI Research

CoT Reasoning Efficient Language Model Reasoning LLM Inference Acceleration Adaptive Sampling LLM Uncertainty Estimation LLM Reasoning+1

Mar 5, 2026cs.LG

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation

Reasoning models often generate far more tokens than a task requires, which raises inference cost and can compound errors. We introduce CRISP (Compressed Reasoning via Iterative Self-Policy Distillation), an on-policy self-distillation method that teaches a model to reason more concisely by distilling its own concise behavior back into itself. The method uses a single idea: condition the same model on a "be concise" instruction to obtain teacher logits, then minimize the per-token reverse KL divergence between the student and this teacher on the student's own rollouts. It requires no ground-truth answers, no token budgets, and no difficulty estimators. The reverse-KL objective is naturally difficulty-adaptive: it compresses easy problems aggressively while preserving the reasoning steps that hard problems require. On Qwen3-14B, CRISP cuts reasoning length by up to 56% on MATH-500 and 38% on the harder AIME 2024, while improving MATH-500 accuracy by up to 3.3 points over the base model and holding AIME 2024 accuracy within about one point. This behavior generalizes across model sizes and families: Qwen3-8B shows the same compression with accuracy preserved, and DeepSeek-R1-Distill-Llama-8B improves accuracy on all five benchmarks while shortening its responses. General capabilities are preserved across all three models. Code is available at https://github.com/HJSang/OPSD_Reasoning_Compression.

Hejian Sang, Yuanda Xu, Zhengze Zhou +3

CoT Distillation Efficient Language Model Reasoning On-Policy Distillation CoT Compression LLM Reasoning

Feb 13, 2026cs.CL

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

Large language models (LLMs) have demonstrated impressive reasoning capabilities by scaling test-time compute via long Chain-of-Thought (CoT). However, recent findings suggest that raw token counts are unreliable proxies for reasoning quality: increased generation length does not consistently correlate with accuracy and may instead signal "overthinking," leading to performance degradation. In this work, we quantify inference-time effort by identifying deep-thinking tokens -- tokens where internal predictions undergo significant revisions in deeper model layers prior to convergence. Across four challenging mathematical and scientific benchmarks (AIME 24/25, HMMT 25, and GPQA-diamond) and a diverse set of reasoning-focused models (GPT-OSS, DeepSeek-R1, and Qwen3), we show that deep-thinking ratio (the proportion of deep-thinking tokens in a generated sequence) exhibits a robust and consistently positive correlation with accuracy, substantially outperforming both length-based and confidence-based baselines. Leveraging this insight, we introduce Think@n, a test-time scaling strategy that prioritizes samples with high deep-thinking ratios. We demonstrate that Think@n matches or exceeds standard self-consistency performance while significantly reducing inference costs by enabling the early rejection of unpromising generations based on short prefixes.

Wei-Lin Chen, Liqian Peng, Tian Tan +5

University of Virginia · Google

LLM Inference Efficiency Inference-Time Scaling Efficient Language Model Reasoning LLM Reasoning

Feb 1, 2026cs.CL

ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

Large reasoning models (LRMs) typically solve reasoning-intensive tasks by generating long chain-of-thought (CoT) traces, leading to substantial inference overhead. We identify a reproducible inference-time phenomenon, termed Self-Compression: when multiple independent and answerable questions are presented within a single prompt, the model spontaneously produces shorter reasoning traces for each question. This phenomenon arises from multi-question contextual pressure during generation and consistently manifests across models and benchmarks. Building on this observation, we propose ConPress (Learning from Contextual Pressure), a lightweight self-supervised fine-tuning approach. ConPress constructs multi-question prompts to induce self-compression, samples the resulting model outputs, and parses and filters per-question traces to obtain concise yet correct reasoning trajectories. These trajectories are directly used for supervised fine-tuning, internalizing compressed reasoning behavior in single-question settings without external teachers, manual pruning, or reinforcement learning. With only 8k fine-tuning examples, ConPress reduces reasoning token usage by 59% on MATH500 and 33% on AIME25, while maintaining competitive accuracy.

Jie Deng, Shining Liang, Jun Li +2

University of Chinese Academy of Sciences, Beijing, China · 2Microsoft, Beijing, China · 3Tongji University, Shanghai, China

Efficient Language Model Reasoning LLM Inference Acceleration Mathematical Reasoning CoT Compression Large Reasoning Models

Jan 29, 2026cs.AI

Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and early token commitments in discrete reasoning traces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, many such methods optimize latent states end to end without a trained interface for intermediate textual readout, and several representative configurations use a pre-defined number of latent steps during inference. In this work, we introduce \textbf{PLaT} (\textbf{P}lanning with \textbf{La}tent \textbf{T}houghts), a framework that decouples latent planning from verbalization. The Planner deterministically evolves latent planning states, while an independent Decoder provides textual readouts when needed. Answer-aware textual stopping allows the latent rollout to use a problem-dependent number of groups rather than a pre-specified chain length. PLaT achieves competitive coverage at larger

k

in several mathematical settings, with lower Pass@1: on Llama-1B GSM8K, it reaches 80.59% Pass@128 versus CODI's 72.37%. These results support PLaT as a candidate-generation interface supplying multiple textual readouts for downstream verification or reranking.

Jiecong Wang, Hao Peng, Zhanyi Wang +2

Beihang University · Didi Chuxing

LLM Planning Efficient Language Model Reasoning Implicit Reasoning in Language Models Continuous Latent Reasoning Efficient Language Model Inference LLM Reasoning+1

Dec 24, 2025cs.CL

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation

Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge distillation (KD) over lengthy sequences with prompt (P), chain-of-thought (CoT), and answer (A) sections makes the process computationally expensive. In this work, we investigate how the allocation of supervision across different sections (P, CoT, A) affects student performance. Our analysis shows that selective KD over only the CoT tokens can be effective when the prompt and answer information is encompassed by it. Building on this insight, we establish a truncation protocol to quantify computation-quality tradeoffs as a function of sequence length. We observe that beyond a specific length, longer training sequences provide marginal returns for downstream performance but require substantially higher memory and FLOPs. To this end, training on only the first

50\%

of tokens of every training sequence can retain, on average,

\approx91\%

of full-sequence performance on math benchmarks while reducing training time, memory usage, and FLOPs by about

50\%

each. Codes are available at https://github.com/weiruichen01/distilling-the-essence.

Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5

1The University of British Columbia · 2LinkedIn · 3Canada Research Chair in NLP and ML

CoT Distillation Selective Knowledge Distillation Efficient Language Model Reasoning Efficient Language Model Training Teacher-Student Learning Knowledge Distillation+2

Dec 16, 2025cs.CL

Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring

The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling LRMs to reason longer, and more accurately. However, a growing body of studies show that LRMs are still inefficient, over-generating verification and reflection steps. To address this challenge, we introduce the Step-Tagging framework, a lightweight sentence-classifier enabling real-time annotation of the type of reasoning steps that an LRM is generating. To monitor reasoning behaviors, we introduced ReasonType: a novel taxonomy of reasoning steps. Building on this framework, we demonstrated that online monitoring of the count of specific steps can produce effective interpretable early stopping criteria of LRM inferences. We evaluate the Step-tagging framework on three open-source reasoning models across standard benchmark datasets: MATH500, GSM8K, AIME and non-mathematical tasks (GPQA and MMLU-Pro). We achieve 20 to 50% token reduction while maintaining comparable accuracy to standard generation, with largest gains observed on more computation-heavy tasks. This work offers a novel way to increase control over the generation of LRMs, and a new tool to study behaviors of LRMs.

Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1

1IBM Research Europe, Dublin, Ireland · 2Trinity College Dublin, Ireland · 3ADAPT Research Centre, Dublin, Ireland

LLM Inference Efficiency Efficient Language Model Reasoning Efficient Language Model Inference Large Reasoning Models Reasoning Trace Analysis

Nov 24, 2025cs.LG

ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models

Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but their inherently sequential decoding incurs substantial latency, motivating parallelization of the generation process. However, existing parallel reasoning approaches suffer from performance degradation compared to their sequential counterparts, and often rely on specialized inference engines. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that matches the accuracy of comparably sized sequential reasoning models while significantly reducing inference latency via three key innovations: 1) a two-stage parallel trajectory generator that produces high-quality parallel chain-of-thought data for supervised fine-tuning; 2) a trie-based rollout design that enables parallel reasoning on any off-the-shelf autoregressive inference engine; and 3) a parallelization-aware reinforcement learning framework that trains the model to balance reasoning accuracy with effective parallelization. Across six challenging math reasoning benchmarks, ThreadWeaver trained on top of Qwen3-8B achieves performance on par with cutting-edge sequential reasoning models (79.9% on AIME24 and 71.9% on average) while delivering up to 1.53x speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.

Long Lian, Sida Wang, Felix Juefei-Xu +7

Meta Superintelligence Labs (MSL), Menlo Park, CA, United States · UC Berkeley, Berkeley, CA, United States · UCSF, San Francisco, CA, United States

RL for Language Model Reasoning Inference-Time Scaling Efficient Language Model Reasoning LLM Inference Acceleration Large Reasoning Models

Nov 7, 2025cs.AI

CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization

Long Chain-of-Thought (CoT) traces can improve reasoning accuracy, but repeatedly generating them is costly for smaller or latency-constrained language models. This paper studies a practical alternative: produce a rich rationale once with a capable \emph{thinking} model, compress it, and reuse the compressed trace as context for a cheaper \emph{answering} model. We introduce CoT-X, an adaptive framework for cross-model CoT transfer. CoT-X segments reasoning traces into semantic units, scores their diagnostic and logical importance, selects budget-feasible evidence paths, and reconstructs a coherent compressed rationale for the answering model. On

7,501

Japanese medical licensing questions spanning

10

specialties, CoT-X improves accuracy over direct truncation by up to

40.5\%

under the same token budget, with the largest gains at

64

--

256

tokens. Across

64

thinking--answering pairs from eight DeepSeek-R1 and Qwen3 models (1.5B--32B parameters), reasoning transfer is most reliable within a model family, yet remains effective across families once compression normalizes the trace. A Gaussian Process Bayesian optimization layer finds near-optimal model--budget configurations with

15

evaluations rather than an exhaustive search over all

64

pairs, reducing evaluation cost by

84\%

. These results show that reasoning quality, token budget, and model compatibility can be optimized jointly, making CoT-style reasoning more practical under realistic deployment constraints.

Ziqian Bi, Yinzhi Wang, Tianyang Wang +6

1Purdue University, USA · 2Baruch College, City University of New York, USA · 3The Ohio State University, USA +6

Efficient Language Model Reasoning LLM Inference Acceleration CoT Compression LLM Reasoning Reasoning Trace Analysis

Oct 27, 2025cs.LG

Learning to Reason Efficiently with Discounted Reinforcement Learning

Large reasoning models (LRMs) often consume excessive tokens, inflating computational cost and latency. More broadly, in goal reaching sequential decision problems we often want to reach the goal quickly, and LRM reasoning can be viewed through this lens. We challenge the assumption that longer responses improve accuracy. By penalizing reasoning tokens using a discounted reinforcement learning setup (interpretable as a small token cost) and analyzing Blackwell optimality in restricted policy classes, we encourage concise yet accurate reasoning, analogous to preferring shorter successful trajectories in a stochastic shortest path problem. Experiments confirm our theoretical results that this approach shortens chains of thought while preserving accuracy.

Alex Ayoub, Kavosh Asadi, Dale Schuurmans +2

Amazon, University of Alberta · Amazon · University of Alberta

Reinforcement Learning RL for Language Model Reasoning Efficient Language Model Reasoning Policy Learning LLM Reasoning Large Reasoning Models

Sep 29, 2025cs.AI

Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models

Current answering paradigms for Large Reasoning Models (LRMs) often fail to account for the fact that some questions may lie beyond the model's operational capability boundary, leading to long but unproductive reasoning. In this paper, we study whether LRMs expose early signals predictive of such cases, and whether these signals can be used to mitigate unproductive reasoning. In black-box settings, we find that reasoning expressions contain failure-predictive signals. In white-box settings, we show that the hidden states of the last input token contain information that is predictive of whether a question will not be solved correctly under our evaluation setup. Building on these observations, we propose two test-time monitoring strategies: reasoning expression monitoring and hidden states monitoring, that reduce token usage by 62.7-93.6%, substantially improving efficiency and reliability while largely preserving accuracy.

Qingjie Zhang, Yujia Fu, Yang Wang +5

Tsinghua University · 2Ant Group

Efficient Language Model Reasoning Efficient Language Model Inference Large Reasoning Models

Sep 28, 2025cs.LG

Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

Many LLMs plan before they act, yet planning and execution are often still entangled in one long generation trace, enforced only through prompts, or split across separate components. We argue that these two stages call for different computation: planning benefits from diversity and breadth, whereas execution demands precision and faithful adherence to a chosen strategy. Treating them as a single undifferentiated chain wastes tokens on routine derivation and makes it costly to explore alternative strategies at test time. We present the \textbf{Explore-Execute Chain (E\textsuperscript{2}C)}, which keeps both stages in one model but separates them structurally: a stochastic \textit{Exploration} phase drafts a concise high-level plan, and a deterministic \textit{Execution} phase carries it out. Causal SFT and RL train this split so that exploration stays informative and execution remains plan-faithful. Once plans are short yet decisive, extra inference compute can be directed to exploration rather than to repeatedly decoding full solutions. On AIME'2024 at

K{=}32

, \textbf{E\textsuperscript{2}C-ReAct Loop} reaches 53.3% accuracy with only 12.4k tokens, outperforming Tree-of-Thoughts (

N{=}32

: 50.0%, 71.3k). The same structure also supports lightweight domain adaptation: \textbf{Exploration-Focused SFT (EF-SFT)} updates only the planning phase, uses 3.5% of the tokens required by standard SFT, and improves medical benchmark accuracy by up to 14.5%.

Kaisen Yang, Tinghe Zhang, Rushi Shah +4

1Qizhi Institute · 2Dept. of Computer Science, Tsinghua University · College of AI, Tsinghua University +4

LLM Inference Efficiency LLM Planning Test-Time Scaling Efficient Language Model Reasoning Structured Reasoning LLM Reasoning

Sep 26, 2025cs.CL

Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation

Distilling reasoning traces from strong teacher models has become the standard recipe for building capable small language models. Yet reasoning traces are 5-20

\times

longer than standard instruction fine-tuning (IFT) outputs, meaning every practitioner who chooses reasoning distillation implicitly forgoes training a larger IFT model on the same compute budget. Whether this trade-off is worthwhile remains unaddressed. We study it with a controlled experiment: a single teacher generates paired IFT and reasoning outputs for identical prompts by toggling only its reasoning mode, isolating supervision format as the sole variable. Training students at five scales (0.5B to 14B) and evaluating on 18 benchmarks, we find that at matched FLOPs, IFT lies on or near the Pareto frontier across the majority of configurations. Reasoning reaches the Pareto frontier only on open-ended tasks at 7B and above. Even there, a sequential curriculum mixing just 25-50% reasoning data with IFT captures most of the accuracy benefit at far lower compute cost.

Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Kevin El Haddad +2

1Diabolocom · 5MICS, CentraleSupélec, Université Paris-Saclay · 2Artefact Research Center +2

CoT Distillation LLM Fine-Tuning Efficient Language Model Reasoning Efficient Language Model Training Language Model Distillation Teacher-Student Learning

Sep 17, 2025cs.SE

SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models

Chain-of-Thought (CoT) prompting can substantially improve the reasoning ability of large language models (LLMs), but it often comes with high inference cost due to long and poorly controlled reasoning traces. This overhead is particularly problematic in software engineering tasks (e.g., code generation), where both latency and output reliability matter. To better understand this trade-off, we conduct an empirical study on widely used code generation benchmarks and observe that many modern reasoning models produce excessively verbose CoTs (often thousands of tokens), which frequently leads to truncation and unstable generation. Using a strict n-gram repetition detector, we find that most observed truncations are associated with degenerate looping behaviors. In addition, a HumanEval/129 case study shows that failed generations can be longer than successful ones, suggesting limited returns from overlong reasoning. Motivated by these findings, we propose SEER (Self-Enhancing Efficient Reasoning), a self-enhancing framework for adaptive CoT compression. SEER improves the conciseness of reasoning while preserving output quality, without relying on external compression tools. SEER refines self-generated CoT data via Best-of-N sampling to suppress looping and redundant traces, then applies a lightweight, data-driven filter to encourage concise yet correct reasoning. It then fine-tunes the model on the filtered data to internalize concise reasoning behaviors. Across four software engineering benchmarks on the evaluated DeepSeek-R1-Distill-Qwen-7B backbone, SEER reduces CoT length by 34.6% on average while improving task performance, with reduced truncation and fewer reasoning loops.

Kerui Huang, Shuhan Liu, Xing Hu +3

The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China · State Key Laboratory for Novel Software Technology, Nanjing University, China

Efficient Language Model Reasoning CoT Compression Large Reasoning Models Overthinking in Language Models LLM Code Generation

May 21, 2025cs.AI

When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

Large reasoning models (LRMs) have achieved remarkable success on complex tasks, yet their tendency to "overthink" leads to inefficiencies. Although "save-thinking" prompts are intended to mitigate this issue, we find that LRMs still frequently enter the "Still-thinking" mode instead of the expected "No-thinking" mode, especially on difficult queries. To analyze this behavioral divergence, we examine LRMs from three perspectives: confidence at the thinking-termination boundary, divergence in internal attention distributions, and attention allocation across prompt segments. We find that high perplexity is associated with later Still-thinking behavior, and that Still-thinking cases allocate more attention to the original question. Based on these observations, we propose an attention intervention method to regulate this behavior. While this intervention suppresses explicit thinking, it also causes a drop in accuracy, suggesting that the suppressed reasoning behavior is often useful for correctness. Our work provides confidence- and attention-level evidence for this behavior, highlighting the trade-off between instruction following, inference efficiency, and reasoning correctness.

Rongzhi Zhu, Yi Liu, Jiancheng Wang +6

State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1

Overthinking in Language Models LLM Interpretability Efficient Language Model Reasoning Causal Interventions in Language Models LLM Reasoning Large Reasoning Models

Feb 21, 2025cs.LG

The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer

Large language models have demonstrated remarkable progress in mathematical reasoning, leveraging chain-of-thought and reinforcement learning. However, many open questions remain regarding the interplay between reasoning token usage and accuracy gains. In particular, when comparing models across generations, it is unclear whether improved performance results from longer reasoning chains or more efficient reasoning. We systematically analyze reasoning chain length across o1-mini and o3-mini variants on the Omni-MATH benchmark, finding that o3-mini (m) achieves superior accuracy without requiring longer reasoning chains than o1-mini. Moreover, we show that accuracy generally declines as reasoning chains grow across all models and compute settings, even when controlling for difficulty of the questions. This accuracy drop is significantly smaller in more proficient models, suggesting that new generations of reasoning models use test-time compute more effectively. Finally, we highlight that while o3-mini (h) achieves a marginal accuracy gain over o3-mini (m), it does so by allocating substantially more reasoning tokens across all problems, even the ones that o3-mini (m) can already solve. These findings provide new insights into the relationship between model capability and reasoning length, with implications for efficiency, scaling, and evaluation methodologies.

Marthe Ballon, Andres Algaba, Vincent Ginis

Data Analytics Lab, Vrije Universiteit Brussel, Brussel 1050, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA

CoT Reasoning Efficient Language Model Reasoning LLM Mathematical Reasoning Large Reasoning Models