Efficient Language Model Reasoning

Overview

This field asks how language models can solve tasks such as math, coding, science questions, and multi-hop retrieval without spending unnecessary computation. Common approaches compress or prune chain-of-thought, stop generation when answers stabilize, and adapt reasoning budgets to task difficulty; others use reinforcement learning, distillation, search, parallel sampling, or smaller-model routing. Newer work also explores latent reasoning and memory or KV-cache management to reduce the cost of generating and retaining explicit thoughts. The central tension is that extra reasoning can help on difficult problems but may waste resources, introduce errors, or even hurt accuracy, so methods are judged by both quality and cost on benchmarks such as GSM8K, MATH, AIME, and GPQA.

Latest papers

CardsList

397 papers39 papers in the last four weeks, down 52% on the four weeks before. 0.3% of all new papers.

  1. Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

    Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM ReasoningStructured Reasoning

  2. Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation

    Sep 26, 2025Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Kevin El Haddad +2CoT DistillationLLM Fine-Tuning

  3. SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models

    Sep 17, 2025Kerui Huang, Shuhan Liu, Xing Hu +3Efficient Language Model ReasoningCoT Compression

  4. SmartThinker: Learning High-Density Reasoning via Step-Level Token Allocation

    Jul 6, 2025Xingyang He, Xiao Ling, Jie LiuEfficient Language Model ReasoningLarge Reasoning Models

  5. When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

    May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language ModelsLLM Interpretability

  6. Optimizing Anytime Reasoning via Budget Relative Policy Optimization

    May 19, 2025Penghui Qi, Zichen Liu, Tianyu Pang +3Efficient Language Model ReasoningLLM Reasoning

  7. The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer

    Feb 21, 2025Marthe Ballon, Andres Algaba, Vincent GinisCoT ReasoningEfficient Language Model Reasoning