LLM Reasoning

LLM: Large Language Model

This field asks how language models can carry out multi-step inference across mathematics, coding, logic, science, and real-world tasks, and how to tell robust reasoning from pattern matching. Common approaches include chain-of-thought prompting, self-consistency and tree search at inference time, tool use and symbolic verification, and post-training with supervised traces, reinforcement learning from verifiable rewards, or process feedback. Researchers are also exploring latent and recurrent computation to reduce reliance on long written traces, alongside benchmarks that test compositionality, robustness, and long-horizon execution. A central tension is that more computation can improve accuracy but raise cost, while plausible explanations and correct final answers do not by themselves establish faithful, reliable reasoning.

Latest papers

931 papers105 papers in the last four weeks, up 27% on the four weeks before. 0.6% of all new papers.

Deductive Logic in Language Models: Horizontal vs Vertical Reasoning
Oct 10, 2025Davide Maltoni, Matteo FerraraTransformer Interpretability CoT Reasoning
Department of Computer Science and Engineering, University of Bologna, Italy
Group Contrastive Policy Optimization: Learning Beyond the Self-Exploration Limit with Golden Answers
Oct 9, 2025Hao Wu, Wei Liu, Shizhe Xiang +3RL for Language Model Reasoning LLM Reasoning
oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning
Oct 9, 2025Ruiling Xu, Yifan ZhangLLM Evaluation Benchmark Design
University of Illinois Urbana-Champaign
Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrs
Oct 3, 2025Sayan Ghosh, Shahzaib Saqib Warraich, Dhruv Tarsadiya +2Text Generation Self-Consistency in Language Models
Thomas Lord Department of Computer Science, University of Southern California
Attention-Aligned Reasoning for Large Language Models
Oct 3, 2025Hongxiang Zhang, Yuan Tian, Tianyi ZhangLLM Reasoning Structured Reasoning in Language Models
Purdue University
How Well Does Mathematical Reasoning Transfer Across Languages in English-Centric LLMs?
Oct 2, 2025Wen Yang, Junhong Wu, Chong Li +2Cross-Lingual Reasoning in Language Models Cross-Lingual Transfer
School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Wuhan AI Research
Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm
Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM Reasoning Structured Reasoning
1Qizhi Institute · 2Dept. of Computer Science, Tsinghua University · College of AI, Tsinghua University +4
Predicting LLM Reasoning Performance with Small Proxy Model
Sep 25, 2025Woosung Koh, Juyoung Suk, Sungjun Han +2Language Model Pretraining LLM Evaluation
Trillion Labs · KAIST AI · ▽Trillion Labs, ♠KAIST AI
Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
Sep 1, 2025Kaviraj Pather, Elena Hadjigeorgiou, Arben Krasniqi +4LLM Interpretability Human-in-the-Loop AI
Department of Computer Science & Engineering University of Mauritius R´eduit 80837, Mauritius · Department of Electrical & Computer Engineering Cyprus University of Technology 3036 Limassol, Cyprus · Faculty of Electrical & Computer Engineering University of Prishtina ”Hasan Prishtina” 10000 Prishtina, Kosovo +4
PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
Aug 29, 2025Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema +7Confidence Estimation in Language Models Best-of-N Sampling
Imperial College London · University of Edinburgh · UCL
Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
Aug 17, 2025Xinda Jia, Jinpeng Li, Zezhong Wang +6Adaptive Reasoning in Language Models LLM Tool Use
School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · Huawei Technologies Co., Ltd., Beijing 100084, China · Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong 999077, China +3
Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
Aug 13, 2025Xiaojun Wu, Xiaoguang Jiang, Huiyang Li +11CoT Distillation Language Model Distillation
1Zhongxing Telecom Equipment(ZTE), China · 2China Mobile Communications Group Co Ltd
FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models
Aug 7, 2025Chuan Li, Qianyi Zhao, Fengran Mo +1Communication-Efficient Distributed Training Medical LLMs
East China Normal University, China · University of Montreal, Canada
From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control
Aug 6, 2025Rui Ha, Rui Pu, Chaozhuo Li +2RL for Language Model Reasoning LLM Inference Acceleration
Beijing University of Posts and Telecommunications, China · 2Chongqing University of Posts and Telecommunications, China
Knowing When to Critique: Task-Adaptive Metacognitive Regulation for Reliable LLM Reasoning
Jul 20, 2025Xinmeng Hou, Ziting Chang, Zhouquan Lu +5LLM Self-Correction Multi-Agent LLMs
Nanyang Technological University · Unicorn Verse · Shanghai Jiao Tong University +3
$π$ -CoT: Prolog-Initialized Chain-of-Thought Prompting for Multi-Hop Question-Answering
Jun 25, 2025Chao Wan, Albert Gong, Mihir Mishra +3Multi-Hop QA LLM Prompting
Cornell University.
Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control
Jun 23, 2025Aryasomayajula Ram BharadwajLanguage Model Steering Efficient Language Model Inference
Independent Researcher
Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
Jun 16, 2025Saurabh Pujar, Ira Ceka, Irene Manotas +3Software Engineering Agents Execution-Guided Code Generation
IBM · Columbia University
BOW: Training Language Models to Reason Over Plausible Next Words
Jun 16, 2025Ming Shen, Zhikun Xu, Jacob Dineen +2RL for Language Model Reasoning Next-Token Prediction
Arizona State University
SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments
May 29, 2025Zixiang Xu, Yanbo Wang, Yue Huang +13LLM Evaluation AI Agent Benchmarks
Mohamed bin Zayed University of Artificial Intelligence · University of Southern California · University of Notre Dame +3
PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics
May 29, 2025Atharva Naik, Prakam, Yash Mathur +6Program Synthesis Inductive Inference
Carnegie Mellon University · Independent Researcher · Ohio State University
Learning Composable Chains-of-Thought
May 28, 2025Fangcong Yin, Zeyu Leo Liu, Liu Leqi +2LLM Fine-Tuning CoT Reasoning
The University of Texas at Austin · Princeton University
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
May 22, 2025Xinghao Chen, Anhao Zhao, Heming Xia +7CoT Reasoning LLM Reasoning
Department of Computing, The Hong Kong Polytechnic University · Ningbo Digital Twin Institute, Eastern Institute of Technology, Ningbo, China
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language Models LLM Interpretability
State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1
Synergizing Large Language Models and Theory for Human-like Causal Reasoning
May 13, 2025Yanxi Zhang, Xin Cong, Zhong Zhang +3Causal Reasoning in Language Models Causal Reasoning
Center for Data Science, AAIS, Peking University · Wangxuan Institute of Computer Technology, Peking University · Tsinghua University +1
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
Mar 31, 2025Rui Wang, Hongru Wang, Boyang Xue +9LLM Inference Efficiency Cost-Aware Inference
University of Edinburgh, Edinburgh, UK · University of Macau, Macau SAR, China · University of Illinois Urbana-Champaign, USA
Thinking beyond the anthropomorphic paradigm benefits LLM research
Feb 13, 2025Lujain Ibrahim, Myra ChengAnthropomorphism in AI LLM Reasoning
University of Oxford · Stanford University
Foundations of Large Language Models
Jan 16, 2025Tong Xiao, Jingbo ZhuLanguage Model Pretraining LLM Alignment
NLP Lab, Northeastern University & NiuTrans Research
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMs
Sep 9, 2024Jinwei He, Feng LuAbductive Reasoning Causal Structure Learning
Beihang University, Beijing, China

Oct 10, 2025J/K move · Enter open · S save

Deductive Logic in Language Models: Horizontal vs Vertical Reasoning

Davide Maltoni, Matteo Ferrara

Department of Computer Science and Engineering, University of Bologna, Italy

Transformer Interpretability CoT Reasoning Implicit Reasoning in Language Models LLM Reasoning

Open paper AskPDF arXiv

Recent language models exhibit significant logical reasoning abilities, yet the mechanisms supporting deductive inference remain poorly understood. This paper studies small transformer-based language models trained from scratch on multi-step deductive tasks, focusing on the distinction between horizontal reasoning, where intermediate steps are generated autoregressively, and vertical reasoning, where inference unfolds implicitly across layers before the first output token is produced. We analyze two synthetic tasks: logical consequence over chains of symbolic implications and root-to-leaf navigation in binary trees. Mechanistic interpretability reveals that Chain-of-Thought supervision enables models to learn rule-based inference rather than statistical shortcuts. In the horizontal setting, a shallow attention-only model develops interpretable circuits for rule completion, rule chaining, and final decision making, largely implemented through induction-head-like mechanisms. We further introduce a truncated pseudoinverse method to decode the information carried by queries, keys, and values. For vertical reasoning, Chain-of-Thought appears to act less as explicit step-by-step guidance and more as a form of curriculum learning, helping the model acquire increasingly complex reasoning patterns. Without Chain-of-Thought, models tend to memorize or exploit dataset biases. These results provide a low-level account of how transformers can implement deductive reasoning and suggest how Chain-of-Thought may serve different functions in horizontal and vertical reasoning.