RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,103 papers170 papers in the last four weeks, up 124% on the four weeks before. 1.0% of all new papers.

What MLLMs Learn about When they Learn about Multimodal Reasoning
Oct 2, 2025Jiwan Chung, Neel Joshi, Pratyusha Sharma +2Multimodal Reasoning Multimodal Model Evaluation
Microsoft Research AI Frontiers · Yonsei University · Seoul National University
Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression
Oct 2, 2025Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad +3RL for Language Model Reasoning LLM Inference Acceleration
UNC Chapel Hill · The University of Texas at Austin · Microsoft Research
Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
Oct 1, 2025Xin-Qiang Cai, Wei Wang, Feng Liu +3Reinforcement Learning RL for Language Model Reasoning
RIKEN AIP, Tokyo, Japan · The University of Tokyo, Tokyo, Japan · The University of Melbourne, Melbourne, Australia
Interactive Learning for LLM Reasoning
Sep 30, 2025Hehai Lin, Shilei Cao, Sudong Wang +5RL for Language Model Reasoning LLM Agent Training
1The Hong Kong University of Science and Technology (Guangzhou) · 2Sun Yat-sen University · 4National University of Singapore +2
Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
Sep 30, 2025Xin Xu, Clive Bai, Kai Yang +4Efficient Language Model Reasoning RL for Language Model Reasoning
LLM Department, Tencent · The Hong Kong University of Science and Technology · The University of Hong Kong
TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models
Sep 27, 2025Song Jin, Juntian Zhang, Ruyu Lyu +7LLM Alignment LLM Personalization
Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University
Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation
Sep 18, 2025Yujun Zhou, Zhenwen Liang, Haolin Liu +7Language Model Self-Improvement RL for Language Models
Tencent AI Lab · University of Notre Dame · University of Virginia
RPRO: Ranked Preference Reinforcement Optimization for Enhancing Medical QA and Diagnostic Reasoning
Aug 31, 2025Chia-Hsuan Hsu, Jun-En Ding, Hsin-Ling Hsu +5Medical LLMs Medical QA
Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei City, Taiwan · Department of Systems Engineering, Stevens Institute of Technology, Hoboken, NJ, USA · Far Eastern Memorial Hospital, New Taipei City, Taiwan +4
ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangRL for Code Generation RL for Language Model Reasoning
Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Aug 13, 2025Maxime Heuillet, Yufei Cui, Boxing Chen +2RL for Language Model Reasoning Efficient Language Model Training
Universit´e Laval (IID), Canada · Mila - Qu´ebec AI Institute, Canada · Huawei Noah’s Ark Lab (Montreal Research Center), Canada +1
From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control
Aug 6, 2025Rui Ha, Rui Pu, Chaozhuo Li +2RL for Language Model Reasoning LLM Inference Acceleration
Beijing University of Posts and Telecommunications, China · 2Chongqing University of Posts and Telecommunications, China
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
Aug 4, 2025Taihang Zhen, Jialiang Hong, Kai Chen +12RL for Language Model Reasoning CoT Compression
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China · China Mobile (Suzhou) Software Technology Co., Ltd., Suzhou 215000, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China +2
Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
Aug 1, 2025Zizhan Ma, Wenxuan Wang, Meidan Ding +7LLM Prompting RL for Language Model Reasoning
The Chinese University of Hong Kong · Renmin University of China · Shenzhen University +3
Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language Models RL for Language Model Reasoning
Heinrich Heine University Düsseldorf, Germany
Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
Jul 22, 2025Chuanhao Yan, Fengdi Che, Xuhan Huang +12RL for Language Models RL for Language Model Reasoning
Shanghai AI Lab · Tsinghua University · University of Alberta +5
The Invisible Leash: Why RLVR May or May Not Escape Its Origin
Jul 20, 2025Fang Wu, Weihao Xuan, Danny Xing +5RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Stanford University, CA, USA · The University of Tokyo, Japan · RIKEN AIP, Tokyo, Japan +2
BOW: Training Language Models to Reason Over Plausible Next Words
Jun 16, 2025Ming Shen, Zhikun Xu, Jacob Dineen +2RL for Language Model Reasoning Next-Token Prediction
Arizona State University
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
Apr 25, 2025Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín +1LLM Fine-Tuning RL for Language Model Reasoning
Department of Computer Systems, Technical University of Madrid, Madrid, Spain · United Arab Emirates University, Al Ain, United Arab Emirates
Kongzi: A Historical Large Language Model with Fact Enhancement
Apr 13, 2025Jiashu Yang, Ningning Wang, Yian Zhao +5Language Modeling RL for Language Model Reasoning
Dalian University of Technology · Peking University · Nanyang Technological University
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model Reasoning Mathematical Reasoning
University of Southern California · Carnegie Mellon University · Mohamed Bin Zayed University of Artificial Intelligence
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
Apr 3, 2025Weibin Liao, Xin Gao, Tianyu Jia +6LLM Planning RL for Language Model Reasoning
School of Computer Science, Peking University · 2Key Laboratory of High Confidence Software Technologies, Ministry of Education · School of Computer Sciences, Beijing University of Posts and Telecommunications +5
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
Amortizing intractable inference in large language models
Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized Inference LLM Fine-Tuning
Mila – Quebec AI Institute, Universit´e de Montr´eal · University of Oxford

Oct 2, 2025J/K move · Enter open · S save

What MLLMs Learn about When they Learn about Multimodal Reasoning

Jiwan Chung, Neel Joshi, Pratyusha Sharma, Youngjae Yu+1

Microsoft Research AI Frontiers · Yonsei University · Seoul National University

Multimodal Reasoning Multimodal Model Evaluation Multimodal Spatial Reasoning RL for Language Model Reasoning Visual Grounding

Open paper AskPDF arXiv

Evaluation of multimodal reasoning models is typically reduced to a single accuracy score, implicitly treating reasoning as a unitary capability. We introduce MathLens, a benchmark of textbook-style geometry problems that exposes this assumption by operationally decomposing performance into perception, reasoning, and multimodal-specific components. Each problem is derived from a symbolic specification and accompanied by visual diagrams, text-only variants, multimodal questions, and targeted perceptual probes, enabling controlled measurement of each component. Using this decomposition, we show that common training strategies are associated with systematically different capability profiles that are invisible under aggregate accuracy. Across the models we study, reinforcement learning primarily improves perceptual grounding and robustness to diagram variation, while our analysis suggests that textual SFT yields gains through reflective reasoning. In contrast, as perception and reasoning improve, a growing fraction of remaining errors fall outside these components and are categorized as multimodal-specific. These results suggest that apparent progress in multimodal reasoning reflects shifting balances among subskills rather than uniform advancement, motivating evaluation beyond scalar accuracy.