RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

124 papers in the last four weeks, up 75% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 780

All topics

Reinforcement Learning Improves Traversal of Parametric Knowledge in LLMs
Nov 8, 2025Renfei Zhang, Manasa Kaniselvan, Rylan Schaeffer abd Niloofar MireshghallahRL for Language Model Reasoning Factual Knowledge in Language Models
Carnegie Mellon University · MIT · Stanford University
OpenSIR: Open-Ended Self-Improving Reasoner
Nov 1, 2025Wai-Chung Kwan, Joshua Ong Jun Leang, Pavlos Vougiouklis +3Self-Play RL RL for Language Model Reasoning
University of Edinburgh · Imperial College London · Huawei Technologies Research & Development (UK) Limited +1
Learning to Reason Efficiently with Discounted Reinforcement Learning
Oct 27, 2025Alex Ayoub, Kavosh Asadi, Dale Schuurmans +2Reinforcement Learning RL for Language Model Reasoning
Amazon, University of Alberta · Amazon · University of Alberta
Representation-Based Exploration for Language Models: From Test-Time to Post-Training
Oct 13, 2025Jens Tuyls, Dylan J. Foster, Akshay Krishnamurthy +1RL for Language Model Reasoning Test-Time Scaling
Princeton University · Microsoft Research NYC
Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts
Oct 10, 2025Xinyi Wang, Jinyi Han, Zishang Jiang +7Reinforcement Learning RL for Language Model Reasoning
School of Data Science, Fudan University · 2Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1
CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts
Oct 10, 2025Jiuheng Lin, Cong Jiang, Zirui Wu +2RL for Language Models RL for Language Model Reasoning
Wangxuan Institute of Computer Technology, Peking University
Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
Oct 9, 2025Yuchen Zhu, Wei Guo, Jaemoo Choi +4RL for Language Model Reasoning Policy Optimization
Georgia Institute of Technology.
Base Models Know How to Reason, Thinking Models Learn When
Oct 8, 2025Constantin Venhoff, Iván Arcuschin, Philip Torr +2LLM Interpretability RL for Language Model Reasoning
University of Oxford, UK · MATS · Poseidon Research.
Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
Oct 5, 2025Zishang Jiang, Jinyi Han, Tingyun Li +7RL for Language Model Reasoning LLM-Guided RL
School of Data Science, Fudan University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1
Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression
Oct 2, 2025Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad +3RL for Language Model Reasoning LLM Inference Acceleration
UNC Chapel Hill · The University of Texas at Austin · Microsoft Research
Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
Oct 1, 2025Xin-Qiang Cai, Wei Wang, Feng Liu +3Reinforcement Learning RL for Language Model Reasoning
RIKEN AIP, Tokyo, Japan · The University of Tokyo, Tokyo, Japan · The University of Melbourne, Melbourne, Australia
Interactive Learning for LLM Reasoning
Sep 30, 2025Hehai Lin, Shilei Cao, Sudong Wang +5RL for Language Model Reasoning LLM Agent Training
1The Hong Kong University of Science and Technology (Guangzhou) · 2Sun Yat-sen University · 4National University of Singapore +2
TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models
Sep 27, 2025Song Jin, Juntian Zhang, Ruyu Lyu +7LLM Alignment LLM Personalization
Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University
ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search
Aug 27, 2025Sining Zhoubian, Dan Zhang, Jie TangRL for Code Generation RL for Language Model Reasoning
Department of Computer Science and Technology, Tsinghua University · Work done when interned at Z.ai. · School of Electronics and Computer Scinence, Univeristy of Southampton
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Aug 13, 2025Maxime Heuillet, Yufei Cui, Boxing Chen +2RL for Language Model Reasoning Efficient Language Model Training
Universit´e Laval (IID), Canada · Mila - Qu´ebec AI Institute, Canada · Huawei Noah’s Ark Lab (Montreal Research Center), Canada +1
From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control
Aug 6, 2025Rui Ha, Rui Pu, Chaozhuo Li +2RL for Language Model Reasoning LLM Inference Acceleration
Beijing University of Posts and Telecommunications, China · 2Chongqing University of Posts and Telecommunications, China
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
Aug 4, 2025Taihang Zhen, Jialiang Hong, Kai Chen +12RL for Language Model Reasoning CoT Compression
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China · China Mobile (Suzhou) Software Technology Co., Ltd., Suzhou 215000, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China +2
Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
Aug 1, 2025Zizhan Ma, Wenxuan Wang, Meidan Ding +7LLM Prompting RL for Language Model Reasoning
The Chinese University of Hong Kong · Renmin University of China · Shenzhen University +3
Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language Models RL for Language Model Reasoning
Heinrich Heine University Düsseldorf, Germany
Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
Jul 22, 2025Chuanhao Yan, Fengdi Che, Xuhan Huang +12RL for Language Models RL for Language Model Reasoning
Shanghai AI Lab · Tsinghua University · University of Alberta +5
BOW: Training Language Models to Reason Over Plausible Next Words
Jun 16, 2025Ming Shen, Zhikun Xu, Jacob Dineen +2RL for Language Model Reasoning Next-Token Prediction
Arizona State University
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
Apr 25, 2025Pablo Miralles-González, Javier Huertas-Tato, Alejandro Martín +1LLM Fine-Tuning RL for Language Model Reasoning
Department of Computer Systems, Technical University of Madrid, Madrid, Spain · United Arab Emirates University, Al Ain, United Arab Emirates
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model Reasoning Mathematical Reasoning
University of Southern California · Carnegie Mellon University · Mohamed Bin Zayed University of Artificial Intelligence
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
Apr 3, 2025Weibin Liao, Xin Gao, Tianyu Jia +6LLM Planning RL for Language Model Reasoning
School of Computer Science, Peking University · 2Key Laboratory of High Confidence Software Technologies, Ministry of Education · School of Computer Sciences, Beijing University of Posts and Telecommunications +5
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
Amortizing intractable inference in large language models
Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized Inference LLM Fine-Tuning
Mila – Quebec AI Institute, Universit´e de Montr´eal · University of Oxford
Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
Date pendingMuhammad Khalifa, Zohaib Khan, Omer Tafveez +2RL Benchmarks Reward Hacking
NVIDIA · University of Michigan · University of Illinois Urbana-Champaign
A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation
Date pendingMiguel Moura Ramos, Duarte M. Alves, André F. T. MartinsLLM Alignment RL for Language Model Reasoning
Instituto Superior Técnico, Universidade de Lisboa · Instituto de Telecomunicações · TransPerfect +1
Complementing reinforcement learning with SFT through logit averaging in the post training of LLMs
Date pendingXingwei Gan, Ying ZhuRL for Language Model Reasoning Group Relative Policy Optimization
UC San Diego
Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning
Date pendingAnthony GX-Chen, Ankit Anand, Gheorghe Comanici +7Reinforcement Learning RL for Language Model Reasoning
New York University · Google DeepMind

Nov 8, 2025, cs.CLJ/K move · Enter open · S save

Reinforcement Learning Improves Traversal of Parametric Knowledge in LLMs

Renfei Zhang, Manasa Kaniselvan, Rylan Schaeffer abd Niloofar Mireshghallah

Carnegie Mellon University · MIT · Stanford University

RL for Language Model Reasoning Factual Knowledge in Language Models Large Reasoning Models

Open paper AskPDF arXiv

Reinforcement learning (RL) is often credited with improving language model reasoning at the expense of knowledge. We challenge this narrative by showing that reasoning models consistently outperform their instruction-tuned versions on pure knowledge recall tasks. These gains do not reflect newly acquired information, but rather an improved procedural skill in navigating and searching existing knowledge hierarchies within the model parameters. Structured prompting, which explicitly guides models through hierarchical traversal -- recovers most of the instruct-reasoning gap across five model families. A controlled RL experiment on unseen, non-extractable facts improves recall of held-out frequent but previously inaccessible facts, ruling out simple data exposure. On depth-stratified retrieval tasks, reasoning models exhibit superior traversal as retrieval depth grows. Layerwise activation analysis further shows that while factual representations maintain high cosine similarity between instruct and reasoning models, query representations diverge noticeably, indicating that reasoning primarily reshapes how models traverse knowledge rather than the knowledge representation itself. Finally, we find that distilled models often fail to match reasoning models on knowledge recall because they imitate self-correction without acquiring the exploratory behavior needed for hierarchical navigation. Together, these findings suggest that improving factual recall in LLMs depends not only on expanding what models know but also on teaching them to navigate it -- motivating future post-training methods that optimize traversal.

Momentum

Related topics

Latest papers 780