RL for Language Model Reasoning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

91 papers in the last four weeks, up 176% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 707

All topics

GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Oct 8, 2026Hexuan Deng, Zihao Yan, Xuebo Liu +8Reinforcement Learning RL for Language Model Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen) · Beijing Zhongguancun Academy · XinzhuAI
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards
Oct 8, 2026Xiaobing Chen, Zhiqi PangRL for Language Model Reasoning RL Post-Training
Harbin Engineering University · Tencent · Harbin Institute of Technology
Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimization
Oct 8, 2026Pengxin Guo, Shuang Zeng, Zonggen Li +3Federated RL RL for Language Model Reasoning
The University of Hong Kong · Sun Yat-sen University
RL-ARC: Calibrating Large Reasoning Models via Reasoning-guided Uncertainty
Oct 8, 2026Gukhyeon Lee, SangKeun LeeLanguage Model Calibration LLM Uncertainty Estimation
Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea
SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning
Oct 8, 2026Wei Yang, Shawn Li, Yuehan Qin +8Self-Evolving Agents Synthetic Data Generation for Language Models
University of Southern California · The George Washington University · Georgia Institute of Technology
Balancing Reference Guidance and Free Generation in Trajectory Rollouts for Reasoning RL
Oct 8, 2026Hanyu Wang, Nakul Agarwal, Hossein Nourkhiz Mahjoub +4LLM Reasoning RL for Language Model Reasoning
The Pennsylvania State University · Honda Research Institute USA · Honda Research Institute Japan
Less from More: Reinforcing Sparse Video Reasoning from Dense References
Oct 7, 2026Wenfang Sun, Yingjun Du, Cees G. M. SnoekVideo Reasoning VLM Reasoning
University of Amsterdam
Decoupling Exploration from Optimization in RLVR
Oct 7, 2026Saif Punjwani, Micah GoldblumRL for Language Model Reasoning RL Exploration
Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University
Coverage-Aware Reasoning with Medical Tokens for Diagnosis Prediction
Oct 7, 2026Kaisong Zhang, Haotian Fang, Junmeng Zhou +3Medical LLMs Clinical Prediction
Fuzhou University · Shanghai Jiao Tong University · Beihang University
RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning
Oct 7, 2026Yongqiang Yao, Jinru Tan, Kaihuan Liang +5RL for Language Models Reinforcement Learning
Shanghai Jiao Tong University · Central South University · SenseTime Research +2
BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
Oct 7, 2026Yingxiang Yang, Weihang Xiao, Zhunxuan Wang +2RL for Language Model Reasoning Group Relative Policy Optimization
Amazon AGI · Virginia Tech
CERO: Where and When to Allocate Rollouts for RL Post-Training
Oct 7, 2026Yiming Zong, Yige Wang, Xing Hu +2Reinforcement Learning RL Post-Training
Department of Industrial Engineering & Decision Analytics, Hong Kong University of Science and Technology · Innovation and Information Management, The University of Hong Kong · Faculty of Engineering & Faculty of Business and Economics, The University of Hong Kong
COPC: Coupled Off-Policy Correction for Asynchronous LLM Reinforcement Learning
Oct 7, 2026Zicheng Hu, Zhijian Zhou, Xuan Zhang +7Asynchronous RL RL Post-Training
Fudan University · Meituan · East China Normal University
Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
Oct 6, 2026Naoki Nishikawa, Taiji SuzukiReward Modeling Transformer-Based RL
The University of Tokyo, RIKEN AIP
Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
Oct 6, 2026Haoxiang Zhang, Qinglin Chen, Hiroaki Hayashi +9Agentic RL RL for Language Model Reasoning
Salesforce AI Research · UC San Diego · Texas A&M University +1
Minimal Witness Reinforcement Learning
Oct 5, 2026T. Y. Tsui, Zihao Ye, Pengxiang Cai +3RL for Language Model Reasoning Credit Assignment in RL
University of Pennsylvania · Shanghai AI Laboratory
Base Models Can Reason By Taking a Cue From Training Data
Oct 5, 2026Sophie L. Wang, Amil Dravid, Rulin Shao +3LLM Prompting RL for Language Model Reasoning
MIT · UC Berkeley · University of Washington +1
LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
Oct 5, 2026Shaokun Zhang, Yifan Zhang, Jian Hu +5Memory-Efficient Fine-Tuning RL for Language Model Reasoning
NVIDIA
Better Call Reward: Reward Hacking as Strategic Abstention in Legal Reasoning Models
Oct 5, 2026Subramanyam Sahoo, Justin ShenkReward Hacking LLM Auditing
Horizon Research.
Ontology Concept Overlap as a Training Signal: Knowledge-Grounded Reinforcement Learning for Clinical Question Answering
Oct 5, 2026Aditya Tanna, Abhishek JindalReward Modeling RL for Language Model Reasoning
Dhirubhai Ambani University Gandhinagar, Gujarat, India
Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers
Oct 5, 2026Xiaoyu Chen, Bo Shao, Tiangang Zhu +5RL for Language Model Reasoning Selective Knowledge Distillation
Institute of Software, Chinese Academy of Sciences · Microsoft · Work done during an internship at Microsoft +1
Learning to Simulate Individuals from Macro Social Signals
Oct 5, 2026Yining Zhao, Bushi Liu, Haofei Yu +5Human Behavior Simulation RL for Language Model Reasoning
University of Illinois Urbana-Champaign · Independent Researcher
OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
Oct 1, 2026Haibo Wang, Jiteng Mu, Jialu Li +5Audio-Visual Understanding RL for Language Model Reasoning
University of California, Davis · Adobe Research
On Language Drift during RLVR Post-Training
Oct 1, 2026Michael Sullivan, Alexander KollerRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Language Science and Technology, Saarland University, Saarbrücken, Germany.
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Oct 1, 2026Qijia He, Ruinan Jin, Jun Luo +2Asynchronous RL RL for Language Model Reasoning
The Ohio State University · Arizona State University
Function-Structured Reinforcement Learning with Executable Verifiers for Mathematical Reasoning
Oct 1, 2026Zihan Liu, Xurong XieLLM Reasoning with Graphs RL for Language Model Reasoning
Department of Computer Science, University College London, UK · Institute of Software, Chinese Academy of Sciences, China
AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models
Oct 1, 2026Yuxiang Wang, Kunyu Feng, Yuancheng Wang +12RL for Language Model Reasoning Audio Reasoning
The Chinese University of Hong Kong, Shenzhen · Tencent Hunyuan · Tsinghua University +2
Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
Oct 1, 2026Shiu-Hong Kao, Yubo Zhao, Zhenyu Tian +3Reinforcement Learning RL for Language Model Reasoning
National University of Singapore · The Hong Kong University of Science and Technology · University of Science and Technology of China
Learning to Ask: Information Acquisition for SLM-LLM Collaboration, under a budget
Oct 1, 2026Yongjun Kim, Xiaoxiao Li, Jaeho LeeRL for Language Model Reasoning Efficient Language Model Reasoning
Pohang University of Science and Technology (POSTECH) · Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute
Does Scaling Reinforcement Learning Really Require More Training?
Oct 1, 2026Bangji Yang, Jiajun Fan, Hongbo Ma +2RL for Language Model Reasoning RL Post-Training
University of Illinois at Urbana-Champaign · Tsinghua University

Oct 8, 2026, cs.LGJ/K move · Enter open · S save

GRPODropout: Less is More for Online Reinforcement Learning Rollouts

Hexuan Deng, Zihao Yan, Xuebo Liu, Shuo Nie+7

Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen) · Beijing Zhongguancun Academy · XinzhuAI

Reinforcement Learning RL for Language Model Reasoning RL Post-Training Group Relative Policy Optimization

Open paper AskPDF arXiv

Reinforcement learning (RL) methods such as GRPO substantially improve large language model reasoning but often suffer from policy entropy collapse: the loss of sampling diversity weakens exploration and limits further improvement. Existing methods address this issue either through algorithm-level interventions, such as reward modification and entropy/KL regularization, or through token-level reweighting. We investigate a complementary perspective: entropy collapse can also be mitigated by changing which generated rollouts contribute to policy updates. Under the same sampling budget, not all rollouts contribute positively to an update, and selectively excluding some can improve learning. To address this, we propose GRPODropout: before the standard update, we use a simple strategy that selectively removes a small number of high-probability positive-advantage rollouts and recenters the retained advantages. To motivate this design, we develop a rollout-level theoretical analysis that guides method design and threshold selection. The method changes only rollout usage, and adds negligible computational overhead. Experiments show higher accuracy than original GRPO and higher actor entropy while using fewer rollout samples for updates, illustrating "less is more." This work provides insight into RL rollout usage: removing some rollouts can improve performance. Code is available at https://github.com/hexuandeng/GRPODropout/.

Momentum

Related topics

Latest papers 707