RL Post-Training

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 206% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents
Jun 24, 2026Changdae Oh, Wendi Li, Seongheon Park +3Process Reward Models RL Post-Training
University of Wisconsin–Madison · Argonne National Laboratory
Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors
Jun 24, 2026Guozheng Ma, Lu Li, Zilin Wang +2Reinforcement Learning RL Post-Training
Nanyang Technical University · Mila - Quebec AI Institute · Université de Montréal +1
Reinforcement Learning Towards Broadly and Persistently Beneficial Models
Jun 22, 2026Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab +5LLM Alignment LLM Safety Alignment
OpenAI
A Formula-Driven Survey and Research Agenda for On-Policy Distillation
Jun 22, 2026Bowen ZhangLanguage Model Distillation On-Policy Distillation
Tsinghua University
On the Position Bias of On-Policy Distillation
Jun 21, 2026Yan Xie, Sijie Zhu, Tiansheng Wen +2Importance Weighting Language Model Distillation
Xidian University · Georgia Institute of Technology · Amazon AGI SF Lab
Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
Jun 21, 2026Zicheng Xu, Ruixuan Zhang, Yu-Neng Chuang +7Reinforcement Learning RL for Language Model Reasoning
Johns Hopkins University · Rice University · University of Haifa +2
Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
Jun 20, 2026Zhao Yang, Yuxuan Jiang, Ting-Chih Chen +18RL for Language Models Credit Assignment in RL
VU Amsterdam, The Netherlands · University of Maryland, Baltimore County, USA · Leiden University, The Netherlands +4
World Engine: Towards the Era of Post-Training for Autonomous Driving
Jun 18, 2026Tianyu Li, Li Chen, Caojun Wang +16Autonomous Driving RL Post-Training
Archon Robotics · Shanghai Innovation Institute · NTU +6
Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training
Jun 17, 2026Jianzhe LinRL for Code Generation Reinforcement Learning
MetaAI · *Work done at Meta
Spotlight: Synergizing Seed Exploration and Spot GPUs for DiT RL Post-Training
Jun 17, 2026Ruiqi Lai, Dakai An, Wei Gao +6Diffusion Transformer RL for Image Generation
NTU Singapore · Hong Kong University of Science and Technology · Alibaba Group
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
Jun 17, 2026Yingyu Shan, Yuhang Guo, Zihao Cheng +7Token-Level Credit Assignment RL for Language Model Reasoning
Beijing Institute of Technology · Beihang University · Independent Researcher
SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR
Jun 16, 2026Siddharth Aphale, Kelly LiuSupervised Fine-Tuning Group Relative Policy Optimization
Stanford University, Stanford, CA, USA.
LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents
Jun 16, 2026Haoyang Fang, Wei Zhu, Boran Han +11Reinforcement Learning Large Language Model-Guided Optimization
†LLMZero Project Core Team. · ∗Work done at Amazon.
Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients
Jun 16, 2026Byung-Kwan Lee, Ximing Lu, Shizhe Diao +6Curriculum RL LLM-Guided RL
NVIDIA
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Jun 16, 2026Zihao Wei, Wenjie Shi, Liang Pang +8Overthinking in Language Models RL for Language Model Reasoning
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
Jun 16, 2026Chao Chen, Chengzu Li, Zhiwei Li +2Reinforcement Learning LLM-Guided RL
1LARK, HKUST (GZ) · University of Cambridge · 3HKUST
Greed Is Learned: Visible Incentives as Reward-Hacking Triggers
Jun 15, 2026Tong Che, Rui WuReward Hacking LLM Safety
Rutgers University
GD $^2$ PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Jun 15, 2026Haotian Liu, Yihao Liu, Jingwei Ni +11Multi-Objective Reinforcement Learning RL for Language Models
Alibaba · Renmin University of China · Peking University +3
How Post-Training Shapes Biological Reasoning Models
Jun 15, 2026Lukas Fesser, Hanlin Zhang, Michelle M. Li +5Neural Network Generalization Supervised Fine-Tuning
Harvard University · Google DeepMind · Google Research
daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization
Jun 15, 2026Dayuan Fu, Mohan Jiang, Tongyu Wang +5RL for Code Generation GPU Kernel Optimization
1SII · 2SJTU · 3GAIR
PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents
Jun 15, 2026Zhenbang Du, Jun Luo, Zhiwei Zheng +8Learning from Demonstration LLM Agent Training
Georgia Institute of Technology · Ohio State University · University of Pennsylvania +1
Select and Improve: Understanding the Mechanics of Post-Training for Reasoning
Jun 11, 2026Akshay Krishnamurthy, Audrey Huang, Nived RajaramanRL for Language Model Reasoning RL Post-Training
Microsoft Research NYC · UIUC
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
Jun 11, 2026Jiayu Yang, Chao Chen, Shengen Wu +6RL for Language Model Reasoning Mechanistic Interpretability
HKUST(GZ) · University of Cambridge · NTU +2
Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
Jun 10, 2026Yucheng Li, Huiqiang Jiang, Yang Xu +14Multi-Token Prediction Speculative Decoding
Qwen Team, Alibaba Inc.
Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization
Jun 10, 2026Frank Xiao, Mary PhuongReward Hacking RL for Language Models
California Institute of Technology
Beyond Uniform Token-Level Trust Region in LLM Reinforcement Learning
Jun 9, 2026Renjie Mao, Xiangxin Zhou, Lvfang Tao +7RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Tencent Hunyuan
Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoning
Jun 8, 2026Wooil JungRL for Language Model Reasoning Group Relative Policy Optimization
Bittensor Agent Arenas as a Trajectory Primitive: Distilling a Shopping Agent from ShoppingBench Subnet Traces
Jun 8, 2026Shardul Bansal, Seth Schilbe, Jarrod BarnesAI Agent Benchmarks LLM Agent Training
1ORO AI · 2Dynamical Systems
Rethinking the Divergence Regularization in LLM RL
Jun 8, 2026Jiarui Yao, Xiangxin Zhou, Penghui Qi +3RL for Language Models Policy Optimization
1Tencent Hunyuan · 2UIUC · 3NUS
3SPO: State-Score-Supervised Policy Optimization for LLM Agents
Jun 8, 2026Yu Han, Kailing Li, Yang Jiao +4Credit Assignment in RL Step-Level Credit Assignment in RL
School of Computer Science and Technology, East China Normal University · Fudan University · KAUST +1

Jun 24, 2026, cs.LGJ/K move · Enter open · S save

Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents

Changdae Oh, Wendi Li, Seongheon Park, Samuel Yeh+2

University of Wisconsin–Madison · Argonne National Laboratory

Process Reward Models RL Post-Training RL for LLM Agents

Open paper AskPDF arXiv

Process reward models enable fine-grained, step-level evaluation of LLMs, yet building them for agentic settings remains prohibitively difficult: long-horizon interactions, irreversible actions, and stochastic environment feedback make both human annotation and Monte Carlo estimation infeasible at scale. In this work, we show that reinforcement learning (RL) post-training already provides the ingredients for effective step-level scoring, eliminating the need for dedicated reward model training altogether. Concretely, we derive an implicit advantage under a general stochastic Markov decision process, which we term progress advantage -- log-probability ratio between the RL-trained policy and its reference policy exactly recovers the optimal advantage function. This formulation makes the resulting signal annotation-free, domain-agnostic, and available as a byproduct of the standard RL post-training pipeline. We validate the effectiveness of the progress advantage across three different applications: test-time scaling, uncertainty quantification, and failure attribution on five benchmarks and four model families. Across all settings, it consistently outperforms confidence-based baselines and, despite requiring no task-specific training, surpasses dedicated trained reward models. We complement these results with deeper analyses on characteristics of progress advantage, offering practical guidance for adoption in real-world agentic systems.

Momentum

Related topics

Latest papers 309