RL Post-Training

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 206% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Sep 8, 2026Youngrok Park, Sangmin Bae, Hojung Jung +6Policy Gradient Weak-to-Strong Generalization
KAIST AI · Microsoft · University of Toronto +3
Miles v0.1: Production-Level Post-Training
Sep 8, 2026RadixArk, :, Tom Chen +11Reinforcement Learning LLM Agent Training
RadixArk
TV-Regulated OPD: Direction Matters in On-Policy Distillation
Sep 8, 2026Han Xiao, Yifan Niu, Dongyi Liu +2Preference Optimization On-Policy Distillation
The Hong Kong University of Science and Technology (Guangzhou) · The University of Edinburgh · The Hong Kong University of Science and Technology
Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
Sep 3, 2026Boyan Li, Bingsen Chen, Chenghao Yang +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Alberta · New York University · NYU Shanghai +3
DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training
Sep 3, 2026Shubham Gandhi, Saurabh Goyal, Kiran Kate +1Agentic RL Step-Level Credit Assignment in RL
1Carnegie Mellon University · 2IBM Research
Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO
Sep 3, 2026Hyun Bin Park, Du-Seong ChangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Artificial Intelligence, Sogang University, Seoul, Republic of Korea
LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL
Sep 3, 2026Sijie Wang, Zhiqiang Tan, Xinrui Yang +1RL for Diffusion Models RL Post-Training
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen
Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation
Sep 2, 2026Zhiwei Zhang, Zechen Sun, Fei Zhao +6On-Policy Distillation RL Post-Training
AllSpark Team
Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents
Sep 1, 2026Liming Pu, Xiaoxia Li, Yifu Liu +2Agentic RL RL Post-Training
Alibaba Research
CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training
Sep 1, 2026Siyuan Li, Xinxin Song, Chen Ruinian +5Reward Hacking Rubric-Based RL
Department of Automation, Tsinghua University · Meituan
On-policy Distillation with Verifiable Reward
Aug 25, 2026Wenze Lin, Jiale Zhao, Xitai Jiang +5RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
LeapLab, Tsinghua University · Qiuzhen College, Tsinghua University · Beihang University +2
PertMind: Eliciting Emergent Biological Reasoning in LLM via Reinforcement Learning on Cellular Perturbation Data
Aug 17, 2026Zhenchao Tang, Xiaogang Xu, Jiafei Wu +16RL for Language Model Reasoning Single-Cell Perturbation Modeling
Zhejiang University
Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning
Aug 13, 2026Yao Zhou, Hang Gao, Fengge Wu +2Group Relative Policy Optimization Step-Level Credit Assignment in RL
Institute of Software Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Scaling Automatic Research Agents via World Models
Aug 12, 2026Xiyuan Yang, Sheikh Sarwar, Jingru Cheng +8World Model Learning RL Post-Training
University of Illinois Urbana-Champaign · Amazon
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Aug 12, 2026Kai Yang, Jingwei Xu, Wanyu Wang +4RL for Language Model Reasoning Constrained RL
School of Computer Science, Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · NovaCore +3
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Aug 11, 2026Zetao Hong, Song Yuan, Yuanhao Ding +4LLM Inference Scheduling KV-Cache Management
State Key Laboratory of Novel Software Technology, Nanjing University · StepFun
MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training
Aug 11, 2026Yikai Wang, Chuansai Zhou, Yuhang Zhou +10RL Post-Training Mixture-of-Experts Models
State Key Laboratory for Novel Software Technology, Nanjing University, China · Huawei, China
RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation
Aug 10, 2026Boxiong Wang, Hui Kang, Geng Sun +3UAV Navigation Vision-Language Navigation
1Jilin University · 4Zhongguancun Academy · 2Tsinghua University +1
Beyond Solvability: Task Learnability as a Static Prior for LLM RL Post-Training
Aug 10, 2026Ting Zhou, Zhenqing Ling, Daoyuan Chen +4Reinforcement Learning RL for Language Model Reasoning
Sun Yat-Sen University · Alibaba Group
Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training
Aug 8, 2026Binwen Tan, Jingchao Wang, Dengzhe Hou +6RL for Language Model Reasoning Multi-Task RL
1Graduate School of Information Sciences, Tohoku University · School of Computer Science, Peking University · 2Unprecedented-scale Data Analytics Center, Tohoku University +3
Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Models
Aug 8, 2026Jianqi Zhang, Xingyu Zhang, Zeen Song +3Time Series Forecasting RL Post-Training
Institute of Software, Chinese Academy of Sciences, Beijing, China
How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
Aug 7, 2026Lichao Ma, Yang Sun, Shuaitao Zhao +9Token-Level Credit Assignment Credit Assignment in RL
1Peking University · 2Meituan LongCat Interaction Team · 3Fudan University +4
Progressive Alignment of Recommender Foundation Model through Multi-Phase Post-Training
Aug 7, 2026Oseong Choi, Hoeinn Kim, Jihoon Lee +2Recommender Systems RL Post-Training
NAVER WEBTOON Seongnam-si, Republic of Korea
CrystalGRPO: Target-Aligned and Coverage-Preserving Reinforcement Learning for Flow-Based Crystal Structure Prediction
Aug 6, 2026Kaixiang Su, Hongfei Xue, Qiang ZhuCrystal Structure Prediction Group Relative Policy Optimization
Department of Computer Science, University of North Carolina at Charlotte · Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte · North Carolina Battery Complexity, Autonomous Vehicle and Electrification (BATT CAVE) Research Center
Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning
Aug 5, 2026Zheyuan Zhang, Manqing Mao, Hong Wang +8RL for Language Models RL Post-Training
University of Notre Dame · Amazon, Inc
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Aug 5, 2026Yi Yang, Cong Qin, Xiaodan Liu +8Agentic RL On-Policy Distillation
Meituan LongCat Interaction · Nanjing University · Peking University +3
ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
Aug 4, 2026Jinhe Bi, Chennan Zhou, Zengjie Jin +10RL for Language Model Reasoning RL Post-Training
National University of Singapore · Ludwig Maximilian University of Munich · Huawei Heisenberg Research Center +6
Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training
Aug 4, 2026Yibei Liu, Jiajun Chen, Qianle Zhang +4Multimodal Large Language Models Risk-Sensitive RL
University of Electronic Science and Technology of China · Zhejiang University · South China University of Technology +1
Instruction-Conditioned Exploration for Reinforcement Learning with Self-Distillation to an Unconditioned Policy
Aug 3, 2026Jim Dilkes, Vahid Yazdanpanah, Sebastian SteinRL for Language Model Reasoning RL Post-Training
University of Southampton Southampton United Kingdom
Toward Plasticity-Preserving KL Regularization for Capability Retention in LLM Reinforcement Learning
Aug 3, 2026Li Wang, Xiaodong Lu, Xiaohan Wang +4Continual Learning for LLMs KL-Regularized RL
1Meituan · 2Nanyang Technological University

Sep 8, 2026, cs.LGJ/K move · Enter open · S save

Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation

Youngrok Park, Sangmin Bae, Hojung Jung, Jongwoo Ko+5

KAIST AI · Microsoft · University of Toronto+3

Policy Gradient Weak-to-Strong Generalization Multi-Teacher Knowledge Distillation On-Policy Distillation RL Post-Training

Open paper AskPDF arXiv

Weak-to-strong generalization asks whether stronger models can learn from weaker supervisors and surpass them. This question is particularly important for successive model generations and multi-domain consolidation, where repeating frontier-scale post-training from scratch can be prohibitively expensive. Yet conventional distillation treats the weak teacher as an optimization target, potentially imposing its capacity ceiling on the student. We introduce On-Policy Reverse Distillation (OPRD), which evaluates the teacher's policy shift relative to its reference policy on student rollouts and amplifies the component of the student's verifier-driven policy gradient along that direction. By rescaling only verifier-supported updates, OPRD preserves the stationary points of policy optimization while accelerating learning beyond the teacher. In both successive model transfer and multi-teacher distillation, OPRD achieves higher performance with fewer student updates than existing RL and distillation approaches. Response-style analysis shows that OPRD students remain closer to models trained with verifier-based RL alone than to their weak teachers, suggesting that teacher guidance accelerates rather than redirects the student's own optimization. Results in conventional strong-to-weak distillation further demonstrate that OPRD effectively combines verifier-driven policy optimization with teacher guidance regardless of capacity ordering.

Momentum

Related topics

Latest papers 309