RL Post-Training

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 206% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

DAPD: Dual-Anchored Policy Distillation
Aug 3, 2026Jianyu Wu, Yizhou Wang, Encheng Su +2On-Policy Self-Distillation Language Model Distillation
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong +1
RMSWeb: Reflection, Failure-Mode Mining, and Salvage-DS for Web Agent Reinforcement Learning
Jul 31, 2026Chengbo Liu, Lifang Zhou, Ruijie Yan +8Reward Shaping Offline RL
1Microsoft, Beijing, China · 2Southeast University, Nanjing, China
Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning
Jul 31, 2026Xujun Che, Yuchen Yuan, Weida Zhao +1Reinforcement Learning KL-Regularized RL
Department of Cybersecurity, University of North Carolina at Charlotte, Charlotte, NC, USA · Department of Information Sciences and Technology, George Mason University, Fairfax, VA, USA · Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA +1
Cross-Benchmark Generalization in Long-Horizon Agents
Jul 31, 2026Sushant Mehta, Logan Ritchie, Liudas Panavas +1Long-Horizon Agent Evaluation RL Post-Training
Surge AI
X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching
Jul 30, 2026Tianyu Yang, Yiming Zeng, Wenzhe Cai +5Robot Navigation Policy Optimization
1Fudan University · 3Sun Yat-sen University · 2Shanghai AI Laboratory +1
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
Jul 30, 2026Tiangang Li, Xiangbo TianRL for Language Models LLM Alignment
School of Computer Science, Wuhan University, Wuhan, China
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Jul 30, 2026Xingjian Wu, Junlin Liu, Xingchen Liu +6Agentic RL RL for Language Model Reasoning
Meituan
TAPO: Transition-Aware Policy Optimization for LLM Agents
Jul 30, 2026Cong Li, Peixi Peng, Yisen Zhao +4Reinforcement Learning RL Post-Training
School of Electronic and Computer Engineering, Peking University · Pengcheng Laboratory
ReCo: Reweighting GRPO Against Distributional Concentration
Jul 29, 2026Junoh Park, Junseo Hwang, Wonguk Cho +1Reasoning Diversity RL for Language Model Reasoning
Graduate School of Data Science, Seoul National University
RLPF: Reinforcement Learning from Performance Feedback for Code Generation
Jul 29, 2026Huihao Jing, Haozhe Cui, Wenbin Hu +9RL for Code Generation Reinforcement Learning
The Hong Kong University of Science and Technology (HKUST)
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
Jul 29, 2026Hei Yi Mak, Shadan Golestan, Hoang Le +10LLM Quantization 4-Bit Quantization
Huawei
Cutting AI Datacenter Energy with Reinforcement Learning: Measured Power Control of LLM Training from One GPU to the Fleet
Jul 27, 2026Eliseo CurcioRL Control Energy-Efficient ML
Advanced Department of Artificial Intelligence and Energy – New York
Deconstructing Off-Policy Ratios: Entropy-Normalized Trust Regions for Asynchronous Reinforcement Learning
Jul 24, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +5Asynchronous RL RL for Language Model Reasoning
Beijing University of Posts and Telecommunications · Baidu Inc. · Peking University +2
Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs
Jul 24, 2026Zixuan Ren, Jinliang Lu, Junhong Wu +5RL for Language Models Language Model Merging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Baidu Inc., Beijing, China
QLPO: Quadrant-weighted Sampling for Length-aware Policy Optimization
Jul 23, 2026Siwei Chen, Siqi Chen, Xupeng Miao +1Overthinking in Language Models RL for Language Model Reasoning
School of Computer Science, Peking University · Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems · Department of Electronic Engineering, Tsinghua University +1
Closing the Lab-to-Store Gap: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids
Jul 22, 2026Roger Sala Sisó, Tiago Silvério, Jakob Sand +1Robotic Manipulation Vision-Language-Action Models
HIVE Robots, Denmark · Department of Engineering Technology, Technical University of Denmark, Denmark
Rewarding Better Thinking for LLM Preference Alignment
Jul 22, 2026Xubo Liu, Wenya Guo, Ruxue Yan +2Process Reward Models Credit Assignment in RL
VCIP, DISSec, College of Computer Science, Nankai University, Tianjin, China
ISO: An RLVR-Native Optimization Stack
Jul 21, 2026Hanqing Zhu, Wenyan Cong, Zhizhou Sha +8Reinforcement Learning with Verifiable Rewards RL Post-Training
The University of Texas at Austin · UIUC · Emory University +3
REGEN: Replay-recycling for Expert-to-Generalist distillation with Offline Reinforcement Learning
Jul 21, 2026Yunjie Chen, Xiaoxin Chen, Fang WangRL for Language Models RL for Language Model Reasoning
vivo AI Lab, Shenzhen, China · Department of Computer Science, Sun Yat-Sen University, Guangzhou, China
Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning
Jul 21, 2026Junyao Yang, Yucheng Shi, Zongxia Li +6Asynchronous RL Proximal Policy Optimization
1Tencent Hy LLM Frontier · 2National University of Singapore · University of Maryland, College Park +3
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Jul 20, 2026Yuxin Xiong, Xunyi Jiang, Rohan Surana +8Reinforcement Learning RL for Language Model Reasoning
University of California San Diego · 2Adobe Research
OR Else: A Differentiable Trust Region for Policy Optimization
Jul 20, 2026Chinmay Rane, Kanishka Tyagi, Michael ManryRL for Language Models Group Relative Policy Optimization
Quantiphi Inc Marlborough, MA, 01752 · Self Machines Inc San Francisco, California, 94402 · Department of Electrical Engineering The University of Texas at Arlington Arlington, TX, 76010
LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Jul 20, 2026Tianzhu Ye, Li Dong, Guanheng Chen +4Reinforcement Learning Rubric-Based RL
Microsoft Research · Tsinghua University · Peking University
Distilled Reinforcement Learning for LLM Post-training
Jul 19, 2026Chen Wang, Zhaochun Li, Jionghao Bai +4RL for Language Model Reasoning Cross-Architecture Knowledge Distillation
College of Elite Engineers, Nankai University · Zhongguancun Academy · Beijing Institute of Technology +4
When Does Muon Help Agentic Reinforcement Learning?
Jul 17, 2026Kai Ruan, Jinghao Lin, Zihe Huang +4Agentic RL Muon Optimizer
Gaoling School of Artificial Intelligence, Renmin University of China · Independent Researcher · Institute of Computing Technology, Chinese Academy of Sciences +3
Understanding Reasoning from Pretraining to Post-Training
Jul 17, 2026Jingyan Shen, Ang Li, Salman Rahman +4Language Model Pretraining RL for Language Model Reasoning
New York University · Columbia University · University of California, Los Angeles +2
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Jul 16, 2026Changhai Zhou, Kieran Liu, Yuhua Zhou +17Efficient Language Model Training Long-Context Modeling
MindLab · Fudan University
TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
Jul 15, 2026Leitian Tao, Baolin Peng, Wenlin Yao +5Credit Assignment in RL Step-Level Credit Assignment in RL
University of Wisconsin–Madison · Microsoft Research
Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape Pre-, Intra-, and Post-CoT Calibration
Jul 15, 2026Shuhao Li, Guodong Du, Anhao Zhao +3Confidence Estimation in Language Models Language Model Calibration
Eastern Institute of Technology, Ningbo · The Hong Kong Polytechnic University
GFlowRL: Scaling Distribution-Matching RL to Large Language Models
Jul 15, 2026Xiaodong Liu, Michael Xu, Jack W. Stokes +3RL for Code Generation Reinforcement Learning
Microsoft Research

Aug 3, 2026, cs.AIJ/K move · Enter open · S save

DAPD: Dual-Anchored Policy Distillation

Jianyu Wu, Yizhou Wang, Encheng Su, Chen Tang+1

Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong+1

On-Policy Self-Distillation Language Model Distillation RL Post-Training Policy Distillation

Open paper AskPDF arXiv

On-policy (self) distillation (OPSD) is increasingly adopted for language-model post-training. It strengthens the teacher with privileged information but can induce a privilege illusion: the student learns privilege-dependent behavior it cannot reproduce from its inference-time context, yet behaves as if the training-time privileged information remained available, ultimately degrading performance. In this paper, we identify information asymmetry between the privileged teacher and the student at inference as the root cause of this failure in OPSD. To resolve this asymmetry, we propose Dual-Anchored Policy Distillation (DAPD), a unified framework with two levels of anchoring. Dual-Path Anchoring (DPA) introduces a self-conditioned bridge and aligns reference and rollout behavior along two matched-information paths, preventing privilege-dependent behavior from being transferred to the inference-time student. Dual-Source Anchoring (DSA) applies these paths in both reference-to-rollout and rollout-to-reference directions, reducing reliance on privileged reference guidance while preserving correctness supervision. Extensive experiments show that DAPD significantly alleviates privilege illusion, outperforming OPSD on Qwen3-4B by +2.00 points on average across tasks. Notably, its gains persist across scales, reaching +2.69 at 4B and +2.78 at 32B.

Momentum

Related topics

Latest papers 309