RL Post-Training

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 206% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 309

All topics

Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models
May 11, 2026Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken +2Diffusion Model Alignment Flow Matching
1TU Munich, MCML · 2MIT CSAIL · 3King’s College London +2
What should post-training optimize? A test-time scaling law perspective
May 11, 2026Muheng Li, Jian Qian, Wenlong MouTest-Time Scaling Best-of-N Sampling
Department of Statistical Sciences, University of Toronto† · Department of AI and Data Science, The University of Hong Kong ⋄
Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
May 11, 2026Xuexiang Wen, Hang Yu, Linchao Zhu +1Intrinsic Reward Methods Reinforcement Learning
1Zhejiang University · 2Ant Group
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
May 11, 2026Changhao Li, Yuchen Zhuang, Chenxiao Gao +4RL for Language Models Inference-Time Optimization
Georgia Institute of Technology
BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learning
May 9, 2026Yuhang Xu, Kaibin Tian, Yang Tian +6Reinforcement Learning Speculative Decoding
1Shanghai Jiao Tong University · 2Bytedance
$ξ$ -DPO: Direct Preference Optimization via Ratio Reward Margin
May 9, 2026Zhengyuan Fan, Zhonghua Wu, Yuxuan Du +1Pairwise Preference Learning Direct Preference Optimization
School of Computer Science, Northwestern Polytechnical University
Reinforcement Learning for Scalable and Trustworthy Intelligent Systems
May 8, 2026Guangchen LanFederated RL Privacy-Preserving Language Models
Elmore Family School of Electrical and Computer Engineering, West Lafayette, Indiana · THE PURDUE UNIVERSITY GRADUATE SCHOOL
Not All Tokens Learn Alike: Attention Entropy Reveals Heterogeneous Signals in RL Reasoning
May 8, 2026Gengyang Li, Zheng-Fan Wu, Siqi Bao +1Token-Level Credit Assignment RL for Language Model Reasoning
1Baidu · School of Software and Microelectronics, Peking University · School of Computer Science, Peking University
Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
May 8, 2026Yuheng Zhang, Chenlu Ye, Shuowei Jin +4RL for Language Model Reasoning Importance Sampling
UIUC · University of Michigan · Amazon
Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training
May 8, 2026Chen Wang, Hexuan Deng, Yining Zhang +5Overthinking in Language Models Reinforcement Learning with Verifiable Rewards
College of Software, Nankai University · 2Zhongguancun Academy · 3Harbin Institute of Technology +4
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
May 8, 2026Xiaoze Liu, Dhananjay Ram, Yuting Zhang +3RL for Language Models Reinforcement Learning
1Purdue University · *Work done during an internship at Amazon. · 2AWS Agentic AI
Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
May 8, 2026Pingbang Hu, Xueshen Liu, Z. Morley Mao +1Supervised Fine-Tuning Training Data Selection
University of Illinois Urbana–Champaign · University of Michigan
How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
May 7, 2026Rui Zhu, Weiheng Bai, Qiushi Wu +3LLM Alignment KV-Cache Compression
Yale University · University of Minnesota Twin Cities · Indiana University Bloomington
A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment
May 7, 2026Hao YuPairwise Preference Learning LLM Alignment
Department of Automation, Tsinghua University
A $^2$ TGPO: Agentic Turn-Group Policy Optimization with Adaptive Turn-level Clipping
May 7, 2026Dingwei Chen, Zefang Zong, Zhipeng Ma +5Credit Assignment in RL Policy Gradient Methods
Tencent Inc · The Chinese University of Hong Kong · Shenzhen MSU-BIT University
OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Models
May 7, 2026Jaehoon Kim, Dongha LeeReinforcement Learning with Verifiable Rewards On-Policy Distillation
Yonsei University
Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex
May 7, 2026Yun Qu, Qi Wang, Yixiu Mao +11RL for Language Model Reasoning Group Relative Policy Optimization
Department of Automation, Tsinghua University · 2LLM Department, Tencent
Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs
May 7, 2026Yujia Chen, Yang Ye, Xiao Chu +2RL for Code Generation Reinforcement Learning
Harbin Institute of Technology, Shenzhen · Huawei Cloud Computing Technologies Co., Ltd.
CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies
May 6, 2026Keyu Chen, Nanfei Ye, Yida Wang +4Reinforcement Learning Autonomous Driving Planning
School of Vehicle and Mobility, Tsinghua University · Li Auto Inc
Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models
May 4, 2026Inoussa Mouiche, Abderaouf BahiLLM Alignment Direct Preference Optimization
WASP Lab, School of Computer Science University of Windsor, ON N9B 3P4
Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
May 3, 2026Kunvar ThamanAI Agent Reliability Reward Hacking
Independent Researcher
Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
May 3, 2026Long Zhao, Qinghe Wang, Jiaan Zhu +5Tensor Parallelism LLM Inference Acceleration
Anhui University · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center · University of Science and Technology of China +2
Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies
May 1, 2026Yi Wang, Xinchen Li, Pengwei Xie +13Robot Policy Learning Robotic Manipulation
Shanghai Innovation Institute. · AGIBOT Finch. · Columbia University.
Exploration Hacking: Can LLMs Learn to Resist RL Training?
Apr 30, 2026Eyon Jang, Damon Falck, Joschka Braun +6RL Exploration LLM Safety
MATS · UC San Diego · Anthropic +1
Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training
Apr 30, 2026Jasmine Brazilek, Juliana SeawellMoral Reasoning in Language Models RL Post-Training
†Compassion Aligned Machine Learning (CaML)
Co-Evolving Policy Distillation
Apr 29, 2026Naibin Gu, Chenxu Yang, Qingyi Si +7Reinforcement Learning with Verifiable Rewards Multi-Teacher Knowledge Distillation
Institute of Information Engineering, CAS, 2School of Cyber Security, UCAS, 3JD.COM
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
Apr 29, 2026Hayate Iso, Tiyasa Mitra, Sudipta Mondal +15Reinforcement Learning Speculative Decoding
NVIDIA
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Apr 29, 2026Bolian Li, Yifan Wang, Yi Ding +3RL for Language Models Reinforcement Learning
Purdue University, West Lafayette, IN, USA
DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training
Apr 29, 2026Tianhao Hu, Xiangcheng Liu, Youshao Xiao +15RL for Language Models Asynchronous RL
Meituan, China
Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models
Apr 27, 2026Dan Shi, Zhuowen Han, Simon Ostermann +3Reinforcement Learning RL for Language Model Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · Saarland University, Saarbrücken, Germany

May 11, 2026, cs.LGJ/K move · Enter open · S save

Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models

Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken, Carles Domingo-Enrich+1

1TU Munich, MCML · 2MIT CSAIL · 3King’s College London+2

Diffusion Model Alignment Flow Matching RL for Diffusion Models RL for Generative Models RL Post-Training

Open paper AskPDF arXiv

Diffusion and flow-matching models scale because pretraining is supervised regression: a clean sample is noised analytically, and a model regresses against a closed-form target. RL post-training aligns the model with a reward. In image generation, this makes samples compose objects correctly, render text legibly, and match human preferences. Existing methods rely on costly SDE rollouts, reward gradients, or surrogate losses, sacrificing pretraining's regression structure. We show that the structure extends to RL post-training. Under KL-regularized reward maximization, the optimal generative process tilts the clean-endpoint distribution towards samples with higher reward and leaves the noising law unchanged. Combining this with the adjoint-matching optimality condition and a REINFORCE identity, we derive Reinforce Adjoint Matching (RAM): a consistency loss that corrects the pretraining target with the reward. At each step, we draw a clean endpoint from the current model, evaluate its reward, noise it as in pretraining, and regress. No SDE rollouts, backward adjoint sweeps, or reward gradients are required. Like the pretraining objective, RAM is simple and scales. On Stable Diffusion 3.5M, RAM achieves the highest reward on composability, text rendering, and human preference, reaching Flow-GRPO's peak reward in up to

50\times

fewer training steps.

Momentum

Related topics

Latest papers 309