RL for Diffusion Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 467% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 120

All topics

Reachability-Aware Diffusion Policy Optimization
Oct 5, 2026Hikmet Simsir, Kutay Demiray, Ozgur S. OguzConstrained RL Diffusion Policy
Department of Computer Engineering, Bilkent University
MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation
Oct 4, 2026Yunyi Chen, Chenru Wang, Xinyi Ye +2Diffusion Model Alignment Dataset Distillation
AGI Lab, Westlake University · Eindhoven University of Technology · Guangdong University of Finance and Economics
Token-Level Video Reinforcement Learning
Oct 1, 2026Yifan Wang, Gordon Guocheng Qian, Yanyu Li +2Video Diffusion Models RL for Video Generation
Northeastern University
iADD: Improving Alignment and Diversity in Diffusion Policy Optimization
Oct 1, 2026Ashok Prasad Neupane, Saugat Adhikari, Pramish Paudel +2Diffusion Model Alignment Generative Modeling
Pulchowk Campus, IOE, Tribhuvan University, Lalitpur, Nepal · INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · NAAMII, Kathmandu, Nepal
Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models
Sep 30, 2026Yue YU, Bowen Zuo, David Crandall +2RL for Language Model Reasoning Group Relative Policy Optimization
Indiana University · University of California, Riverside
Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
Sep 30, 2026Stipe Frković, Metod Jazbec, Christian A. NaessethMasked Diffusion Models Speculative Decoding for Diffusion Language Models
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam
CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models
Sep 30, 2026Shu Yu, Chaochao LuDiffusion Model Fine-Tuning Compositional T2I Generation
Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Fudan University, Shanghai, China
ExploreNet: Learning Where to Explore in Diffusion GRPO
Sep 29, 2026Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1Diffusion Model Fine-Tuning RL Exploration
University of Washington
MeanFlowAdvantage: Stable Reward Fine-Tuning for Few-Step Average-Velocity Generators
Sep 29, 2026Haocheng Tang, Tianchi Xie, Xingqiao LinFlow Matching MeanFlow
Northeastern University Boston, MA 02115, USA · Tsinghua University Beijing, 100084, PRC · Carnegie Mellon University Pittsburgh, PA 15213, USA
Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL
Sep 29, 2026Songlin Yang, Xiaotong Zhao, Jiacheng Zhang +5Multi-Objective Reinforcement Learning Cross-Modal Alignment
Tencent Video · The University of Hong Kong
Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA Generalization
Sep 28, 2026Zhiyuan Ma, Jiaming Li, Lingzhen Li +7Reinforcement Learning Vision-Language-Action Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policies
Sep 27, 2026Jialeng Ni, Nathan Zhao, Kunpeng SongRobot Policy Learning Reinforcement Learning
University of Michigan, Ann Arbor, MI, USA. · XPENG Robotics, Santa Clara, CA, USA.
Elucidating the Design Space of Regression-based Diffusion Reinforcement Learning
Sep 27, 2026Toyota Li, David Zhao, Alan ZhaoReinforcement Learning Constrained RL
Tencent
DOHF: Online Diffusion Fine-tuning with Doob's $h$ -transform Guidance
Sep 25, 2026Zhengyi Guo, Jiayuan Sheng, Wenpin Tang +1Diffusion Model Alignment Diffusion Model Guidance
AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation
Sep 24, 2026Zhiyu Xu, Weilong Yan, Yufei Shi +4Audio-Video Generation Audio-Visual Alignment
Shanghai AI Laboratory · The Hong Kong Polytechnic University · National University of Singapore +2
Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models
Sep 22, 2026Xiaoyi Yu, Enver Sangineto, Pei Fu +6RL for Diffusion Models Diffusion Language Models
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · MiLM Plus, Xiaomi Inc., Beijing, China · University of Modena and Reggio Emilia, Italy +1
CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models
Sep 14, 2026Blake Olson, Yuhang Song, Emmett McQuinn +1RL for Language Model Reasoning Curriculum RL
Google DeepMind
DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization
Sep 10, 2026Arjun Sohal, Yuchi Zhao, Miroslav Bogdanovic +1RL for Robotics Step-Level Credit Assignment in RL
University of Toronto, Toronto, ON, Canada · Vector Institute for Artificial Intelligence, Toronto, ON, Canada · Acceleration Consortium, University of Toronto, Toronto, ON, Canada +2
LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL
Sep 3, 2026Sijie Wang, Zhiqiang Tan, Xinrui Yang +1RL for Diffusion Models RL Post-Training
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen
Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Aug 14, 2026Yixian Xu, Yuanrui Zhang, Shengjie Luo +2Diffusion Model Alignment Policy Gradient Methods
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · ByteDance Seed
RL-Native Distillation: Exploiting Scored Trajectories for Few-Step Image Generation
Aug 10, 2026Yuhan Li, Fangao Zeng, Sicong Kang +7Diffusion Model Distillation Few-Step Diffusion Sampling
Shanghai Jiao Tong University · Taobao & Tmall group of Alibaba
PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model
Aug 7, 2026Renye Yan, Jikang Cheng, You Wu +4Intrinsic Reward Methods Reinforcement Learning
Peking University Beijing, China
Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models
Aug 7, 2026Renye Yan, Jikang Cheng, You Wu +4Diffusion Model Alignment Temporal Credit Assignment
Peking University Beijing, China · Stanford University Stanford, California, USA
Learning When to Stop: Prefix-Optimal Dynamic Diffusion Policies for Continuous Control
Aug 5, 2026Rohit Kumar Salla, Manoj Saravanan, Simon StepputtisDiffusion Policy Continuous Control
Virginia Polytechnic Institute and State University (Virginia Tech), Blacksburg, Virginia, USA
Latent Reward Registers for Diffusion Preference Alignment
Aug 4, 2026Yuanshen Guan, Zipeng Feng, Chengru Song +2Diffusion Model Alignment Diffusion Model Guidance
1Kling Team · University of Science and Technology of China
Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models
Aug 3, 2026Wonseok Lee, Jimyeong Kim, Jungmin Ko +1RL for Language Model Reasoning RL for Diffusion Models
Interdisciplinary Program in Artificial Intelligence, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Artificial Intelligence Institute, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Department of Intelligence and Information, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea
Learning Sampling Parameters for Diffusion Models
Jul 26, 2026Arisrei Lim, Yossi GandelsmanDiffusion Model Guidance Diffusion Model Sampling
Reve
JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models
Jul 20, 2026Ruiyi Ding, Jie Li, He Kang +3Backpropagation Diffusion Transformer
1KlingAI Research · University of Science and Technology of China · 2Fudan University +2
Reinforcement Learning: From Algorithms To Foundation Models
Jul 20, 2026Zihan DingReinforcement Learning RL for Video Generation
Department of Electrical Computer Engineering, Princeton University
Mask-Aware Policy Gradients for Diffusion Language Models
Jul 16, 2026Haran Raajesh, Kulin Shah, Adam Klivans +1Masked Diffusion Models RL for Language Model Reasoning
Department of Computer Science The University of Texas at Austin

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Reachability-Aware Diffusion Policy Optimization

Hikmet Simsir, Kutay Demiray, Ozgur S. Oguz

Department of Computer Engineering, Bilkent University

Constrained RL Diffusion Policy RL for Diffusion Models Model-Free RL Safe RL

Open paper AskPDF arXiv

Diffusion policies provide expressive action distributions for continuous-control reinforcement learning. However, safety-aware online diffusion policy optimization remains underexplored, particularly methods that use predictive reachability information without an explicit dynamics model. We propose Reachability-Aware Diffusion Policy Optimization (RADPO), a model-free method that combines predictive first-hit safety estimation with cumulative-cost budget feedback. RADPO learns a discounted first-hit reachability value that captures the discounted risk of a cost event, assigns larger weight to events that occur sooner, and uses this signal to shape the reward. A separate dual-like multiplier adjusts the shaping strength according to realized episodic costs relative to a prescribed budget. The diffusion actor improves through weighted denoising regression on candidate actions scored by the reward critic. Our approach requires neither a learned dynamics model, action gradients through the critics, nor differentiation through the reverse diffusion sampler. We establish theoretical properties of the reachability value and show that accumulated reachability penalty provides a conservative surrogate for future discounted cumulative cost. Across ten continuous-control safety tasks, RADPO achieves competitive reward-cost trade-offs, with substantial reductions in constraint violations on several tasks relative to the compared baselines. Our theoretical and empirical analysis supports that combining reachability with cumulative budget feedback is a viable approach to safety-aware diffusion policies.

Momentum

Related topics

Latest papers 120