RL for Diffusion Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 467% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 120

All topics

Diffusion-APO: Trajectory-Aware Direct Preference Alignment for Video Diffusion Transformers
May 8, 2026Jingyuan Zhu, Biaolong Chen, Le Zhang +3Diffusion Model Alignment Video Diffusion Models
Alibaba Group
DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
May 8, 2026Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade +4Group Relative Policy Optimization Credit Assignment in RL
The Ohio State University · Apple
MARBLE: Multi-Aspect Reward Balance for Diffusion RL
May 7, 2026Canyu Zhao, Hao Chen, Yunze Tong +3Multi-Objective Reinforcement Learning Diffusion Model Alignment
Zhejiang University · Work done during an internship at HiThink. · HiThink +1
MotionGRPO: Overcoming Low Intra-Group Diversity in GRPO-Based Egocentric Motion Recovery
May 7, 2026Nanjie Yao, Junlong Ren, Wenhao Shen +1Diffusion Model Guidance Markerless Motion Capture
The Hong Kong University of Science and Technology (Guangzhou), China · Nanyang Technological University, Singapore
Leveraging Verifier-Based Reinforcement Learning in Image Editing
Apr 30, 2026Hanzhong Guo, Jie Wu, Jie Liu +6Reward Modeling Reinforcement Learning
School of Computing and Data Science, The University of Hong Kong · ByteDance Seed · CUHK +1
AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generation
Apr 29, 2026Xu Wang, Zexian Li, Litong Gong +2Diffusion Model Distillation Few-Step Diffusion Sampling
Shanghai Jiao Tong University · Alimama Tech
A Systematic Post-Train Framework for Video Generation
Apr 28, 2026Zeyue Xue, Siming Fu, Jie Huang +9Diffusion Model Alignment Video Diffusion Models
The University of Hong Kong · JD Explore Academy · Tsinghua University +2
V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think
Apr 25, 2026Bingda Tang, Yuhui Zhang, Xiaohan Wang +3Diffusion Model Alignment Group Relative Policy Optimization
Stanford University · Tsinghua University · Amazon FAR +1
ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control
Apr 22, 2026Shelly Golan, Michael Finkelson, Ariel Bereslavsky +2Multi-Objective Reinforcement Learning RL for Image Generation
Tel Aviv University · Lightricks · Adobe Research
FASTER: Value-Guided Sampling for Fast RL
Apr 21, 2026Perry Dong, Alexander Swerdlow, Dorsa Sadigh +1Diffusion Sampling Diffusion Policy
Stanford University
HP-Edit: A Human-Preference Post-Training Framework for Image Editing
Apr 21, 2026Fan Li, Chonghuinan Wang, Lina Lei +9Diffusion Model Alignment Image Editing Evaluation
Huawei Noah’s Ark Lab · Nankai University · Harbin Institute of Technology
Learning to Credit the Right Steps: Objective-aware Process Optimization for Visual Generation
Apr 21, 2026Rui Li, Ke Hao, Yuanzhi Liang +4Multi-Objective Reinforcement Learning Video Diffusion Models
University of Science and Technology of China HeFei, China · Shanghai Jiao Tong University ShangHai, China · TeleAI ShangHai, China
Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning
Apr 21, 2026Linwei Dong, Ruoyu Guo, Ge Bai +3Diffusion Model Distillation Few-Step Diffusion Sampling
1Zhejiang University · 2Bytedance Inc. · 3Zhejiang Lab
Discrete Tilt Matching
Apr 20, 2026Yuyuan Chen, Shiyi Wang, Peter Potaptchik +2Masked Diffusion Models RL for Language Model Reasoning
1Harvard University · University of Oxford · 3IAIFI +1
UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models
Apr 20, 2026Jiaqi Wang, Haoge Deng, Ting Pan +5T2I Generation Group Relative Policy Optimization
Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence
RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
Apr 16, 2026Hao Gao, Shaoyu Chen, Yifan Zhu +4Diffusion-Based Planning RL for Diffusion Models
Huazhong University of Science & Technology · Horizon Robotics
SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning
Mar 18, 2026Xi Ye, Wenjia Yang, Yangyang Xu +4Diffusion Model Alignment Video Diffusion Models
Tsinghua University · Ant Group · University of Chinese Academy of Sciences
Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
Mar 13, 2026David McAllister, Miika Aittala, Tero Karras +4RL for Image Generation RL for Diffusion Models
UC Berkeley · NVIDIA
LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models
Feb 15, 2026Shufan Li, Yuchen Zhu, Jiuxiang Gu +6Unified Multimodal Models Multimodal Reasoning
1Adobe · 2UCLA · 3Georgia Tech
Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
Feb 4, 2026Jaemoo Choi, Yuchen Zhu, Wei Guo +6T2I Generation Diffusion Model Fine-Tuning
Georgia Institute of Technology · National University of Singapore · Nanjing university.
ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule
Jan 26, 2026Yilie Huang, Wenpin Tang, Xunyu ZhouDiffusion Sampling Adaptive Sampling
Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027, USA. · Department of Industrial Engineering and Operations Research & Data Science Institute, Columbia University, New York, NY 10027, USA.
ReDiF: Resource-Efficient Few-Step Diffusion Distillation via Reinforcement Learning
Dec 28, 2025Amirhossein Tighkhorshid, Zahra Dehghanian, Hamid R. RabieeDiffusion Distillation RL for Diffusion Models
Computer Engineering Sharif University of Technology
Multi-dimensional Preference Alignment by Conditioning Reward Itself
Dec 11, 2025Jiho Jang, Jinyoung Kim, Kyungjune Baek +1Diffusion Model Alignment Diffusion Model Guidance
Seoul National University · Sejong University
Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
Dec 1, 2025Sebastian Sanokowski, Kaustubh Patil, Majid KhadivRL Control Diffusion Policy
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University Munich · Practical Project Student Exchange Program, Technical University Munich · MIT World Peace University
Distribution Matching Distillation Meets Reinforcement Learning
Nov 17, 2025Dengyang Jiang, Dongyang Liu, Zanyi Wang +12Diffusion Model Alignment Diffusion Model Distillation
1HKUST · 2Alibaba Group · 3SIAT, CAS +3
Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
Oct 9, 2025Yuchen Zhu, Wei Guo, Jaemoo Choi +4RL for Language Model Reasoning Policy Optimization
Georgia Institute of Technology.
Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models
Sep 29, 2025Shuchen Xue, Chongjian Ge, Shilong Zhang +2Score Matching Diffusion Model Alignment
UCAS · Adobe Research · HKU +1
MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE
Jul 29, 2025Junzhe Li, Yutao Cui, Tao Huang +8Diffusion Model Alignment Flow Matching
Hunyuan, Tencent, China · School of Computer Science, Peking University, China · Computer Center, Peking University, China +2
DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving
Jul 5, 2025Ziying Song, Lin Liu, Hongyu Pan +7End-to-End Autonomous Driving Diffusion-Based Planning
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · School of Artificial Intelligence (School of Software), Yanshan University, Qinhuangdao, China · Horizon Robotics +3
DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
Date pendingQuanhao Li, Junqiu Yu, Kaixun Jiang +7Diffusion Model Distillation Multi-Task RL
Fudan University, China · Alibaba Group, China

May 8, 2026, cs.CVJ/K move · Enter open · S save

Diffusion-APO: Trajectory-Aware Direct Preference Alignment for Video Diffusion Transformers

Jingyuan Zhu, Biaolong Chen, Le Zhang, Aixi Zhang+2

Alibaba Group

Diffusion Model Alignment Video Diffusion Models Direct Preference Optimization RL for Diffusion Models

Open paper AskPDF arXiv

Efficiently aligning large-scale video diffusion models with human intent requires a scalable and trajectory-aware pathway that bridges the inherent discrepancy between training noise distributions and practical inference trajectories. While existing paradigms such as Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) attempt to address this, they are often hindered by either reliance on bias-prone, complex reward models or suboptimal timestep sampling. In this paper, we propose Diffusion-APO (Aligned Preference Optimization), a trajectory-aware algorithm that resolves this misalignment by synchronizing training noise with inference-time denoising paths to maximize gradient signal efficacy. To translate this algorithmic innovation into a practical solution, we introduce a unified and modular RLHF framework that integrates online ranking, half-online anchoring, offline refinement, and distillation-aware drift correction. This framework enables flexible, multi-stage preference alignment across diverse data and computational constraints without relying on scalar-reward-based policy gradients. Through extensive experiments, we demonstrate that Diffusion-APO consistently outperforms standard baselines in visual quality and instruction following, while effectively preserving generative fidelity during model acceleration, providing a robust, end-to-end pathway for scalable video diffusion alignment.

Momentum

Related topics

Latest papers 120