RL for Diffusion Models

RL: Reinforcement Learning

The field studies how to adapt diffusion generators using task rewards, human preferences, or environment feedback rather than relying only on imitation or supervised denoising. Common approaches treat denoising steps as policy actions and use PPO or GRPO policy gradients, reward-weighted flow or score matching, distribution matching, and critic-guided control; distillation can make these updates cheaper. Applications span image and video alignment, reasoning in discrete diffusion language models, and robot policies, evaluated on benchmarks such as GenEval, GSM8K, and MuJoCo tasks. Key tensions include estimating useful learning signals across denoising steps, preventing reward hacking without losing diversity or fidelity, and achieving gains without expensive rollouts.

Latest papers

182 papers27 papers in the last four weeks, up 80% on the four weeks before. 0.2% of all new papers.

Skillful and realistic precipitation forecasting with reinforced diffusion
Oct 28, 2026Ozan Sener, Anand Avati, David Hafner +7Precipitation Forecasting Diffusion Models
Apple · ETH Zurich · Colorado State University
DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models
Oct 26, 2026Hongze Chen, Miao Pan, Lizhou Liang +7Diffusion Language Model Training RL for Diffusion Models
Zhejiang University · National University of Singapore · SAP
Doob-Based Generative Policy Optimization
Oct 20, 2026Maojiang Su, Qijie Zhu, Yu Xiao +4Policy Optimization RL for Diffusion Models
Department of Computer Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University · Department of Statistics and Data Science, Northwestern University +1
DualHeads: Reward-tilted few-step diffusion made simple
Oct 20, 2026Yuanzhi Zhu, Eleftherios Tsonis, Simo Alami Chehboune +3Diffusion-Based Image Generation Diffusion Model Fine-Tuning
LIX, École Polytechnique, CNRS, IPP · Inria at Univ. Grenoble Alpes, CNRS, LJK
When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning
Oct 20, 2026Wang RuLatent Space Geometry Diffusion Model Fine-Tuning
Department of Chemical Engineering, Tsinghua University
Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning
Oct 20, 2026Wei Cai, Jian Zhao, Yuchen Yuan +1Hybrid Language Models Parallel Reasoning in Language Models
Peking University · Institute of Artificial Intelligence (TeleAI), China Telecom
Distribution-Based Regularization for Reward Fine-Tuning Diffusion and Flow Models
Oct 17, 2026Romeo Passaro, Jeongjae Lee, Jong Chul Ye +4Generative Modeling Diffusion Model Alignment
University of Oxford · AITHYRA · KAIST +2
DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation
Oct 16, 2026Jiayuan Wang, Tongyao PangRL for Image Generation RL for Diffusion Models
Qiuzhen College, Tsinghua University, Beijing 100084, China · Yau Mathematical Sciences Center, Shuangqing Complex, Tsinghua University, Beijing 100084, China
Routing Policy Optimization for Online Diffusion Reinforcement Learning
Oct 15, 2026Qiuhong Shen, Yiding Yang, Yuxin Zhang +8RL for Image Generation RL for Diffusion Models
National University of Singapore · ByteDance
Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
Oct 15, 2026Zhiyuan Ma, Jiabao Wei, Tianyu Wang +21RL for Diffusion Models RL for Generative Models
Huazhong University of Science and Technology · Beijing Institute of Technology · Tsinghua University +9
ForkGRPO: Efficient Exploration and Fine-Grained Credit Assignment for Diffusion Language Models
Oct 14, 2026Zhaoyan Jia, Chengwei Zhou, Gourav Datta +1RL for Language Model Reasoning RL for Diffusion Models
University of Southern California, Los Angeles, USA · Case Western Reserve University, Cleveland, USA
Outcome-Routed Distillation and Reinforcement Learning for Diffusion Language Models
Oct 13, 2026Wentao Lu, Tianyu Zhu, Jesse ClarkDiffusion Language Model Distillation Diffusion Language Models
Celeris San Francisco, CA
A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs
Oct 13, 2026Jiahao Kuang, Yangfan Gao, Tao Gui +4Parallel Decoding for Diffusion Language Models Parallel Decoding
East China Normal University · Fudan University
RL Post-Training Diffusion Language Models Along Their Own Generation Paths
Oct 12, 2026Vaibhav Singh, Eugene BelilovskyLanguage Model Post-Training RL for Diffusion Models
Mila, Concordia University
CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance
Oct 10, 2026Shutong Ding, Zhaoyang Liu, Yimiao Zhou +3Diffusion Policy Robotic RL
ShanghaiTech University, Shanghai, China · InstAdapt, Shanghai, China
MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design
Oct 9, 2026Jixin Tang, Ji Guo, Jun ZhaoInverse Design Reward Design for RL
Fudan University
DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
Oct 8, 2026Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang +6Diffusion Model Fine-Tuning RL for Diffusion Models
HPC-AI Lab, National University of Singapore · Kling AI · HKUST
Reachability-Aware Diffusion Policy Optimization
Oct 5, 2026Hikmet Simsir, Kutay Demiray, Ozgur S. OguzConstrained RL Diffusion Policy
Department of Computer Engineering, Bilkent University
MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation
Oct 4, 2026Yunyi Chen, Chenru Wang, Xinyi Ye +2Diffusion Model Alignment Dataset Distillation
AGI Lab, Westlake University · Eindhoven University of Technology · Guangdong University of Finance and Economics
IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
Oct 2, 2026Vladislav Gromadskii, David Li, Samson Gourevitch +4Diffusion Model Fine-Tuning Masked Diffusion Models
Applied AI Institute, Moscow, Russia · MBZUAI · CMAP, École Polytechnique +3
Token-Level Video Reinforcement Learning
Oct 1, 2026Yifan Wang, Gordon Guocheng Qian, Yanyu Li +2Video Diffusion Models RL for Video Generation
Northeastern University
iADD: Improving Alignment and Diversity in Diffusion Policy Optimization
Oct 1, 2026Ashok Prasad Neupane, Saugat Adhikari, Pramish Paudel +2Diffusion Model Alignment Generative Modeling
Pulchowk Campus, IOE, Tribhuvan University, Lalitpur, Nepal · INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · NAAMII, Kathmandu, Nepal
Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models
Sep 30, 2026Yue Yu, Bowen Zuo, David Crandall +2RL for Language Model Reasoning Group Relative Policy Optimization
Indiana University · University of California, Riverside
Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
Sep 30, 2026Stipe Frković, Metod Jazbec, Christian A. NaessethMasked Diffusion Models Speculative Decoding for Diffusion Language Models
University of Amsterdam · UvA-Bosch Delta Lab, University of Amsterdam
CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models
Sep 30, 2026Shu Yu, Chaochao LuDiffusion Model Fine-Tuning Compositional T2I Generation
Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Fudan University, Shanghai, China
ExploreNet: Learning Where to Explore in Diffusion GRPO
Sep 29, 2026Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1Diffusion Model Fine-Tuning RL Exploration
University of Washington
MeanFlowAdvantage: Stable Reward Fine-Tuning for Few-Step Average-Velocity Generators
Sep 29, 2026Haocheng Tang, Tianchi Xie, Xingqiao LinFlow Matching MeanFlow
Northeastern University Boston, MA 02115, USA · Tsinghua University Beijing, 100084, PRC · Carnegie Mellon University Pittsburgh, PA 15213, USA
Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL
Sep 29, 2026Songlin Yang, Xiaotong Zhao, Jiacheng Zhang +5Multi-Objective Reinforcement Learning Cross-Modal Alignment
Tencent Video · The University of Hong Kong
Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA Generalization
Sep 28, 2026Zhiyuan Ma, Jiaming Li, Lingzhen Li +7Reinforcement Learning Vision-Language-Action Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policies
Sep 27, 2026Jialeng Ni, Nathan Zhao, Kunpeng SongRobot Policy Learning Reinforcement Learning
University of Michigan, Ann Arbor, MI, USA. · XPENG Robotics, Santa Clara, CA, USA.

Oct 28, 2026J/K move · Enter open · S save

Skillful and realistic precipitation forecasting with reinforced diffusion

Ozan Sener, Anand Avati, David Hafner, Jierui Lin+6

Apple · ETH Zurich · Colorado State University

Precipitation Forecasting Diffusion Models RL for Diffusion Models

Open paper AskPDF arXiv

Accurate precipitation forecasting remains elusive as physics-based models struggle to resolve mesoscale convection and machine learning often yields physically inconsistent, blurred results that consistently underestimate extreme events. We present FitzRoy, the first system that simultaneously achieves state-of-the-art predictive skill and physical realism over the continental United States. Through a reinforced diffusion framework, where a conditional discrete diffusion model is refined via reinforcement learning, FitzRoy effectively resolves the long-standing trade-off between pointwise accuracy and structural fidelity. Across lead times up to 24 hours, FitzRoy consistently matches or exceeds state-of-the-art models while preserving structural realism and capturing extreme events. Its low-latency inference further enables deployment in real-time operational settings. In case studies such as the 2025 Texas Hill Country floods and the 2026 Missouri floods, FitzRoy signals record-breaking accumulations that existing models largely missed. By bridging the gap between realism and skill, FitzRoy establishes a new frontier for high-fidelity precipitation forecasting.