Diffusion Model Alignment

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Explicit Critic Guidance for Aligning Diffusion Models
May 26, 2026Zhengyang Liang, Qihang Zhang, Ceyuan YangDiffusion Model Alignment Diffusion Model Guidance
University of Toronto · Vector Institute · The Chinese University of Hong Kong
Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
May 26, 2026Austin Wang, Jiaqi Han, Stefano Ermon +1Diffusion Model Alignment Preference Optimization
Caltech · Stanford University
Reinforcing Few-step Generators via Reward-Tilted Distribution Matching
May 25, 2026Yushi Huang, Xiangxin Zhou, Ruoyu Wang +3Diffusion Model Alignment Diffusion Model Distillation
Tencent Hunyuan · Hong Kong University of Science and Technology · Westlake University
AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis
May 25, 2026Shipeng Cao, Biao Qian, Haipeng Liu +2Diffusion Model Alignment Cross-Attention
Key Laboratory of Knowledge Engineering With Big Data, Ministry of Education, Hefei University of Technology, China · Department of Automation, Tsinghua University, China
Towards Anatomically Plausible Human Image Generation via Synthetic Localized Preferences
May 25, 2026Bao Li, Yuliang Xiu, Zhen LiuDiffusion Model Alignment Direct Preference Optimization
Westlake University · The Chinese University of Hong Kong, Shenzhen
DRM: Diffusion-based Reward Model With Step-wise Guidance
May 25, 2026Jaxon Zhang, Binxin Yang, Hubery Yin +2Diffusion Model Alignment Diffusion Model Guidance
Peking University · Work done during an internship at WeChat Vision, Tencent Inc. · WeChat Vision, Tencent Inc.
Inference-Time Alignment of Diffusion Models via Trust-Region Iterative Twisted Sequential Monte Carlo
May 24, 2026Weixin Wang, Yu Yang, Wei Deng +1Diffusion Model Alignment Inference-Time Optimization
Duke University · Morgan Stanley
Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models
May 22, 2026Jade Zou, Tao Huang, Weijie Kong +7Diffusion Model Alignment Flow Matching
Peking University · Tencent Hunyuan
Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators
May 21, 2026Zachary Novack, Stephen Brade, Haven Kim +8Diffusion Model Alignment Controllable Music Generation
UC San Diego · MIT · Adobe
Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models
May 20, 2026Kesong Li, Yixuan Xu, Kuo-kun Tseng +3Diffusion Model Alignment Flow Matching
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China · Alibaba Group, Hangzhou, Zhejiang, China
Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis
May 20, 2026Jinjin Zhang, Xiefan Guo, Di HuangDiffusion Model Alignment Latent Diffusion Models
Beihang University
Pareto-Enhanced Portrait Generation: Vision-Aligned Text Supervision for Alignment, Realism, and Aesthetics
May 20, 2026Yunlong Wang, Jinjin Shi, Wenbin Gao +3Diffusion Model Alignment Diffusion Transformer
Xiaomi Corporation Nanjing, China · Xiaomi Corporation Beijing, China
When Preference Labels Fall Short: Aligning Diffusion Models from Real Data
May 19, 2026Weiyan Chen, Weijian Deng, Yao Xiao +5Diffusion Model Alignment Preference Alignment
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Australian National University, Canberra, Australia +2
Stitched Value Model for Diffusion Alignment
May 19, 2026Hyojun Go, Hyungjin Chung, Prune Truong +8Diffusion Model Alignment Diffusion Model Guidance
1ETH Zurich · 2Google · University of Copenhagen
Boosting Text-to-Image Diffusion Models via Core Token Attention-Based Seed Selection
May 19, 2026Yunzhe Zhang, Hongfu Liu, Pengyu HongDiffusion Model Alignment T2I Generation
Brandeis University
Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL
May 18, 2026Junyi Wu, Weijian Luo, Haoyang Zheng +2Diffusion Model Alignment One-Step Generative Modeling
Purdue University · hi-lab, Xiaohongshu Inc.
SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training
May 18, 2026Komal Kumar, Ankan Deria, Abhishek Basu +3Diffusion Model Alignment Diffusion Model Fine-Tuning
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE · Michigan State University (MSU), USA
TILT: Model-Intrinsic Reward Alignment For Compositional Diffusion
May 16, 2026Debottam Dutta, Jianchong Chen, Jaehoon Hahm +1Diffusion Model Alignment Diffusion Model Guidance
Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA · Zhejiang University, China · Physics, University of Illinois Urbana-Champaign, Urbana, IL, USA
Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers
May 16, 2026Shaodong Xu, Zhendong Wang, Litong Gong +4Diffusion Model Alignment Diffusion Transformer
University of Science and Technology of China
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
May 15, 2026Xiaoxuan He, Siming Fu, Zeyue Xue +9Diffusion Model Alignment Video Diffusion Models
Zhejiang University · Joy Future Academy · Independent Researcher +1
Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models?
May 15, 2026Renye Yan, Jikang Cheng, Shikun Sun +7Diffusion Model Alignment Diffusion Sampling
1Peking University · 2Tsinghua University · 3Nanjing University +1
LPH-VTON: Resolving the Structure-Texture Dilemma of Virtual Try-On via Latent Process Handover
May 14, 2026Yixin Liu, Baihong Qian, Jinglin Jiang +6Diffusion Model Alignment Virtual Try-On
Shanghai Jiao Tong University
Pareto-Guided Optimal Transport for Multi-Reward Alignment
May 13, 2026Ying Ba, Tianyu Zhang, Mohan Zhou +5Diffusion Model Alignment Reward Hacking
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE 4Tomorrow Advancing Life +2
CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis
May 12, 2026Yunsung Chung, Alex El Darzi, Carlo El Khoury +3Diffusion Model Alignment Healthcare
Department of Computer Science, Tulane University, New Orleans, LA 70118 · School of Medicine, Tulane University, New Orleans, LA 70112
Efficient Adjoint Matching for Fine-tuning Diffusion Models
May 12, 2026Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu +5Diffusion Model Alignment Stochastic Optimal Control
Seoul National University · Georgia Institute of Technology · Sungkyunkwan University
The tractability landscape of diffusion alignment: regularization, rewards, and computational primitives
May 12, 2026Ankur Moitra, Andrej Risteski, Dhruv RohatgiDiffusion Model Alignment KL-Regularized RL
Gradient-Free Noise Optimization for Reward Alignment in Generative Models
May 12, 2026Jeongsol Kim, Hongeun Kim, Jian Wang +1Diffusion Model Alignment Stochastic Optimal Control
KAIST AI1 · Snap Inc.2
Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models
May 11, 2026Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken +2Diffusion Model Alignment Flow Matching
1TU Munich, MCML · 2MIT CSAIL · 3King’s College London +2
Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study
May 10, 2026Yuhan Wang, Zihan Li, Han Liu +7Diffusion Model Alignment Conditional Diffusion Models
1UC Santa Cruz · 2Siemens Healthineers · University of Washington
TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment
May 9, 2026Jiaming Li, Chenyu Zhu, Nanxi Yi +9Diffusion Model Alignment Reward Hacking
1MAIR Lab, Huazhong University of Science and Technology · 2Kuaishou Technology · 3Nanyang Technological University +2

May 26, 2026, cs.LGJ/K move · Enter open · S save

Explicit Critic Guidance for Aligning Diffusion Models

Zhengyang Liang, Qihang Zhang, Ceyuan Yang

University of Toronto · Vector Institute · The Chinese University of Hong Kong

Diffusion Model Alignment Diffusion Model Guidance RL for Diffusion Models

Open paper AskPDF arXiv

Online reinforcement learning is becoming increasingly important for aligning diffusion models with non-differentiable objectives. However, existing methods still face limitations in assigning fine-grained credit along denoising trajectories and in realizing stable value-based optimization. We propose a state-aligned latent actor-critic framework for diffusion post-training, in which the diffusion model serves as its own timestep-conditioned value function and predicts values directly on noisy latent states. This enables trajectory-level PPO training, supports stable actor-critic optimization with simple conditioning and value pretraining strategies, and naturally allows the learned critic to be reused for inference-time steering. We further extend the framework to multi-reward optimization, where joint training with complementary rewards helps alleviate reward hacking. Across both UNet- and DiT-based backbones, our method consistently outperforms prior group-relative RL and actor-critic baselines on single-reward and multi-reward benchmarks, while test-time steering provides additional gains in generation quality.

Momentum

Related topics

Latest papers 131