Diffusion Model Alignment

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Flow-OPD: On-Policy Distillation for Flow Matching Models
May 8, 2026Zhen Fang, Wenxuan Huang, Yu Zeng +8Diffusion Model Alignment Flow Matching
University of Science and Technology of China · University of California, Los Angeles · 3The Chinese University of Hong Kong +1
SARA: Semantically Adaptive Relational Alignment for Video Diffusion Models
May 8, 2026Jiesong Lian, Zixiang Zhou, Ruizhe Zhong +6Diffusion Model Alignment Video Diffusion Models
1Huazhong University of Science and Technology · 2Tencent Hunyuan · 3Shanghai Jiao Tong University +1
Diffusion-APO: Trajectory-Aware Direct Preference Alignment for Video Diffusion Transformers
May 8, 2026Jingyuan Zhu, Biaolong Chen, Le Zhang +3Diffusion Model Alignment Video Diffusion Models
Alibaba Group
GPO-V: Jailbreak Diffusion Vision Language Model by Global Probability Optimization
May 8, 2026Yu Pan, Andi Zhang, Yi Wang +2Diffusion Model Alignment Adversarial Attacks on VLMs
ShanghaiTech University · University of Warwick · SUN YAT-SEN UNIVERSITY
MARBLE: Multi-Aspect Reward Balance for Diffusion RL
May 7, 2026Canyu Zhao, Hao Chen, Yunze Tong +3Multi-Objective Reinforcement Learning Diffusion Model Alignment
Zhejiang University · Work done during an internship at HiThink. · HiThink +1
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models
May 7, 2026Zhikai Li, Yue Zhao, Edward Zhongwei Zhang +4Diffusion Model Alignment Preference Optimization
Institute of Automation, Chinese Academy of Sciences · University of California, Berkeley · University of California, Santa Barbara
Fusion in Your Way: Aligning Image Fusion with Heterogeneous Demands via Direct Preference Optimization
May 7, 2026Weijian Su, Songqian Zhang, Yuqi Han +3Diffusion Model Alignment Multi-Source Image Fusion
School of Computer Science and Technology, Dalian University of Technology · Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education · Institute of Image Processing and Understanding, North Minzu University
Threshold-Guided Optimization for Visual Generative Models
May 6, 2026Jinbin Bai, Yu Lei, Qingyu Shi +6Diffusion Model Alignment Preference Alignment
National University of Singapore · Collov Labs · Peking University +2
Towards General Preference Alignment: Diffusion Models at Nash Equilibrium
May 6, 2026Jiaming Hu, Jiamu Bai, Haoyu Wang +2Diffusion Model Alignment Preference Optimization
Boston University · Penn State University · University at Albany
AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers
May 5, 2026Ruibin Min, Yexin Liu, Aimin Pan +5Diffusion Model Alignment Diffusion Transformer
Sun Yat-sen University · The Hong Kong University of Science and Technology · VNET Group +2
A Systematic Post-Train Framework for Video Generation
Apr 28, 2026Zeyue Xue, Siming Fu, Jie Huang +9Diffusion Model Alignment Video Diffusion Models
The University of Hong Kong · JD Explore Academy · Tsinghua University +2
ViPO: Visual Preference Optimization at Scale
Apr 27, 2026Ming Li, Jie Wu, Justin Cui +3Diffusion Model Alignment Direct Preference Optimization
University of Central Florida · 2ByteDance Seed · ‡ This work was done during the first author’s internship at USA +1
Learning from Noisy Preferences: A Semi-Supervised Learning Approach to Direct Preference Optimization
Apr 27, 2026Xinxin Liu, Ming Li, Zonglin Lyu +2Pairwise Preference Learning Diffusion Model Alignment
University of Central Florida
Oracle Noise: Faster Semantic Spherical Alignment for Interpretable Latent Optimization
Apr 26, 2026Haosen Li, Wenshuo Chen, Lei Wang +3Diffusion Model Alignment Text-Image Alignment
1The Hong Kong University of Science and Technology (Guangzhou) · 2Griffith University & Data61/CSIRO
$Z^2$ -Sampling: Zero-Cost Zigzag Trajectories for Semantic Alignment in Diffusion Models
Apr 26, 2026Haosen Li, Wenshuo Chen, Shaofeng Liang +3Diffusion Model Alignment Diffusion Sampling
1The Hong Kong University of Science and Technology (Guangzhou) · 2Griffith University & Data61/CSIRO
V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think
Apr 25, 2026Bingda Tang, Yuhui Zhang, Xiaohan Wang +3Diffusion Model Alignment Group Relative Policy Optimization
Stanford University · Tsinghua University · Amazon FAR +1
HP-Edit: A Human-Preference Post-Training Framework for Image Editing
Apr 21, 2026Fan Li, Chonghuinan Wang, Lina Lei +9Diffusion Model Alignment Image Editing Evaluation
Huawei Noah’s Ark Lab · Nankai University · Harbin Institute of Technology
Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models
Apr 19, 2026Jeongjae Lee, Jinho Chang, Jeongsol Kim +1Diffusion Model Alignment Diffusion Model Fine-Tuning
Graduate School of AI KAIST, Korea
Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models
Apr 17, 2026Jiaoyang Ruan, Xin Gao, Yinda Chen +5Diffusion Model Alignment Diffusion Language Model Inference
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · University of Science and Technology of China, Hefei, China +1
LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories
Apr 16, 2026Zhanhao Liang, Tao Yang, Jie Wu +2Diffusion Model Alignment Flow Matching
1The Australian National University · 2ByteDance Seed
Reward-Aware Trajectory Shaping for Few-step Visual Generation
Apr 16, 2026Rui Li, Bingyu Li, Yuanzhi Liang +3Diffusion Model Alignment Diffusion Model Distillation
University of Science and Technology of China HeFei, China · TeleAI ShangHai, China
RefAlign: Representation Alignment for Reference-to-Video Generation
Mar 26, 2026Lei Wang, YuXin Song, Ge Wu +5Diffusion Model Alignment Video Diffusion Models
1PCA Lab, VCIP, College of Computer Science, Nankai University, China · Baidu Inc., China · College of Artificial Intelligence, Jilin University, China +1
SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning
Mar 18, 2026Xi Ye, Wenjia Yang, Yangyang Xu +4Diffusion Model Alignment Video Diffusion Models
Tsinghua University · Ant Group · University of Chinese Academy of Sciences
V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising
Mar 17, 2026Han Lin, Xichen Pan, Zun Wang +4Diffusion Model Alignment Pixel-Space Diffusion Models
1UNC Chapel Hill · 2NYU · 3Meta +1
VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment
Mar 17, 2026Tengjiao Yin, Jinglei Shi, Heng Guo +1Diffusion Model Alignment Multi-View Consistency
VCIP & TMCC & DISSec, College of Computer Science, Nankai University · Beijing University of Posts and Telecommunications · LIX, École Polytechnique, IP Paris
Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models
Mar 15, 2026Niklas Schweiger, Daniel Cremers, Karnik RamDiffusion Model Alignment Black-Box Optimization
Technical University of Munich, Germany · Munich Center for Machine Learning, Germany
Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration
Jan 27, 2026Zhengjian Yao, Jiakui Hu, Kaiwen Li +5Diffusion Model Alignment Diffusion Models for Image Restoration
Biomedical Engineering Department, College of Future Technology, Peking University, Beijing, China
Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
Dec 29, 2025Dohyun Kim, Seungwoo Lyu, Seung Wook Kim +1Diffusion Model Alignment Diffusion Model Fine-Tuning
Dept. of CSE, Korea University · 2NVIDIA
Multi-dimensional Preference Alignment by Conditioning Reward Itself
Dec 11, 2025Jiho Jang, Jinyoung Kim, Kyungjune Baek +1Diffusion Model Alignment Diffusion Model Guidance
Seoul National University · Sejong University
Early Estimation of Language to Latent Alignment in Diffusion Models
Dec 9, 2025Vasco Ramos, Regev Cohen, Idan Szpektor +1Diffusion Model Alignment Vision-Language Alignment
NOVALINCS, NOVA University of Lisbon, Portugal · Google Research

May 8, 2026, cs.CVJ/K move · Enter open · S save

Flow-OPD: On-Policy Distillation for Flow Matching Models

Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao+7

University of Science and Technology of China · University of California, Los Angeles · 3The Chinese University of Hong Kong+1

Diffusion Model Alignment Flow Matching RL for Image Generation Flow-Based Generative Modeling On-Policy Distillation T2I Diffusion Models

Open paper AskPDF arXiv

Existing Flow Matching (FM) text-to-image models suffer from two critical bottlenecks under multi-task alignment: the reward sparsity induced by scalar-valued rewards, and the gradient interference arising from jointly optimizing heterogeneous objectives, which together give rise to a 'seesaw effect' of competing metrics and pervasive reward hacking. Inspired by the success of On-Policy Distillation (OPD) in the large language model community, we propose Flow-OPD, the first unified post-training framework that integrates on-policy distillation into Flow Matching models. Flow-OPD adopts a two-stage alignment strategy: it first cultivates domain-specialized teacher models via single-reward GRPO fine-tuning, allowing each expert to reach its performance ceiling in isolation; it then establishes a robust initial policy through a Flow-based Cold-Start scheme and seamlessly consolidates heterogeneous expertise into a single student via a three-step orchestration of on-policy sampling, task-routing labeling, and dense trajectory-level supervision. We further introduce Manifold Anchor Regularization (MAR), which leverages a task-agnostic teacher to provide full-data supervision that anchors generation to a high-quality manifold, effectively mitigating the aesthetic degradation commonly observed in purely RL-driven alignment. Built upon Stable Diffusion 3.5 Medium, Flow-OPD raises the GenEval score from 63 to 92 and the OCR accuracy from 59 to 94, yielding an overall improvement of roughly 10 points over vanilla GRPO, while preserving image fidelity and human-preference alignment and exhibiting an emergent 'teacher-surpassing' effect. These results establish Flow-OPD as a scalable alignment paradigm for building generalist text-to-image models. The codes and weights will be released in: https://github.com/CostaliyA/Flow-OPD .

Momentum

Related topics

Latest papers 131