Preference Optimization for Diffusion Models

Latest papers

17 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization
Oct 14, 2026Ziyi Fang, Jingzhi Li, Xiaojun Jia +3Preference Optimization for Diffusion Models Safe T2I Generation
Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of Science and Technology Beijing +3
Emo-BPO: Emotion Bidirectional Preference Optimization for Diffusion-based Emotional TTS
Oct 9, 2026Jiacheng Shi, Hongfei Du, Xinyuan Song +3Preference Optimization for Diffusion Models Emotional Speech Synthesis
College of William & Mary, USA · Emory University, USA · George Mason University, USA
Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models
Oct 7, 2026Gaurav Patel, Jun Fang, Greg Ver Steeg +2Preference Optimization for Diffusion Models Diffusion Model Unlearning
Purdue University · Amazon AGI
Geometry-Aware Preference Optimization for Text-to-Image Diffusion Models
Oct 4, 2026Lei Wang, Zhen Wang, Yuexiang Xie +1Preference Optimization for Diffusion Models Diffusion Model Alignment
Sun Yat-Sen University, Guangzhou, China · Alibaba Group, Hangzhou, China
Optimizers for Diffusion Models: A Controlled Benchmark
Sep 19, 2026Arman Bolatov, Egor Shulgin, David Li +6Preference Optimization for Diffusion Models Diffusion Models
CISPA Helmholtz Center for Information Security · KAUST · MBZUAI +1
ToPO: Token-Conditioned Preference Routing for Attention-Based Latent Diffusion Models
Sep 3, 2026Juntao Xu, Shihong Li, Hoi Fan Au +1Diffusion Model Alignment Latent Diffusion Models
Tsinghua University · University of Electronic Science and Technology of China · Stanford University
Step Back to Move Forward: Reflection-Aware Preference Optimization for Visual Generation
Sep 3, 2026Junlong Wu, Jiuzhou Lin, Jia Sun +7Preference Optimization for Diffusion Models Diffusion Model Alignment
Tsinghua University Beijing, China · Kuaishou Technology Beijing, China
Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models
Aug 7, 2026Renye Yan, Jikang Cheng, You Wu +4Diffusion Model Alignment Temporal Credit Assignment
Peking University Beijing, China · Stanford University Stanford, California, USA
Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
May 26, 2026Austin Wang, Jiaqi Han, Stefano Ermon +1Diffusion Model Alignment Preference Optimization
Caltech · Stanford University
Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models
May 20, 2026Kesong Li, Yixuan Xu, Kuo-kun Tseng +3Diffusion Model Alignment Flow Matching
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China · Alibaba Group, Hangzhou, Zhejiang, China
Efficient Adjoint Matching for Fine-tuning Diffusion Models
May 12, 2026Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu +5Diffusion Model Fine-Tuning RL for Diffusion Models
Seoul National University · Georgia Institute of Technology · Sungkyunkwan University
Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs
May 10, 2026Yunhong Lu, Qichao Wang, Hengyuan Cao +2T2I Generation Rectified Flow
Zhejiang University · Shanghai Institute for Advanced Study-Zhejiang University · Shanghai Institute for Mathematics and Interdisciplinary Sciences
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models
May 7, 2026Zhikai Li, Yue Zhao, Edward Zhongwei Zhang +4Diffusion Model Alignment Preference Optimization
Institute of Automation, Chinese Academy of Sciences · University of California, Berkeley · University of California, Santa Barbara
Towards General Preference Alignment: Diffusion Models at Nash Equilibrium
May 6, 2026Jiaming Hu, Jiamu Bai, Haoyu Wang +2Diffusion Model Alignment Preference Optimization
Boston University · Penn State University · University at Albany
CRAFT: Aligning Diffusion Models with Fine-Tuning Is Easier Than You Think
Mar 19, 2026Zening Sun, Zhengpeng Xie, Lichen Bai +3Diffusion Model Alignment Diffusion Model Fine-Tuning
HKUST (GZ) · HIT (SZ)
Rethinking Preference Alignment for Diffusion Models with Classifier-Free Guidance
Feb 21, 2026Zhou Jiang, Yandong Wen, Zhen LiuDiffusion Model Alignment Preference Alignment
Westlake University · The Chinese University of Hong Kong, Shenzhen
Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
Dec 29, 2025Dohyun Kim, Seungwoo Lyu, Seung Wook Kim +1Diffusion Model Alignment Diffusion Model Fine-Tuning
Dept. of CSE, Korea University · 2NVIDIA

Oct 14, 2026J/K move · Enter open · S save

SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization

Ziyi Fang, Jingzhi Li, Xiaojun Jia, Xingxing Wei+2

Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of Science and Technology Beijing+3

Preference Optimization for Diffusion Models Safe T2I Generation Diffusion Model Alignment T2I Diffusion Models

Open paper AskPDF arXiv

Safety alignment of text-to-image diffusion models aims to suppress harmful content without compromising useful generation capabilities. However, preferences over completed images compress multidimensional generation factors into a coarse signal, potentially driving changes beyond those required for safety. To address this, we propose Safety Transition Preference Optimization (SaTPO), which enforces step-wise alignment during denoising. SaTPO leverages the insight that an unsafe prompt and its safe rewrite naturally form a tightly coupled preference pair when conditioned on a shared parent state and noise realization. By anchoring optimization on identical parent states, SaTPO eliminates historical trajectory divergence, confining safety adaptation to an incremental alignment while preserving task-relevant semantics. Multi-scale drift constraints and benign replay further limit behavioral deviation and preserve ordinary generation. We additionally construct CoProPV, a refined dataset ensuring visually verified contrasts. Experiments on Stable Diffusion v1.5 and SDXL demonstrate improved safety-quality trade-offs. Compared with AlignGuard, PerspectiveVision unsafe-detection rates on Inappropriate Image Prompts decrease from 12.70% to 6.00% and from 15.04% to 9.30%, respectively, while FID improves from 32.28 to 25.95 and from 36.50 to 27.51. Warning: this paper includes potentially offensive content.