cs.CVNov 9, 2026

Direct Trajectory Optimization for Inference-Time Alignment of Text-to-Image Diffusion Models

Authors: Huixuan Zhang, Li Lin, Junzhe Zhang, Xiaojun Wan

Organizations: Wangxuan Institute of Computer Technology, Peking University

Abstract

Alignment of text-to-image diffusion models with reward functions is a key challenge in controllable generation. Existing approaches typically rely on optimizing model inputs or fine-tuning denoising networks, which requires computationally expensive backpropagation through large denoising models. In this work, we propose Direct Trajectory Optimization (DTO), a training-free framework that performs alignment by directly optimizing the sampling trajectory of diffusion models. Specifically, DTO treats intermediate denoising scores and sampled noise as the optimization variables, eliminating the need for gradients through the denoising network. To mitigate the out-of-distribution problem, we introduce Reward-Optimized Score Guidance, which interpolates between the optimized scores and the pretrained model predictions to stabilize sampling, while providing a provable theoretical guarantee of reward improvement under constrained distributions. Empirically, DTO improves alignment performance across three diffusion models on two benchmarks, outperforming existing inference-time alignment methods while maintaining high efficiency. These results suggest that trajectory-level optimization is a promising direction for efficient and effective diffusion model alignment.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models

    May 28, 2026Jaa-Yeon Lee, Yeobin Hong, Taesung Kwon +1Diffusion Model AlignmentText-Image Alignment

  2. Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation

    Nov 25, 2025Taehoon Kim, Henry Gouk, Timothy HospedalesDiffusion Model AlignmentReward Hacking