cs.CVNov 8, 2026

Trajectory-Regularized Distribution Distillation for One-Step Generation

Authors: An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun

Organizations: ByteDance Seed · Zhejiang University

Abstract

Distilling flow matching models for one-step generation generally follows two paradigms. Trajectory alignment supervises the student with the teacher's probability flow ODE, but its performance degrades under very small sampling budgets. Distribution matching directly optimizes the generated distribution, but estimating its reverse Kullback-Leibler gradient typically requires a separate network to track the evolving student score. We propose trajectory-regularized distribution distillation (TRD), which combines distribution-level and trajectory-level supervision through a momentum-mixture policy predicted in a single student forward pass. This policy serves simultaneously as a one-step generator, an estimator of the student's marginal velocity that removes the need for a second trainable network in distribution matching, and a closed-form time-dependent velocity field supervised along the teacher ODE trajectory. Experiments on SANA-1.6B and Stable Diffusion 3.5 show that TRD outperforms representative consistency and distribution matching baselines in generation quality and training efficiency for both one-step and few-step generation, with a trade-off in sample diversity.

Explore similar work

CardsList
  1. Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment

    Jun 28, 2026Chen Wang, Peiran Yun, Pan Xie +1Diffusion Model DistillationDistribution Matching

  2. Consistent Distribution Matching for Data-Free Diffusion Distillation

    Oct 6, 2026Yuxiang Fu, Qi Yan, Zike Wu +4Diffusion Model DistillationDiffusion Model Inference Acceleration

  3. Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation

    May 8, 2026Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade +3Flow MatchingDiffusion Language Model Inference