VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation
Organizations: Hong Kong University of Science and Technology · The Chinese University of Hong Kong, Shenzhen
Abstract
Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student's changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student's sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.
Figures & tables
| Model | NFE | Verifiable | ImgQ | Preference | Div. | ||||
|---|---|---|---|---|---|---|---|---|---|
| GenEval | OCR | Aest. | ImgRwd | PickScore | HPSv2 | HPSv3 | LPIPS | ||
| SD3.5-Medium (teacher, CFG=4.5) | 50 2 | 0.65 | 0.58 | 5.37 | 0.86 | 0.8628 | 0.2805 | 9.78 | 0.61 |
| SD3.5-Medium (teacher, CFG=7) | 50 2 | 0.67 | 0.62 | 5.37 | 0.96 | 0.8640 | 0.2854 | 9.87 | 0.60 |
| DMD2 | 4 | 0.60 | 0.53 | 5.55 | 1.00 | 0.8648 | 0.2977 | 11.18 | 0.53 |
| D-DMD | 4 | 0.55 | 0.56 | 5.51 | 0.99 | 0.8567 | 0.2897 | 10.93 | 0.57 |
| DPDMD | 4 | 0.62 | 0.33 | 5.28 | 0.80 | 0.8530 | 0.2707 | 8.13 | 0.59 |
| Configuration | Verifiable | ImgQ | Preference | Div. | ||||
|---|---|---|---|---|---|---|---|---|
| GenEval | OCR | Aest. | ImgRwd | PickScore | HPSv2 | HPSv3 | LPIPS | |
| (A-form only) | 0.64 | 0.55 | 5.53 | 1.00 | 0.8671 | 0.2949 | 11.20 | 0.58 |
| (B-form only) | 0.61 | 0.57 | 5.57 | 0.99 | 0.8665 | 0.2987 | 11.47 | 0.59 |
| w/o transition-aware field | 0.58 | 0.53 | 5.56 | 0.99 | 0.8655 | 0.2964 | 11.13 | 0.56 |
| w/o shared backbone | 0.62 | 0.56 | 5.55 | 0.99 | 0.8669 | 0.2971 | 11.33 | 0.57 |
| 0.63 | 0.58 | 5.56 | 1.01 | 0.8674 | 0.2983 | 11.47 | 0.58 | |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Diffusion Model Distillation
Diffusion model distillation transfers a large model’s generative behavior to faster or smaller models that need fewer sampling steps.
See all 213 papers