Diffusion Model Distillation

Overview

Diffusion model distillation studies how to make models that generate images, video, audio, text, and other data through repeated denoising faster or smaller while retaining their capabilities. Common approaches train a student to match a teacher’s denoising trajectory or output distribution, progressively reduce sampling steps, or use consistency and flow-matching objectives; compression and guidance distillation can further reduce cost. Newer work also uses on-policy and self-distillation, where the student’s own generation paths guide training, and reward-based objectives to improve preference alignment. A central tension is reducing inference cost without losing diversity, fine detail, temporal consistency, or reliability, especially in very few-step and long-horizon generation.

Latest papers

CardsList

212 papers31 papers in the last four weeks, down 24% on the four weeks before. 0.2% of all new papers.

  1. Beyond Alignment: Supervising the Generator for EEG-to-Image Reconstruction

    Nov 14, 2026Sangmin Jo, Wootaek Jeong, Heung-Il SukEEG-to-Image ReconstructionEEG-Based Visual Decoding

  2. Restoring Diversity in Distilled Image and Video Generators by Probing Activations

    Nov 13, 2026Furkan Horoz, Ozgur Kara, Kaan Oktay +4Diffusion Model DistillationDiffusion-Based Image Generation

  3. D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms

    Nov 13, 2026Pallabjyoti Deka, Lewis Wilson, Stephen McLaughlin +2Image Super-ResolutionDiffusion Model Distillation

  4. eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

    Nov 12, 2026Tong Zhao, Xianfang Zeng, Yucheng Han +4Diffusion Model DistillationFew-Step Diffusion Sampling

  5. Disagreement-Aware Distribution Matching Distillation for Few-Step Diffusion Models

    Nov 9, 2026Zikai Zhou, Zhengpeng Xie, Shitong Shao +4Diffusion Model DistillationDistribution Matching Distillation

  6. FisherDistill: Diffusion Distribution Distillation via Closed-Form Fisher Critics

    Nov 5, 2026Mahesh Bhosale, Yuanhao Zhai, Siwei Lyu +3Diffusion Model DistillationDiffusion-Based Image Generation

  7. VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

    Nov 4, 2026Yihong Luo, Tianyu Pang, Tianyang HuDiffusion Model DistillationFew-Step Diffusion Sampling

  8. Decompose the Distillation: Interpretable Single-Pass Guidance for Diffusion Models

    Nov 3, 2026Wuyang Zhang, Xiaolin XuDiffusion GuidanceClassifier-Free Guidance

  9. NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

    Nov 3, 2026Dar-Yen Chen, Yi-Zhe SongDiffusion Model DistillationMultimodal Diffusion Models

  10. From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models

    Nov 3, 2026Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar +2Unified Multimodal ModelsMultimodal Diffusion Models

  11. StepX-OPD: Rethinking Multi-Teacher On-Policy Distillation in Diffusion Models

    Nov 3, 2026Chenru Wang, Xuanyang Zhang, Yucheng Han +5Diffusion Model DistillationT2I Diffusion Models

  12. Efficient Hybrid Distillation: Synergizing Score and Adversarial Objectives for One-Step Diffusion

    Nov 2, 2026Fei Peng, Junqiang Wu, Haoxian Tan +5Diffusion Model DistillationScore Distillation Sampling

  13. History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

    Nov 2, 2026Hangyi Kuang, Chengzhi Wang, Yefei He +8Video GenerationLong-Horizon Video Generation

  14. Intrinsic Distribution Matching

    Oct 29, 2026Deyuan Liu, Yansen Han, Kaiwen Zheng +7moment matchingT2I Generation

  15. ROAD: Modern Adversarial Training for Simple One-Step Distillation at Scale

    Oct 29, 2026Xiaoou Li, Shan You, Tao HuangDiffusion Model DistillationDiffusion Distillation

  16. ALADDIN: Adversarial Latent Diffusion Distillation for One-Step Identity Swapping

    Oct 27, 2026Ivan Shchekotov, Timofei Gritsaev, Viacheslav Pirogov +1Image GenerationDiffusion Model Distillation

  17. A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation

    Oct 27, 2026Wenwu Tang, Dong Wang, Yun Cheng +1Diffusion Model DistillationDiffusion Models

  18. Beyond Distribution Matching: Self-Supervised Representation Forcing for Few-Step Video Generation

    Oct 26, 2026Junyi Chen, Haiyu Zhang, Zhoujie Fu +2Diffusion Model DistillationVideo Diffusion Models

  19. Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension

    Oct 26, 2026Vinay Saji Mathew, Soundar R. Kumara, Gretta D Kellogg +1Consistency ModelsConsistency Distillation

  20. Cumulative Marginal Matching for One-Step Diffusion

    Oct 25, 2026Paribesh Regmi, Rui LiOne-Step Generative ModelingFew-Step Diffusion Sampling

  21. Learning the Autoregressive: Direct Score-Difference Estimation with Amortized Future Credit for Video Distillation

    Oct 25, 2026Zixuan Duan, Yabo Chen, Xunzhi Xiang +5Autoregressive Video GenerationDiffusion Model Distillation

  22. PRIMFLOW: Learning Trajectory Primitives for Few-Step Distillation

    Oct 19, 2026Hongyi Fang, Benjia ZhouDiffusion Model DistillationFew-Step Diffusion Sampling

  23. TriTD: Tri-Partite Trajectory-Distribution Distillation for Real-Time Autoregressive Video Generation

    Oct 19, 2026Jia Li, Xurui Peng, Haowei Zhu +7Autoregressive Video GenerationReal-Time Interactive Video Generation

  24. TamedFlow: Trajectory-Aware Distillation for Teacher-Assisted Diffusion Sampling

    Oct 19, 2026Shuaiqi Cheng, Jiahao Zhang, Boyuan Gu +5Diffusion SamplingDiffusion Model Inference Acceleration