Diffusion Model Distillation

Overview

Diffusion model distillation studies how to make models that generate images, video, audio, text, and other data through repeated denoising faster or smaller while retaining their capabilities. Common approaches train a student to match a teacher’s denoising trajectory or output distribution, progressively reduce sampling steps, or use consistency and flow-matching objectives; compression and guidance distillation can further reduce cost. Newer work also uses on-policy and self-distillation, where the student’s own generation paths guide training, and reward-based objectives to improve preference alignment. A central tension is reducing inference cost without losing diversity, fine detail, temporal consistency, or reliability, especially in very few-step and long-horizon generation.

Latest papers

CardsList

213 papers31 papers in the last four weeks, down 24% on the four weeks before. 0.2% of all new papers.

  1. DLM-One: Diffusion Language Models for One-Step Sequence Generation

    May 30, 2025Tianqi Chen, Shujian Zhang, Mingyuan ZhouDiffusion Language ModelsDiffusion Model Distillation

  2. Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

    May 28, 2025Yi Zhang, Peng Wang, Difan ZouPhysics-Informed Diffusion ModelsDiffusion Model Distillation

  3. DICE: Distilling Classifier-Free Guidance into Text Embeddings

    Feb 6, 2025Zhenyu Zhou, Defang Chen, Can Wang +2Diffusion Model AlignmentDiffusion Model Guidance