Diffusion Model Distillation

Diffusion model distillation studies how to make models that generate images, video, audio, text, and other data through repeated denoising faster or smaller while retaining their capabilities. Common approaches train a student to match a teacher’s denoising trajectory or output distribution, progressively reduce sampling steps, or use consistency and flow-matching objectives; compression and guidance distillation can further reduce cost. Newer work also uses on-policy and self-distillation, where the student’s own generation paths guide training, and reward-based objectives to improve preference alignment. A central tension is reducing inference cost without losing diversity, fine detail, temporal consistency, or reliability, especially in very few-step and long-horizon generation.