Diffusion Distillation

Diffusion distillation trains a smaller or faster student to reproduce a multi-step diffusion teacher, often reducing generation to a handful of steps or even one. Common approaches include trajectory and consistency matching, distribution matching, on-policy velocity regression, and progressive distillation; some use reinforcement learning or specialize models for different denoising phases. These methods are applied to image and video generation as well as audio, molecules, and robot actions, with evaluation spanning image benchmarks such as ImageNet and COCO and domain-specific measures of temporal consistency or physical validity. The central challenge is balancing speed and quality with diversity and reliable conditioning, since gains in single-sample fidelity can come at the cost of broader distribution coverage or stability over long sequences.