Consistency Distillation

Consistency distillation studies how to compress multi-step generative models into students that generate in one or a few steps, making diffusion and flow-based systems faster to use. Common approaches enforce agreement between points along a teacher’s probability-flow or rectified-flow trajectory, often alongside flow-matching or distribution-matching objectives; related methods adapt these ideas to video, audio, actions, and masked language models. Researchers also use trajectory selection, uncertainty-aware losses, and modality-specific noise parametrizations to improve training and preserve detail. The central tension is reducing inference steps without accumulating errors or losing quality, especially when the teacher’s supervision is unreliable or the student must handle complex, multimodal outputs.