cs.LGJan 30, 2026

Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation

Authors: Youngjoong Kim, Duhoe Kim, Woosung Kim, Jaesik Park

Organizations: Department of Computer Science and Engineering, Seoul National University · Interdisciplinary Program in Artificial Intelligence, Seoul National University

Abstract

Consistency models have been proposed for fast generative modeling, achieving results competitive with diffusion and flow models. However, these methods exhibit instability and limited reproducibility when training from scratch, motivating subsequent work to explain and stabilize these issues. While these efforts have provided valuable insights, the explanations remain fragmented, and the theoretical relationships remain unclear. In this work, we provide a theoretical examination of consistency models by analyzing them from a flow map-based perspective. This joint analysis clarifies how training and convergence behavior can give rise to degenerate solutions. Building on these insights, we revisit self-distillation as a practical remedy for certain forms of suboptimal convergence. To avoid excessive gradient norms during its optimization, we reformulate its objective empirically, yielding improved Self-Distillation (iSD). We further extend iSD with training-time classifier-free guidance, yielding iSD-T, which improves performance and reproducibility in the studied settings. We demonstrate that iSD extends beyond image generation to diffusion-based policy learning, without reliance on pretrained diffusion models for initialization.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Dual-End Consistency Model

    Feb 11, 2026Linwei Dong, Ruoyu Guo, Ge Bai +3One-Step Generative ModelingConsistency Distillation

  2. Consistent Distribution Matching for Data-Free Diffusion Distillation

    Oct 6, 2026Yuxiang Fu, Qi Yan, Zike Wu +4Diffusion Model DistillationDiffusion Model Inference Acceleration

  3. Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension

    Oct 26, 2026Vinay Saji Mathew, Soundar R. Kumara, Gretta D Kellogg +1Consistency ModelsConsistency Distillation