Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension

Vinay Saji Mathew; Soundar R. Kumara; Gretta D Kellogg; William KM Lai

cs.LGOct 26, 2026

Stable Continuous-Time Consistency Distillation: An Empirical Study with a Multistep Extension

Authors: Vinay Saji Mathew, Soundar R. Kumara, Gretta D Kellogg, William KM Lai

Organizations: The Pennsylvania State University · State University of New York at Buffalo

Abstract

Continuous-time consistency models are an attractive route to few-step generation. They remove the discretization error inherent to the timestep grid that discrete-time consistency models depend on. Until recently, however, training them was unstable. Lu & Song (2025) address this with the TrigFlow formulation and a coordinated set of architectural and training-objective changes. We conduct an independent empirical study of this framework on CIFAR-10 and ImageNet-64. No open-weight TrigFlow teachers exist at these scales, so we train them from scratch and then verify the consistency distillation (sCD) and consistency training (sCT) behavior of students initialized from these teachers. On CIFAR-10, our students reproduce the reported FID at one and two steps. On ImageNet-64, they fall short, a gap that appears related to our teacher and to architectural choices the paper leaves unspecified. We also report a parameter-count gap of about 5% between the model size stated in the paper and a faithful re-implementation, which we attribute to the scale-and-shift conditioning projections in Adaptive Double Normalization. We then propose Multistep sCD (MS-sCD), a continuous-time, segment-conditioned multistep generalization of sCD. The segment-conditioning idea is borrowed from the discrete-time multistep consistency distillation of Heek et al. (2024); we build on their preliminary continuous-time experiments within the TrigFlow/sCM framework. The rest of the algorithm follows sCD. MS-sCD replaces sCM's two-step intermediate time with a fixed segment-boundary schedule and recovers sCD at M = 1. We report FID at M = 2, 4, 8 and compare against the discrete-time MSCD baseline. Code, model weights, and training scripts are available on GitHub (Mathew et al., 2026).

Figures & tables

Dataset	Method	NFE	FID
ImageNet-64	EDM2-S ( Karras et al., 2024 )	63	1.58
ImageNet-64	TrigFlow teacher ( Lu and Song, 2025 )	63	1.70
ImageNet-64	TrigFlow teacher (ours)	63	1.83
CIFAR-10	TrigFlow teacher ( Lu and Song, 2025 )	35	2.15
CIFAR-10	TrigFlow teacher (ours)	35	2.08

Table 1: TrigFlow teacher FID, 50k samples, standard ADM evaluation. ImageNet-64 is class-conditional with the EDM2-S backbone. CIFAR-10 is unconditional with the DDPM++ backbone. The TrigFlow sampler is DPM-Solver-2S ( Lu et al., 2022 ) , with Heun’s intermediate step.

Method	Backbone	NFE=1 FID	NFE=2 FID	Framework
iCT-deep ( Song and Dhariwal, 2024 )	ADM-style	3.25	2.77	discrete, EDM
ECM-S * (ECT) ( Geng et al., 2025 )	EDM2-S	4.05	2.79	discrete, EDM
MSCD † ( Heek et al., 2024 )	UViT-small	4.3	2.0	discrete
sCM sCD-S ‡ ( Lu and Song, 2025 )	EDM2-S/TrigFlow	2.97	2.07	TrigFlow
sCM sCT-S ‡ ( Lu and Song, 2025 )	EDM2-S/TrigFlow	3.23	2.93	TrigFlow
Ours sCD	EDM2-S/TrigFlow	3.59	2.66	TrigFlow

Table 2: sCD and sCT results on ImageNet-64 (EDM2-S), without classifier-free guidance (CFG).

Method	NFE=1 FID	NFE=2 FID	Framework
iCT ( Song and Dhariwal, 2024 )	2.83	2.46	discrete, EDM
ECT ( Geng et al., 2025 )	3.60	2.11	discrete, EDM
CD-LPIPS ( Song et al., 2023 )	3.55	2.93	discrete, EDM
sCM sCD ( Lu and Song, 2025 )	3.66	2.52	TrigFlow
sCM sCT ( Lu and Song, 2025 )	2.85	2.06	TrigFlow
Ours sCD	3.59	2.39	TrigFlow

Table 3: sCD and sCT results on unconditional CIFAR-10, DDPM++ backbone, 50k samples. sCM paper numbers from Lu and Song (2025) , Table 1.

Panel of Figure 1 — Figure 1: Green arrows are PF-ODE trajectories (data $\leftrightarrow$ noise) and red arrows show the consistency map. (a) For $M=1$ the segment CM equation 14 reduces to sCM: $f^{(0)}_{\theta}$ collapses every trajectory point to the data origin $(x_{0},0)$ . (b) For $M>1$ , MS-sCD splits the time axis at $0=t_{0}<t_{1}<\cdots<t_{M}=t_{\max}$ and trains one segment-conditioned map $f^{(k)}_{\theta}$ per segment $k$ . Each $f^{(k)}_{\theta}$ collapses trajectory points inside segment $k$ to the segment’s lower boundary $(x_{t_{k}},t_{k})$ .

Method	$M$	NFE	FID
sCM sCD-S ( Lu and Song, 2025 ) (appendix Table 8)	1	1	2.97
sCM sCD-S ( Lu and Song, 2025 ) (appendix, 2-step)	1	2	2.07
MSCD on EDM2-S (ours)	2	2	3.44
MSCD on EDM2-S (ours)	4	4	2.32
MSCD on EDM2-S (ours)	8	8	1.79
MS-sCD (ours)	2	2	2.53

Table 4: MS-sCD vs. baselines on ImageNet-64 (EDM2-S, class-conditional, no CFG, 50k samples). sCM-S baselines are from Lu and Song (2025) , appendix Table 8, where the main-paper sCM table reports the XL variant for ImageNet-64. The MSCD baselines on the EDM2-S backbone are our reproductions of Heek et al. (2024) ’s method on this architecture.

Method	$M$	NFE	FID
MS-sCD (ours)	2	2	2.60
MS-sCD (ours)	4	4	2.35
MS-sCD (ours)	8	8	2.20
Ours sCT	1	2	2.09
TrigFlow teacher (ours)	—	35	2.08
TrigFlow teacher (ours)	—	199	2.22

Table 5: MS-sCD on unconditional CIFAR-10 (DDPM++ backbone, 50k samples). Teacher rows use DPM-Solver-2S on our TrigFlow teacher; the 35-NFE entry matches Table 1 . Two-step sCT is from Table 3 .

Figure 2: FID vs. NFE on ImageNet-64 (EDM2-S, class-conditional, no CFG, 50k samples). Solid lines: our EDM2-S/TrigFlow reproduction. Dashed lines: sCM’s reported sCD-S and sCT-S numbers ( Lu and Song, 2025 , appendix Table 8) . The two multistep students extend past the

\text{NFE}{\leq}2

regime of the single-step CMs by varying the number of segments

M\in\{2,4,8\}

: continuous MS-sCD (orange) leads at

M{=}2

, while our discrete MSCD reproduction (purple) overtakes it by

M{=}8

, both approaching the teacher reference (horizontal lines: ours

1.83

, sCM-reported

1.70

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	NFE	FID
sCD iterated	1	3.59
sCD iterated	2	2.66
sCD iterated	4	2.48
sCD iterated	8	2.32
MS-sCD	2	2.53
MS-sCD	4	2.26

Appendix

Table 6: Iterative single-step sCD vs. MS-sCD on ImageNet-64 (EDM2-S, class-conditional, no CFG, 50k samples). The sCD results at NFE

{=}1

2

are from Table 2 ; MS-sCD results are from Table 4 .

	Teacher	sCD	sCT	MS-sCD	MSCD-EDM	MM
Framework	TrigFlow	TrigFlow	TrigFlow	TrigFlow	EDM	EDM
Model	EDM2-S	EDM2-S	EDM2-S	EDM2-S	EDM2-S	EDM-S
Batch size	2048	2048	2048	2048	2048	2048
GPUs	32	32	32	32	32	16
$P_{\text{mean}}$	$-0.8$	$-1.0$	$-1.0$	$-1.0$	$-0.8$	—
$P_{\text{std}}$	$1.6$	$1.6$	$1.6$	$1.6$	$1.6$	—

Appendix

Table 7: Hyperparameters for all experiments.

Model	NFE	Best	Mean $\pm$ std
TrigFlow teacher	63	1.83	$1.8334\pm 0.0005$
sCD	1	3.59	$3.6423\pm 0.0330$
sCD	2	2.66	$2.6856\pm 0.0153$
sCT	1	4.34	$4.4403\pm 0.0557$
sCT	2	3.98	$4.0093\pm 0.0198$
MS-sCD ( $M{=}2$ )	2	2.53	$2.5761\pm 0.0231$

Appendix

Table 8: ImageNet-64 seed FID mean

\pm

std (EDM2-S, class-conditional, no CFG, 50k samples). Best matches the main-text FID.

Model	NFE	FID	P	R	D	C
TrigFlow teacher	63	1.83	0.802	0.708	1.165	0.968
sCD	1	3.59	0.776	0.703	1.062	0.950
sCD	2	2.66	0.780	0.708	1.104	0.960
sCT	1	4.34	0.722	0.737	0.878	0.900
sCT	2	3.98	0.719	0.747	0.890	0.904
MS-sCD ( $M{=}2$ )	2	2.53	0.785	0.702	1.121	0.960

Appendix

Table 9: ImageNet-64 precision (P), recall (R), density (D), and coverage (C) on the best-seed 50k-sample sets (EDM2-S, class-conditional, no CFG), using unnormalized 2048-d Inception-2015 features and

k{=}5

( Naeem et al., 2020 ) .

Panel of Figure 3 — Figure 3: Teacher and sCD student samples.

Panel of Figure 4 — Figure 4: sCT student and 8-step MSCD samples.

Consistency Models

Consistency Models are generative models trained to map points along a diffusion trajectory to the same clean sample, enabling fast few-step generation.

See all 25 papers

Consistent Distribution Matching for Data-Free Diffusion Distillation
Oct 6, 2026Yuxiang Fu, Qi Yan, Zike Wu +4Diffusion Model Distillation Diffusion Model Inference Acceleration
University of British Columbia · Vector Institute for AI · Canada CIFAR AI Chair
Dual-End Consistency Model
Feb 11, 2026Linwei Dong, Ruoyu Guo, Ge Bai +3One-Step Generative Modeling Consistency Distillation
1Zhejiang University · 2Bytedance Inc. · 3Zhejiang Lab
Continuous-Time Distribution Matching for Few-Step Diffusion Distillation
May 7, 2026Tao Liu, Hao Yan, Mengting Chen +8Diffusion Distillation Few-Step Diffusion Sampling
1VCIP, College of Computer Science, Nankai University · 2Alibaba Group · College of Artificial Intelligence, Jilin University

Oct 6, 2026J/K move · Enter open · S save

Consistent Distribution Matching for Data-Free Diffusion Distillation

Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang+3

University of British Columbia · Vector Institute for AI · Canada CIFAR AI Chair

Diffusion Model Distillation Diffusion Model Inference Acceleration Distribution Matching Distillation Data-Free Knowledge Distillation

Open paper AskPDF arXiv

Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256

\times

256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.

Abstract

Consistency Models

Explore similar work

Consistent Distribution Matching for Data-Free Diffusion Distillation

Dual-End Consistency Model

Continuous-Time Distribution Matching for Few-Step Diffusion Distillation