Organizations: Laboratoire MAP5, UMR 8145, Université Paris Cité, CNRS · Heriot-Watt University, School of Mathematical and Computer Sciences & Maxwell Institute for Mathematical Sciences · École Polytechnique · Laboratoire de Recherche d’EPITA, Le Kremlin-Bicêtre, France
Consistency models (CMs) have become a leading approach for generating high-quality samples in few steps. However, adding steps can improve or degrade sample quality in ways that are highly sensitive to the schedule and that existing theory does not fully explain. To provide accuracy guarantees and guide CM sampler design, we analyze multistep CM sampling as a composition of noising and approximate denoising operators. Under explicit, verifiable stability assumptions, we derive a non-asymptotic error bound that separates contraction of the initialization error from accumulation of approximation error. The bound assigns distinct roles to the schedule: large early noise levels drive contraction, while small late noise levels control the residual bias. As a corollary, we obtain explicit constants for strongly log-concave and semi-log-concave targets. We further establish a complementary guarantee whose assumptions, one-step accuracy and stability, can be estimated for a given trained model. Experiments show that the contraction and approximation profiles entering our bounds can be reliably measured and closely match the predicted functional forms. Together, these results provide a meaningful convergence theory for multi-step CMs and a practical route to sampler design.
Figures & tables
Figure 1: Certificate over exact error at the repeated VE level t=1 , from N(0,2.52Id) , with m=0.8 and, when inexact, m=0.55 ; offsets are ωt=cBm(γt) with M0=0.03 and M1=0.05 .
Figure 2: Approximate q~t and the predicted profile qt of Corollary 3.4 . The analytical curve is used as a fitted functional form; it is not a certificate of the learned map’s global contraction factor.
Figure 3: Approximate target-only one-step error Etπ for the TrigFlow and VE LCMs, compared with the analytical VE shape of Corollary 3.4 used as an upper envelope.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Regime
Initialization
Learned step
Certificate overstates
Outcome
Mean
mean
mean
nothing
exact
Initial variance
variance
none
inherited variance, by κt
ratio ∼qt−n
Learned variance
variance or none
variance
inherited variance, by κt
no uniform Et
Both
both kinds
both kinds
the same, diluted by the mean
loose, then tight
Appendix
Table 1: Regimes of Section E.3 . The second and third columns give the kind of error introduced by the initialization and by the learned step, none meaning exact. The certificate also adds the sizes of variance errors of opposite directions, which may cancel ( Section E.2 ).
Figure 4: Error in the mean only: m=0.8 , the curvature fitted in Section 4 , m=m , ηr=M0+M1/r with M0=0.03 , M1=0.05 , and s0=m−\nicefrac12 . Left: from ζ0=2 , along tk=2(0.85)k−1 in VE coordinates and tk=1.3(0.85)k−1 in TrigFlow coordinates, the certificate ( 3 ) (markers) and the exact error (lines) coincide; stars mark the optimal step count. Right: at the repeated VE level t=1 , the error moves to the plateau Et/(1−qt) from above, at, or below it.
Figure 5: Choosing the level of a second step, in VE coordinates with m=0.8 and d=1 : a first step at t1=4 from N(0,2.52Id) , then a second at level t2 , for no error in the mean and for the offsets ηr and 4ηr , with ηr as in Fig. 4 , on a logarithmic scale. Exact error solid, certificate dashed; circles mark the best level ( t2=0 : no second step), stars the certificate’s choice and dots the exact error at that choice. Left: the learned variance is too large ( m=0.55 ), like the initialization, so no cancellation occurs. Right: it is too small ( m=1.1 ), so the learned step makes the law too narrow while the initialization is too wide, and the two variance errors cancel.
Figure 6: One-step SD 1.5 experiment. The measured empirical W2 error is compared with the plug-in recursive estimate qtD0+Etπ .
Consistency Models are generative models trained to map points along a diffusion trajectory to the same clean sample, enabling fast few-step generation.
The slow iterative sampling nature remains a major bottleneck for the practical deployment of diffusion and flow-based generative models. While consistency models (CMs) represent a state-of-the-art distillation-based approach for efficient generation, their large-scale application is still limited by two key issues: training instability and inflexible sampling. Existing methods seek to mitigate these problems through architectural adjustments or regularized objectives, yet overlook the critical reliance on trajectory selection. In this work, we first conduct an analysis on these two limitations: training instability originates from loss divergence induced by unstable self-supervised term, whereas sampling inflexibility arises from error accumulation. Based on these insights and analysis, we propose the Dual-End Consistency Model (DE-CM) that selects vital sub-trajectory clusters to achieve stable and effective training. DE-CM decomposes the PF-ODE trajectory and selects three critical sub-trajectories as optimization targets. Specifically, our approach leverages continuous-time CMs objectives to achieve few-step distillation and utilizes flow matching as a boundary regularizer to stabilize the training process. Furthermore, we propose a novel noise-to-noisy (N2N) mapping that can map noise to any point, thereby alleviating the error accumulation in the first step. Extensive experimental results show the effectiveness of our method: it achieves a state-of-the-art FID score of 1.70 in one-step generation on the ImageNet 256x256 dataset, outperforming existing CM-based one-step approaches.
Linwei Dong, Ruoyu Guo, Ge Bai +3
1Zhejiang University · 2Bytedance Inc. · 3Zhejiang Lab
We develop a principled framework for analyzing and designing noise schedules in diffusion models. We show that one can recast this design problem as an optimal control problem, whose state is the Fisher information of the diffusion process, which evolves according to an ODE, and whose control input is the noise schedule. The objective of the optimal control problem is a functional involving the Fisher information, which is shown to be an upper bound on the Kullback-Leibler sampling error. By solving this optimal control problem, we obtain sufficient conditions on noise schedules under which O~(d/n) sampling error is achievable, where d is the data dimension and n is the number of discretization steps. While existing theoretical works also prove that O~(d/n) sampling error bounds are achievable, these results hold for specific noise schedules that differ from those used in practice. Under additional assumptions on the data distribution, we derive a family of noise schedules that recovers the functional forms of schedules widely used in practice. We show that these schedules achieve the O~(d/n) error bound when their parameters are chosen according to our analysis. Our theory offers a principled approach to noise schedule design, yielding explicit functional forms and guidance for selecting their parameters.
Seo Taek Kong, Weina Wang, R. Srikant
ECE & CSL, University of Illinois Urbana-Champaign · Computer Science Department, Carnegie Mellon University · ECE, CSL & NCSA, University of Illinois Urbana-Champaign
Accurate sampling from continuous-time generative models can require many model evaluations. Low-step sampling reduces cost, but coarse updates introduce endpoint errors that can accumulate and degrade generation quality. We introduce TraceSampler, a post-training method that corrects these errors without modifying the pretrained model. Rather than optimising low-step updates directly, TraceSampler models endpoint error as a sample-specific prediction problem. A higher-step reference integration over the same interval provides the supervised target, while a compact representation makes high-dimensional error predictable from the sampling trace. This representation combines shared residual structure, trace-derived directions, and a low-rank remainder, so shallow per-step MLP heads predict sample-dependent coefficients. These coefficients drive a two-stage learned residual correction: the first stage predicts the full endpoint residual, while an intermediate backbone evaluation provides new trace information to predict the remaining residual; this evaluation is reused by the following step. TraceSampler is evaluated with Rectified Flow on CIFAR-10, SiT-XL/2 on ImageNet-256, and SD3.5 Medium for text-to-image generation. At 10 steps, it reduces trajectory MSE relative to matched-step Euler by 90.5% on CIFAR-10 and 85.6% on ImageNet-256, while achieving FID scores of 2.692 and 2.189, close to 120-step Heun references of 2.601 and 2.056 with 24× fewer backbone evaluations. On SD3.5 Medium, TraceSampler improves image quality and prompt following; at 10 steps, ImageReward rises from 0.752 to 0.906 while trajectory MSE falls by 70.7%. These gains preserve Euler's backbone-evaluation budget, add less than 0.4% inference FLOPs, and incur negligible wall-clock overhead, showing that coarse-step sampling error can be made predictable from the sampling trace and efficiently corrected.
George Matthew, Guoqiang Zhang, Kenta Niwa
University of Exeter, United Kingdom · NTT Communication Science Laboratories, Japan