Iterating Consistency Models: Stability, Error Bounds and Noise Schedules

Alessio Spagnoletti; Abdul-Lateef Haji-Ali; Andrés Almansa; Alain Oliviero Durmus; Eric Moulines; Marcelo Pereyra

stat.MLOct 2, 2026

Iterating Consistency Models: Stability, Error Bounds and Noise Schedules

Authors: Alessio Spagnoletti, Abdul-Lateef Haji-Ali, Andrés Almansa, Alain Oliviero Durmus, Eric Moulines, Marcelo Pereyra

Organizations: Laboratoire MAP5, UMR 8145, Université Paris Cité, CNRS · Heriot-Watt University, School of Mathematical and Computer Sciences & Maxwell Institute for Mathematical Sciences · École Polytechnique · Laboratoire de Recherche d’EPITA, Le Kremlin-Bicêtre, France

Abstract

Consistency models (CMs) have become a leading approach for generating high-quality samples in few steps. However, adding steps can improve or degrade sample quality in ways that are highly sensitive to the schedule and that existing theory does not fully explain. To provide accuracy guarantees and guide CM sampler design, we analyze multistep CM sampling as a composition of noising and approximate denoising operators. Under explicit, verifiable stability assumptions, we derive a non-asymptotic error bound that separates contraction of the initialization error from accumulation of approximation error. The bound assigns distinct roles to the schedule: large early noise levels drive contraction, while small late noise levels control the residual bias. As a corollary, we obtain explicit constants for strongly log-concave and semi-log-concave targets. We further establish a complementary guarantee whose assumptions, one-step accuracy and stability, can be estimated for a given trained model. Experiments show that the contraction and approximation profiles entering our bounds can be reliably measured and closely match the predicted functional forms. Together, these results provide a meaningful convergence theory for multi-step CMs and a practical route to sampler design.

Figures & tables

Panel of Figure 1 — Figure 1: Certificate over exact error at the repeated VE level $t=1$ , from $\mathcal{N}(0,2.5^{2}\operatorname{Id})$ , with $m=0.8$ and, when inexact, $\widetilde{m}=0.55$ ; offsets are $\omega_{t}=c\,B_{\widetilde{m}}(\gamma_{t})$ with $M_{0}=0.03$ and $M_{1}=0.05$ .

Panel of Figure 2 — Figure 2: Approximate $\tilde{q}_{t}$ and the predicted profile $q_{t}$ of Corollary 3.4 . The analytical curve is used as a fitted functional form; it is not a certificate of the learned map’s global contraction factor.

Panel of Figure 3 — Figure 3: Approximate target-only one-step error $E^{\pi}_{t}$ for the TrigFlow and VE LCMs, compared with the analytical VE shape of Corollary 3.4 used as an upper envelope.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Regime	Initialization	Learned step	Certificate overstates	Outcome
Mean	mean	mean	nothing	exact
Initial variance	variance	none	inherited variance, by $\kappa_{t}$	ratio $\sim q_{t}^{-n}$
Learned variance	variance or none	variance	inherited variance, by $\kappa_{t}$	no uniform $E_{t}$
Both	both kinds	both kinds	the same, diluted by the mean	loose, then tight

Appendix

Table 1: Regimes of Section E.3 . The second and third columns give the kind of error introduced by the initialization and by the learned step, none meaning exact. The certificate also adds the sizes of variance errors of opposite directions, which may cancel ( Section E.2 ).

Panel of Figure 4 — Figure 4: Error in the mean only: $m=0.8$ , the curvature fitted in Section 4 , $\widetilde{m}=m$ , $\eta_{r}=M_{0}+M_{1}/r$ with $M_{0}=0.03$ , $M_{1}=0.05$ , and $s_{0}=m^{-\nicefrac{{1}}{{2}}}$ . Left: from $\zeta_{0}=2$ , along $t_{k}=2(0.85)^{k-1}$ in VE coordinates and $t_{k}=1.3(0.85)^{k-1}$ in TrigFlow coordinates, the certificate ( 3 ) (markers) and the exact error (lines) coincide; stars mark the optimal step count. Right: at the repeated VE level $t=1$ , the error moves to the plateau $E_{t}/(1-q_{t})$ from above, at, or below it.

Panel of Figure 5 — Figure 5: Choosing the level of a second step, in VE coordinates with $m=0.8$ and $d=1$ : a first step at $t_{1}=4$ from $\mathcal{N}(0,2.5^{2}\operatorname{Id})$ , then a second at level $t_{2}$ , for no error in the mean and for the offsets $\eta_{r}$ and $4\eta_{r}$ , with $\eta_{r}$ as in Fig. 4 , on a logarithmic scale. Exact error solid, certificate dashed; circles mark the best level ( $t_{2}=0$ : no second step), stars the certificate’s choice and dots the exact error at that choice. Left: the learned variance is too large ( $\widetilde{m}=0.55$ ), like the initialization, so no cancellation occurs. Right: it is too small ( $\widetilde{m}=1.1$ ), so the learned step makes the law too narrow while the initialization is too wide, and the two variance errors cancel.

Panel of Figure 6 — Figure 6: One-step SD 1.5 experiment. The measured empirical W2 error is compared with the plug-in recursive estimate $\widehat{q}_{t}\widehat{D}_{0}+\widehat{E}_{t}^{\pi}$ .

Consistency Models

Consistency Models are generative models trained to map points along a diffusion trajectory to the same clean sample, enabling fast few-step generation.

See all 25 papers

Dual-End Consistency Model
Feb 11, 2026Linwei Dong, Ruoyu Guo, Ge Bai +3One-Step Generative Modeling Consistency Distillation
1Zhejiang University · 2Bytedance Inc. · 3Zhejiang Lab
Noise Schedule Design for Diffusion Models: An Optimal Control Perspective
May 21, 2026Seo Taek Kong, Weina Wang, R. SrikantDiffusion Sampling Diffusion-Based Image Generation
ECE & CSL, University of Illinois Urbana-Champaign · Computer Science Department, Carnegie Mellon University · ECE, CSL & NCSA, University of Illinois Urbana-Champaign
TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling
Oct 9, 2026George Matthew, Guoqiang Zhang, Kenta NiwaFew-Step Diffusion Sampling Diffusion Model Inference Acceleration
University of Exeter, United Kingdom · NTT Communication Science Laboratories, Japan

Feb 11, 2026J/K move · Enter open · S save

Dual-End Consistency Model

Linwei Dong, Ruoyu Guo, Ge Bai, Zehuan Yuan+2

1Zhejiang University · 2Bytedance Inc. · 3Zhejiang Lab

One-Step Generative Modeling Consistency Distillation Diffusion-Based Image Generation Flow-Based Generative Modeling Consistency Models

Open paper AskPDF arXiv

The slow iterative sampling nature remains a major bottleneck for the practical deployment of diffusion and flow-based generative models. While consistency models (CMs) represent a state-of-the-art distillation-based approach for efficient generation, their large-scale application is still limited by two key issues: training instability and inflexible sampling. Existing methods seek to mitigate these problems through architectural adjustments or regularized objectives, yet overlook the critical reliance on trajectory selection. In this work, we first conduct an analysis on these two limitations: training instability originates from loss divergence induced by unstable self-supervised term, whereas sampling inflexibility arises from error accumulation. Based on these insights and analysis, we propose the Dual-End Consistency Model (DE-CM) that selects vital sub-trajectory clusters to achieve stable and effective training. DE-CM decomposes the PF-ODE trajectory and selects three critical sub-trajectories as optimization targets. Specifically, our approach leverages continuous-time CMs objectives to achieve few-step distillation and utilizes flow matching as a boundary regularizer to stabilize the training process. Furthermore, we propose a novel noise-to-noisy (N2N) mapping that can map noise to any point, thereby alleviating the error accumulation in the first step. Extensive experimental results show the effectiveness of our method: it achieves a state-of-the-art FID score of 1.70 in one-step generation on the ImageNet 256x256 dataset, outperforming existing CM-based one-step approaches.

Abstract

Consistency Models

Explore similar work

Dual-End Consistency Model

Noise Schedule Design for Diffusion Models: An Optimal Control Perspective

TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling