Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance

Inho Kong; Sojin Lee; Youngjoon Hong; Hyunwoo J. Kim

cs.CVMar 4, 2026

Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance

Authors: Inho Kong, Sojin Lee, Youngjoon Hong, Hyunwoo J. Kim

Organizations: Korea University · KAIST · Seoul National University · Korea Institute for Advanced Study

Abstract

Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality. Autoguidance (AG) has extended this idea, but it relies on an auxiliary network and leaves solver-induced errors unaddressed. In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes a critical factor that deteriorates sample quality. Our key observation is that these errors align with the dominant eigenvector, motivating us to leverage the solver-induced error as a guidance signal. We propose Embedded Runge-Kutta Guidance (ERK-Guid), which exploits detected stiffness to reduce LTE and stabilize sampling. We theoretically and empirically analyze stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid. Our experiments on both synthetic datasets and the popular benchmark dataset, ImageNet, demonstrate that ERK-Guid consistently outperforms state-of-the-art methods. Code is available at https://github.com/mlvlab/ERK-Guid.

Figures & tables

Panel of Figure 1 — Figure 1: Toy 2D example of stiffness and local drift variations. (a) Visualization of stiffness across the 2D plane, colored by magnitude. We sample a non-stiff region (lines AB and CD) and a stiff region (lines EF and GH). (b, c) In the non-stiff region, the drift projected onto the local eigenbasis exhibits minimal variation along both axes, indicating stable dynamics. (d, e) Conversely, in the stiff region, the projected drift shows pronounced variation along the dominant eigenvector axis (d), while remaining relatively parallel along the subdominant axis (e). Table 1 shows that for a single step in the stiff region, both the local truncation error (LTE) and the ERK solution difference are heavily amplified along the dominant axis (EF) due to these drift variations. This strong alignment validates the ERK difference as a practical proxy for the severe LTE in our proposed ERK-Guid.

Figure 2: Eigenvector alignment across stiffness. (a) Cosine similarity between the dominant eigenvector and the local truncation error (LTE) increases with stiffness. (b) The ERK solution difference exhibits a similar trend to LTE, suggesting it can serve as a reliable proxy for the LTE direction in high stiffness regions. (c) Our ERK-Guid consistently achieves higher cosine similarity with the eigenvector, highlighting its strong alignment in stiff regions. CFG and Autoguidance exhibit weaker or mixed alignment with the dominant eigenvector in stiff regions, supporting the complementarity of our method.

Table 4

Dataset	ImageNet 64 $\times$ 64 (FID $\downarrow$ )			FFHQ 64 $\times$ 64 (FID $\downarrow$ )
NFEs	6	8	10	6	8	10
Heun Hairer et al. (1993)	89.63	37.65	16.46	142.4	57.21	29.54
+ ERK-Guid	85.19	35.93	13.85	132.8	54.72	23.36
DPM-Solver Lu et al. (2022)	44.83	12.42	6.84	83.17	22.84	9.46
+ ERK-Guid	31.59	10.57	6.54	49.0	10.42	4.64
DEIS Zhang and Chen (2023)	12.57	6.84	5.34	12.25	7.59	5.56

Table 4: Quantitative results of plug-and-play adaptation of ERK-Guid to solver methods on ImageNet-64 and FFHQ-64.

Figure 5: Qualitative comparison on PixArt- $\alpha$ Chen et al. (2023) . We perform text-to-image generation to compare DPM-Solver with our ERK-Guid. As shown in the blue zoomed-in regions, ERK-Guid captures fine semantic details more accurately.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	ImageNet 64 $\times$ 64			FFHQ 64 $\times$ 64
NFEs	6	8	10	6	8	10
Heun	0.75	1.25	1.0	0.5	1.25	1.5
DPM-Solver	1.25	1.25	1.25	1.25	1.25	1.25
DEIS	2.5	2.5	2.5	3.5	3.5	3.5

Appendix

Table 5: Values of the guidance magnitude hyperparameter

w_{\mathrm{stiff}}

Table 8

Panel of Figure 6 — Figure 6: Choice of ablation functions. (a) Functional comparison of $\alpha(z)$ and its quadratic approximation. (b) Comparison of $\alpha(z)$ , $|z|$ , and $z^{2}$ , under the FID–FD-DINOv2 trade-off, where the quadratic form $z^{2}$ achieves the most favorable balance between stability and fidelity.

#step	Method	FD-DINOv2 $\downarrow$	FID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	IS $\uparrow$
32	Autoguidance	50.4	1.36	0.691	0.628	262
	+ERK-Guid	47.6	1.36	0.694	0.630	268
	+ERK-Proj	44.9	1.36	0.710	0.605	274

Appendix

Table 8: Quantitative results of ERK-proj.

Solver	Pair of states	$h$	$\beta$
Heun	$\mathbf{x}_{\sigma_{i}},\,\mathbf{x}_{\sigma_{i}}^{\mathrm{Euler}}$	$\sigma_{i}-\sigma_{i+1}$	$\{0,1\}$
DPM-Solver (2S)	$\mathbf{x}_{\sigma_{i}+\delta},\,\mathbf{x}_{\sigma_{i}}$	$\sigma_{i}-\sigma_{i+1}$	$\{0,1\}$
DEIS	$\mathbf{x}_{\sigma_{i}},\,\mathbf{x}_{\sigma_{i-1}}$	$\sigma_{i-1}-\sigma_{i}$	$\{0,-1\}$

Appendix

Table 9: Guidance configuration for Heun, DPM-Solver, and DEIS.

	Adaptive step-size	Threshold	NFE (Avg.) $\downarrow$	FD-DINOv2 $\downarrow$	FID $\downarrow$
(A) ERK-Guid (Ours)	✗	–	63	86.2	2.56
(B) Heun	✗	–	63	90.1	2.58
(C)	✓	$\tau=0.5$	90.7	88.9	2.57
(D)	✓	$\tau=1.0$	85.5	89.4	2.57
(E)	✓	$\tau=2.0$	79.5	90.0	2.58
(F)	✓	$\tau=5.0$	67.6	90.1	2.58

Appendix

Table 10: Evaluation of adaptive step-size control under different stiffness thresholds. (A) ERK-Guid and (B) Heun do not adopt adaptive step-size. (C–G) adapt step-size using thresholds

\tau=0.5,1,2,5,10

Method	Stochasticity	$r$	NFE	FD-DINOv2 $\downarrow$	FID $\downarrow$
	✓	0.05	64	91.5	2.65
	✓	0.10	64	93.0	2.66
Predictor-Corector	✓	0.15	64	98.9	2.90
Song et al. (2021b)	✗	0.01	64	90.6	2.61
	✗	0.02	64	87.4	2.73
	✗	0.03	64	88.6	3.33

Appendix

Table 11: Quantitative results of predictor-corrector and ERK-Guid (Ours).

Figure 7: Convergence of stiffness estimation under JVP power iteration. The plot shows the absolute error between the estimated stiffness at iteration k and the final converged value. The solid line denotes the median across all seeds and timesteps, while the dashed lines indicate the 25th and 75th percentiles. Errors decrease rapidly and stabilize, demonstrating reliable convergence of the iteration procedure.

Figure 8: Qualitative results of ERK-Guid across adjusting guidance scale.

Figure 9: Qualitative results of ERK-Guid across guidance scales.

Explore similar work

Analytic Distribution of Classifier-Free Guidance for Schedule Design
Jul 22, 2026Enze Jiang, Zheng MaClassifier-Free Guidance Diffusion Sampling
School of Mathematical Sciences Shanghai Jiao Tong University Shanghai, China
CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control
Oct 5, 2026Max Collins, Dasith de Silva Edirimuni, Jordan Vice +2Diffusion Model Guidance Classifier-Free Guidance
School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009
DiffRGD: An Inference-Time Diffusion Guidance Through Riemannian Gradient Descent
Jun 25, 2026Jia-Wei Liao, Li-Xuan Peng, Mei-Heng Yueh +3Diffusion Models for Image Restoration Diffusion Model Guidance
1National Taiwan University · 4Academia Sinica · 2National Tsing Hua University +1

Jul 22, 2026J/K move · Enter open · S save

Analytic Distribution of Classifier-Free Guidance for Schedule Design

Enze Jiang, Zheng Ma

School of Mathematical Sciences Shanghai Jiao Tong University Shanghai, China

Classifier-Free Guidance Diffusion Sampling Diffusion-Based Image Generation Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Classifier-free guidance (CFG) is the default mechanism for conditional generation in diffusion models, but the distribution sampled by its deterministic guided dynamics is not captured by the usual product-distribution heuristic

p_0^ωq_0^{1-ω}

. We analyze CFG through the probability flow ODE and derive exact analytic path-integral representations of the induced distributions for both constant and time-dependent guidance. The resulting formulas show that CFG modifies

p_{t_0}

by an exponential path-integral correction, and that a time-dependent schedule enters this correction through the weight

ω(t)-1

. This characterization explains how score discrepancies accumulate along sampling trajectories and motivates Distribution-Guided CFG (DG-CFG), a schedule that balances timestep contributions while accounting for signal strength and low-noise score-error amplification. A toy model with analytic scores closely verifies the predicted distributions. Across Stable Diffusion~~1.5, Stable Diffusion~~2.1, and Stable DiffusionXL, DG-CFG yields a stronger diversity--fidelity trade-off and robustly mitigates the saturation and quality degradation caused by strong constant or heuristic guidance. Complete NFE experiments on Stable Diffusion1.5 and Stable Diffusion~2.1 confirm that these gains persist across sampling budgets, while fixed-quality experiments on both backbones show that DG-CFG reaches target metrics with fewer sampling steps.

Abstract

Explore similar work

Analytic Distribution of Classifier-Free Guidance for Schedule Design

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

DiffRGD: An Inference-Time Diffusion Guidance Through Riemannian Gradient Descent