Equal Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models

Wei Guo; Yaowen Zhang; Xingtong Ge; Jun Zhang

cs.AIOct 8, 2026

Equal Path Cost, Unequal Output Effects: Understanding Perturbation Propagation in Diffusion Models

Authors: Wei Guo, Yaowen Zhang, Xingtong Ge, Jun Zhang

Organizations: iComAI Lab, The Hong Kong University of Science and Technology · Department of Physics, The Hong Kong University of Science and Technology

Abstract

Diffusion models have achieved remarkable success in generative modeling, with their sampling procedures routinely modified to control generation and improve efficiency. These modifications introduce perturbations along the sampling trajectory, raising a central question: how do such perturbations affect generated output? To address this question, we develop a theoretical framework to investigate perturbation propagation, combining dynamical analysis of the sampling process with an information-theoretic characterization of output responses. Within this framework, we quantify perturbation strength using the Kullback--Leibler (KL) divergence between perturbed and reference trajectory distributions, termed as path cost, which is shown to bound, but do not determine, changes in the output distribution. Building on this analysis, we derive a response identity that connects the propagation and accumulation of local perturbations with the information captured by a selected feature mean, explaining why changes in the output distribution can remain undetected by its first-order response. We test our theoretical analysis through controlled interventions at equal path cost in pretrained diffusion models, revealing distinct patterns of output sensitivity across sampling stages and spatial frequencies. To assess whether our framework can diagnose perturbations arising from practical approximations, we apply it to cache-based acceleration and show that our propagation analysis reliably identifies sampling intervals where caching causes larger image errors.

Figures & tables

Figure 1: Equal path costs, different output-distribution changes. Two temporal perturbation sequences act only on the first coordinate of a two-dimensional Gaussian reverse diffusion, each with path cost $\mathcal{C}=2$ nats. Their contributions reinforce (top) or cancel (bottom) at the output. (a) Coordinate density evolution on a shared color scale, with perturbed (white) and reference (dashed) means. (b) Perturbed (solid) and reference (dashed) output densities. (c) Output KL (colored) and bridge cost (gray). Construction details are in Appendix B .

Figure 2: Output responses at equal path cost within each model. Rows show perturbation frequencies and columns show the sampling windows in which perturbations are applied, from early to late. Each row is divided by its maximum response. White dots mark the peak windows and numbers on the right give the maxima before normalization.

Figure 3: Local predictions and finite perturbation responses in DiT. Mean decoded-image RMS of local derivatives and symmetric finite differences (

\varepsilon=0.8

) on the same 32 trajectories, where shaded bands indicate one standard error of the mean across samples. The right panel reports finite/local peak ratios with dashed line denoting equal amplitudes. All runs use 128 stochastic steps.

	JiT-B			JiT-L			ADM
Feature	$\|\widehat{\chi}_{f}\|$	$s_{f}$	$\widehat{B}_{f}$	$\|\widehat{\chi}_{f}\|$	$s_{f}$	$\widehat{B}_{f}$	$\|\widehat{\chi}_{f}\|$	$s_{f}$	$\widehat{B}_{f}$
DC	$3.31$	$127$	$0.000677$	$3.36$	$114$	$0.000875$	$70.7$	$106$	$0.447$
DCT-1	$2.41$	$48.5$	$0.00248$	$2.46$	$45.2$	$0.00296$	$24.9$	$42.5$	$0.343$
DCT-4	$1.43$	$18.9$	$0.00577$	$1.46$	$21.4$	$0.00465$	$6.9$	$23.5$	$0.0865$
DCT-16	$0.384$	$3.88$	$0.00982$	$0.366$	$4.34$	$0.00712$	$0.274$	$4.32$	$0.00402$

Table 1: Feature responses relative to baseline fluctuations. Perturbations act at window 1 at equal path cost, with

\varepsilon=0.25

and

N=512

for JiT-B/L and

256

for ADM. We report absolute mean response

|\widehat{\chi}_{f}|

, baseline standard deviation

s_{f}

, and normalized response

\widehat{B}_{f}=\widehat{\chi}_{f}^{2}/s_{f}^{2}

Figure 4: Predicted and observed image MSE under caching. Black solid lines show observed image MSE and blue dashed lines show predictions from the propagation-based predictor, averaged over the same 64 samples per model. Each candidate reuses the prediction cached at the start of one eight-step window, and windows are ordered from noise toward output.

Model	Our predictor	Local defect	Path cost
JiT-B	0.9524	0.2143	-0.4762
JiT-L	0.9762	0.2381	-0.4286
ADM	0.9762	-0.9048	-0.9048
DiT	0.9762	0.5000	-0.3095

Table 2: Spearman correlations with observed mean image MSE across eight caching windows, using the runs in Figure 4 . Bold values mark the highest correlation for each model.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$N$	$\varepsilon$
JiT-B/16	512	0.25
JiT-L/16	512	0.25
ADM	256	0.25
DiT-XL/2	256	0.80

Appendix

Table 3: Sample counts

N

and perturbation amplitudes

\varepsilon

for the controlled perturbation experiments in Section 4 . All use 128 stochastic steps, eight injection windows, and path Fisher

F_{\mathrm{path}}=1

Figure 5: DiT image responses to equal-cost perturbations. A fixed sample under positive and negative DCT-4 perturbations in windows 4 and 8. Difference maps show per-pixel RGB RMS relative to the reference $Y^{0}$ , with full-image RMS below each map. Central-response annotations measure the RMS of $(Y^{+}-Y^{-})/(2\varepsilon)$ ; the window-4/window-8 ratio is $19.88$ . Differences use unclipped pixels and a shared color scale across both examples.

Model	Feature	$s_{f}$	W1	W2	W3	W4	W5	W6	W7	W8
JiT-B	DC	$127$	$3.31$	$0.704$	$0.361$	$0.212$	$0.128$	$0.0786$	$0.0462$	$0.0187$
	DCT-1	$48.5$	$2.41$	$0.661$	$0.345$	$0.206$	$0.132$	$0.0785$	$0.0442$	$0.0131$
	DCT-4	$18.9$	$1.43$	$0.538$	$0.303$	$0.189$	$0.12$	$0.0768$	$0.0407$	$0.0164$
	DCT-16	$3.88$	$0.384$	$0.264$	$0.183$	$0.131$	$0.0908$	$0.061$	$0.0371$	$0.0149$
JiT-L	DC	$114$	$3.36$	$0.709$	$0.358$	$0.215$	$0.132$	$0.0783$	$0.0424$	$0.0165$
	DCT-1	$45.2$	$2.46$	$0.67$	$0.351$	$0.21$	$0.129$	$0.0821$	$0.0432$	$0.0168$

Appendix

Table 4: Feature mean responses across sampling windows. Signed mean responses

\widehat{\chi}_{f}

for windows W1–W8, ordered from early to late sampling, with baseline standard deviations

s_{f}

. All use

\varepsilon=0.25

, with

N=512

for JiT-B/L and

256

for ADM.

Model	Feature	W1	W2	W3	W4	W5	W6	W7	W8
JiT-B	DC	$0.000677$	$3.06\mathrm{e}{-5}$	$8.05\mathrm{e}{-6}$	$2.77\mathrm{e}{-6}$	$1.01\mathrm{e}{-6}$	$3.82\mathrm{e}{-7}$	$1.32\mathrm{e}{-7}$	$2.17\mathrm{e}{-8}$
	DCT-1	$0.00248$	$0.000186$	$5.07\mathrm{e}{-5}$	$1.81\mathrm{e}{-5}$	$7.38\mathrm{e}{-6}$	$2.62\mathrm{e}{-6}$	$8.32\mathrm{e}{-7}$	$7.26\mathrm{e}{-8}$
	DCT-4	$0.00577$	$0.000813$	$0.000258$	$0.000101$	$4.08\mathrm{e}{-5}$	$1.66\mathrm{e}{-5}$	$4.66\mathrm{e}{-6}$	$7.52\mathrm{e}{-7}$
	DCT-16	$0.00982$	$0.00465$	$0.00222$	$0.00114$	$0.000548$	$0.000247$	$9.13\mathrm{e}{-5}$	$1.48\mathrm{e}{-5}$
JiT-L	DC	$0.000875$	$3.89\mathrm{e}{-5}$	$9.92\mathrm{e}{-6}$	$3.56\mathrm{e}{-6}$	$1.35\mathrm{e}{-6}$	$4.73\mathrm{e}{-7}$	$1.39\mathrm{e}{-7}$	$2.1\mathrm{e}{-8}$
	DCT-1	$0.00296$	$0.00022$	$6.03\mathrm{e}{-5}$	$2.16\mathrm{e}{-5}$	$8.2\mathrm{e}{-6}$	$3.3\mathrm{e}{-6}$	$9.14\mathrm{e}{-7}$	$1.39\mathrm{e}{-7}$

Appendix

Table 5: Feature responses on the baseline fluctuation scale. Normalized responses

\widehat{B}_{f}=\widehat{\chi}_{f}^{2}/s_{f}^{2}

for the measurements in Table 4 .

Explore similar work

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis
Jun 13, 2025Bernardo P. Schaeffer, Ricardo M. S. Rosa, Glauco ValleStochastic Differential Equations Diffusion Sampling
Instituto de Matem´atica, Universidade Federal do Rio de Janeiro
Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution
Sep 30, 2026Shixuan Liu, Joan Serrà, Kin Wai Cheuk +6Gradient-Based Attribution Training Data Attribution
University of Illinois at Urbana-Champaign · Sony AI · University of Texas at Austin +1
Attributing HOW, Not Just WHICH: Counterfactual Response Trajectories for Diffusion Models
Oct 8, 2026Haoqian Zhang, Ziyuan Yang, Zerui Shao +1Training Data Attribution Diffusion Models
Sichuan University

Jun 13, 2025, cs.LGJ/K move · Enter open · S save

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis

Bernardo P. Schaeffer, Ricardo M. S. Rosa, Glauco Valle

Instituto de Matem´atica, Universidade Federal do Rio de Janeiro

Stochastic Differential Equations Diffusion Sampling Score-Based Generative Modeling Diffusion Models

Open paper AskPDF arXiv

Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.

Abstract

Explore similar work

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis

Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution

Attributing HOW, Not Just WHICH: Counterfactual Response Trajectories for Diffusion Models