Instruction-based image editors built on diffusion transformers run tens of denoising steps. Training-free acceleration either reuses features across steps (step caching) or stops computing the regions an edit leaves untouched (region freezing), and the two have been combined before. We ask how a high speed-up target should be split between them. On Qwen-Image-2.1, freezing does more than remove tokens: it lowers the cost of every step, so the step cache can reach the same overall speed-up with a conservative threshold that keeps its error small. A per-step cost model fitted on development runs predicts held-out latency with a median error of 4.0% and reproduces the speed ceilings of the region methods we re-implemented (2.73× predicted vs. 2.70× measured for RegionE). Driving frozen tokens to the model's one-step x0 estimate beats returning them to the reference image on every development sample, and in 90% of measured cases the composed error stays below the sum of the two individual errors. The resulting method, FreezePlan, keeps five full steps and gives the rest of the budget to TaylorSeer. On a pre-registered test split it runs 5.05× faster end to end at 29.8 dB PSNR to the original output, with no significant drop in a VLM judge score; TaylorSeer at 5.02× reaches 23.5 dB. The curve extends to 5.99× at 28.0 dB. Swapping in the region selectors of prior methods under our schedule lowers median PSNR by only 0.5--0.6 dB, which places the gain in the schedule and budget split rather than in the selector.
Figures & tables
Fig. 1: FreezePlan (Sec. III-E ). (a) Schedule: τ full steps, one region decision, compact steps over active tokens with step caching, one optional refresh. (b) In a compact step, frozen tokens move straight to their anchor c=x^0(στ) and their K/V is appended once to the prefix cache that active tokens attend to. The cached-step pattern in (a) is illustrative.
Fig. 2: Frozen-token anchor (development set, unchanged tokens). (a) Latent error reduction of the x0 anchor over the reference anchor vs. the step of the estimate (median and IQR over 44 samples). (b) Decoded PSNR of the unchanged region to the dense output and to the input.
Schedule
Full steps
Ceiling
Measured max.
RegionE [ 16 ]
13
2.73 ×
2.70 × (+TS)
SpotEdit [ 15 ]
7
4.45 ×
3.90 × (+TS)
FreezePlan
5
5.62 ×
5.48 ×
FreezePlan (no refresh)
4
6.48 ×
5.99 ×
TABLE I: Speed ceilings implied by Eq. ( 1 ) (all non-full steps free) and the highest measured speed-up on the test split.
Fig. 3: Composition on the development set. (a) Measured composed PSNR vs. the PSNR implied by summing the individual MSEs; 90% of points lie above the diagonal. (b) Speed-up vs. cache threshold δ ; labels give PSNR. With freezing, 5 × is reached at δ=0.3 .
Method
Speed ↑
PSNR ↑
Edit ↑
SSIM ↑
Δ VLM
TaylorSeer δ=0.8
5.02 ×
23.45
19.71
0.707
− 1.59 [ − 1.93, − 1.25]
TaylorSeer δ=1.0
5.66 ×
19.39
17.11
0.522
− 4.11 [ − 4.67, − 3.53]
DBCache δ=0.8
5.03 ×
23.33
19.65
0.700
− 1.67 [ − 2.01, − 1.33]
6 steps
5.17 ×
22.22
17.51
0.869
− 0.23 [ − 0.38, − 0.10]
SpotEdit+TS †
3.90 ×
26.14
20.00
0.884
− 0.44 [ − 0.78, − 0.17]
RegionE+TS †
2.70 ×
32.93
27.60
0.960
+ 0.12 [ − 0.10, + 0.47]
TABLE II: Test split B (44 samples, pre-registered). Medians; Δ VLM: mean paired difference to the dense output with 95% CI. † Fastest setting the method reaches.
Fig. 4: Speed–fidelity on test split B. (a) PSNR to the dense output vs. real speed-up; (b) Δ VLM. Shaded: the ≥ 5 × regime.
Variant
Speed
PSNR
ρ
Δ VLM
FreezePlan selector
5.03 ×
29.77
0.41
− 0.06
RegionE selector [ 16 ]
5.23 ×
29.29
0.31
+ 0.05
SpotEdit selector [ 15 ]
5.01 ×
29.14
0.43
− 0.16
reference anchor (SpotEdit-style)
5.06 ×
27.84
0.41
− 0.40
no frozen K/V
5.15 ×
29.02
0.41
− 0.21
no global fallback
5.11 ×
28.84
0.37
− 0.15
TABLE III: Top: region selectors under the schedule and cache of FreezePlan (post hoc). Bottom: one component removed at a time. Test split B; ρ : mean fraction of tokens computed per step.
Fig. 5: “Replace the cup in her hands with a slice of strawberry cake” on an original anime illustration (outside the benchmark), at about 5 × real speed-up. Labels: speed-up ⋅ PSNR to the 40-step output; bottom row: crop of the edited region. Step caching alone (TaylorSeer, DBCache) breaks down; fewer steps give a plausible but different cake; the re-implemented region methods (SpotEdit, RegionE, both with TaylorSeer) cannot reach this speed with their schedules. FreezePlan (5.11 × ) matches the dense output.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Share of cases below the bound
90% (95% CI 86–94%)
Median ratio composed / bound
0.76 (95% CI 0.68–0.81)
Exceptions
18 (all at δ=0.3 )
with refresh at step 16 / 21
15 / 3
distinct samples
15 of 30
Excess over the bound (median / max.)
1.5 / 4.0 dB
Cache share of the bound (exceptions / others)
0.56 / 0.40
Appendix
TABLE IV: Composed MSE relative to the additive bound (180 cases). CIs bootstrap over samples.
Configuration
Measured
Predicted
Time error
TaylorSeer δ=0.3 / 0.8 / 1.0
4.09 / 5.02 / 5.66 ×
4.15 / 5.14 / 5.84 ×
1.3–2.3%
8 / 7 / 6 steps*
4.15 / 4.61 / 5.17 ×
4.03 / 4.45 / 4.97 ×
3.4–4.5%
RegionE+TS*
2.66–2.70 ×
2.61–2.64 ×
2.5%
SpotEdit+TS*
3.81–3.90 ×
4.09–4.16 ×
6.4%
FreezePlan+TS (all variants)
5.05–5.99 ×
4.78–5.59 ×
4.9–6.0%
All 1056 runs
median 4.0%, P90 7.9%
Appendix
TABLE V: Out-of-sample prediction on split B (1056 runs). Configurations marked * do not appear in the development set.
Subset
Samples
Fallback
ρ (no fallback)
Global categories
20
9 (style 4/4, background 4/4, tone 1/4)
0.37
Local categories
24
2 (colour 1, text 1)
0.40
All
44
11 (25%)
0.38
Appendix
TABLE VI: Fallback rate and active fraction on split B.
Fig. 6: Split B, about 5 × ; rows chosen by the rule above (crops of the edited region below each row). Labels: speed-up ⋅ PSNR to the 40-step output.
Fig. 7: All five original anime illustrations, about 5 × , same configurations as split B.
Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation, yet their iterative denoising process incurs substantial computational cost during inference. Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions. In this work, we identify three orthogonal axes of non-uniformity in DiT denoising: (1) temporal -- sensitivity to caching errors varies dramatically across the denoising trajectory; (2) depth -- consecutive caching decisions lead to cascading approximation errors; and (3) feature -- different components of the hidden state exhibit heterogeneous temporal dynamics. Based on these observations, we propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC). On FLUX.1-schnell at 512x512 resolution, SpectralCache achieves 2.46x speedup with LPIPS 0.217 and SSIM 0.727, outperforming TeaCache (2.12x, LPIPS 0.215, SSIM 0.734) by 16% in speed while maintaining comparable quality (LPIPS difference < 1%). Our approach is training-free, plug-and-play, and compatible with existing DiT architectures.
Step-level caching accelerates diffusion models by exploiting temporal redundancy across denoising steps. Existing methods make per-step cache decisions using threshold-based heuristics, without directly optimizing for final output quality. As a result, their inference latency varies across inputs and is difficult to control at deployment. In this work, we propose BudCache, which inverts this formulation: rather than letting per-step error thresholds dictate the runtime cost, we fix the compute budget in advance and search for the cache policy that best preserves the final output. To tackle the combinatorial complexity of step selection, we combine Simulated Annealing with deterministic Hill Climbing. This offline search identifies high-quality cache policies within minutes and introduces no online search or thresholding overhead during inference. When the compute budget is very tight, we further introduce cache-aware schedule alignment, which adapts the time discretization to the selected cache policy to reduce cache-induced trajectory mismatch. Experiments on FLUX.1-dev and Wan2.1 show that BudCache achieves better generation quality than heuristic caching baselines under the same inference budgets. Code is available at https://github.com/Westlake-AGI-Lab/BudCache
Diffusion-based image editing offers strong semantic controllability, but remains computationally expensive due to iterative high-resolution denoising over all spatial tokens. Dynamic-resolution sampling reduces this cost by performing early steps at reduced resolution. However, existing approaches prioritize upsampling using low-level heuristics such as edge detection or channel variance, which are weakly aligned with editing semantics and may lead to structural inconsistency. Moreover, spatial regions are often upsampled without verifying whether semantic modification is actually required, resulting in redundant high-resolution computation and accumulated errors. Therefore, we propose SpecEdit, a training-free dynamic-resolution framework tailored for diffusion-based image editing. SpecEdit follows a draft-and-verify scheme: a low-resolution draft first estimates the semantic outcome, after which token-level discrepancies are used to identify edit-relevant tokens for high-resolution denoising, while the remaining tokens stay at a coarse resolution. Experiments on Qwen-Image-Edit and FLUX.1-Kontext-dev demonstrate up to 10x and 7x acceleration, while maintaining strong quality. SpecEdit is complementary to step distillation and other acceleration techniques, achieving up to 13x speedup when combined with existing methods. Our code is in supplementary material and will be released on GitHub.
Zhengan Yan, Shikang Zheng, Haoran Qin +9
EPIC Lab , Shanghai Jiao Tong University · Shandong University · Alibaba Group +2