SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Models

Jiwoo Chung; Sangeek Hyun; MinKyu Lee; Byeongju Han; Geonho Cha; Dongyoon Wee; Youngjun Hong; Jae-Pil Heo

cs.CVFeb 22, 2026

SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Models

Authors: Jiwoo Chung, Sangeek Hyun, MinKyu Lee, Byeongju Han, Geonho Cha, Dongyoon Wee, Youngjun Hong, Jae-Pil Heo

Organizations: Sungkyunkwan University · NAVER Cloud

Abstract

Diffusion models are a strong backbone for visual generation, but their inherently sequential denoising process leads to slow inference. Previous methods accelerate sampling by caching and reusing intermediate outputs based on feature distances between adjacent timesteps. However, existing caching strategies typically rely on raw feature differences that entangle content and noise. This design overlooks spectral evolution, where low-frequency structure appears early and high-frequency detail is refined later. We introduce Spectral-Evolution-Aware Cache (SeaCache), a training-free cache schedule that bases reuse decisions on a spectrally aligned representation. Through theoretical and empirical analysis, we derive a Spectral-Evolution-Aware (SEA) filter that preserves content-relevant components while suppressing noise. Employing SEA-filtered input features to estimate redundancy leads to dynamic schedules that adapt to content while respecting the spectral priors underlying the diffusion model. Extensive experiments on diverse visual generative models and the baselines show that SeaCache achieves state-of-the-art latency-quality trade-offs.

Figures & tables

Panel of Figure 1 — Figure 1: Conceptual illustration and motivation of the proposed caching scheme (SeaCache) compared with previous caching schemes. The lower panel shows a denoising trajectory of a cat image where coarse low-frequency structure appears at early steps and fine high-frequency details emerge at later steps, illustrating the spectral evolution of iterative generative models. SeaCache applies a Spectral-Evolution-Aware (SEA) Filter to raw diffusion features so that the distance measure better captures timestep-aware spectral residuals between timesteps.

Panel of Figure 2 — Figure 2: Latency-quality trade-off in oracle experiments. We compare cache decisions based on raw output differences and SEA-filtered output differences (Sec. 4.1 ) on FLUX [ 29 , 28 ] and Wan2.1 1.3B [ 63 ] . The refresh ratio is the fraction of timesteps that run a full denoiser evaluation instead of reusing cached features. For each criterion, PSNR is computed between the cached sample and the corresponding full timestep (no-cache) sample, averaged over each prompt set [ 49 , 23 ] . At matched refresh ratios, the filtered criterion consistently achieves higher PSNR with respect to the full-compute trajectory, validating the effectiveness of a spectrum-aware distance for cache scheduling.

Figure 3: Overview of SeaCache. Given input features

I_{t}

and

I_{t+1}

, SeaCache first applies FFT, multiplies by the timestep-dependent SEA filters

G_{t}^{\mathrm{norm}}

and

G_{t+1}^{\mathrm{norm}}

, and then applies iFFT to obtain spectral-evolution-aware features

\mathcal{P}(G_{t}^{\mathrm{norm}},I_{t})

and

\mathcal{P}(G_{t+1}^{\mathrm{norm}},I_{t+1})

(Sec. 4.1 ). A spectrum-aware dynamic caching module (Sec. 4.2 ) measures the relative distance

\widetilde{\Delta}_{t}

between consecutive filtered features, accumulates it over timesteps, and either reuses the cached output or refreshes the denoiser when the threshold

\delta

is exceeded. The underlying diffusion model remains unchanged, so SeaCache acts as a plug-and-play cache policy that replaces only the distance metric.

Panel of Figure 4 — Figure 4: Visualization of timestep-dependent denoising filters. (a) Optimal linear denoising responses $G_{t}(f)$ across timesteps, where early steps primarily pass low-frequencies and later steps gradually include higher frequencies, reflecting spectral evolution. (b) Corresponding normalized filters $G^{\mathrm{norm}}_{t}(f)$ with unit mean gain, which stabilize filtered feature energy across timesteps and are used as SEA filters for cache scheduling.

Panel of Figure 5 — Figure 5: Relative $\ell_{1}$ across the generation process. Stepwise relative $\ell_{1}$ distances between consecutive timesteps for different feature choices, averaged over ten samples for each model. Input denotes distances on the timestep-modulated input features $I_{t}$ . Output is the last block outputs $O_{t}$ . SEA(Input) , SEA(Output) applies the SEA filter to the input and output features, respectively. Poly(Input) corresponds to the polynomial-fitted input distance which is designed to approximate output differences from input features. SEA-filtered inputs closely track SEA-filtered outputs across timesteps, whereas other inputs show weaker alignment.

Method	Latency (s)	TFLOPs	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$
Original (50 steps)	20.9	2976	–	–	–
Vanilla 25 steps	10.5	1487	15.553	0.409	0.668
Vanilla 15 steps	6.4	892	17.842	0.305	0.740
TeaCache ( $\delta$ =0.3)	11.4	1547	20.762	0.211	0.810
TaylorSeer ( $\mathcal{S}$ =3)	9.8	1191	22.783	0.163	0.828
SeaCache ( $\delta$ =0.3)	9.4	1098	26.285	0.106	0.893

Table 1: Quantitative comparison in FLUX.1-dev [ 29 , 28 ] .

Method ( $\approx$ 50%)	Rank $\downarrow$	Method ( $\approx$ 30%)	Rank $\downarrow$
TeaCache ( $\delta$ =0.3) [ 33 ]	2.01	TeaCache ( $\delta$ =0.6) [ 33 ]	2.07
TaylorSeer ( $\mathcal{S}$ =3) [ 36 ]	2.08	TaylorSeer ( $\mathcal{S}$ =5) [ 36 ]	1.98
SeaCache ( $\delta$ =0.3)	1.91	SeaCache ( $\delta$ =0.6)	1.96

Table 2: Comparison of average rank on CycleReward [ 4 ] .

Method	Latency (s)	TFLOPs	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$
Original (50 steps)	182.6	14038	–	–	–
Vanilla 25 steps	93.7	7019	19.97	0.263	0.731
Vanilla 15 steps	56.8	4211	17.49	0.371	0.662
TeaCache ( $\delta$ =0.12)	98.5	6994	23.40	0.133	0.805
TaylorSeer ( $\mathcal{S}$ =2)	96.9	7299	24.14	0.152	0.820
SeaCache ( $\delta$ =0.19)	90.8	6747	32.39	0.047	0.932

Table 3: Quantitative comparison in HunyuanVideo [ 27 ] .

Method	Latency (s)	TFLOPs	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$
Original (50 steps)	176.3	8214	–	–	–
TeaCache ( $\delta$ =0.09)	86.6	4107	20.84	0.171	0.721
TaylorSeer ( $\mathcal{S}$ =2)	93.1	4189	16.15	0.336	0.543
SeaCache ( $\delta$ =0.2)	83.9	3942	26.60	0.075	0.873
TeaCache ( $\delta$ =0.15)	63.6	2957	18.88	0.245	0.645
TaylorSeer ( $\mathcal{S}$ =3)	67.1	2956	14.18	0.455	0.453

Table 4: Quantitative comparison in Wan2.1 1.3B [ 63 ] .

Figure 6: Qualitative comparison of SeaCache and baselines on FLUX at refresh ratios of approximately $30\%$ and $50\%$ .

Figure 7: Qualitative comparison of text-to-video generative models at refresh ratios of approximately $30\%$ and $50\%$ .

Panel of Figure 8 — Figure 8: Ablation on spectrum-aware filtering. Trade-offs for different cache metrics on FLUX and HunyuanVideo . Results are averaged over 200 prompts for FLUX and 20 randomly selected from VBench for HunyuanVideo , with the other settings fixed.

Figure 9: Plug-and-play adaptation to DiCache. PSNR-refresh ratio trade-off on FLUX when applying the SEA-based cache metric to DiCache [ 6 ] . “DiCache+Ours” denotes DiCache combined with our SEA filter, while “DiCache” uses the original metric.

Figure 10: Refresh pattern across timesteps on FLUX . Per-timestep refresh ratio at a $30\%$ budget. (a) SeaCache automatically concentrates refreshes on early timesteps, whereas (b) TeaCache spreads refreshes more uniformly over the trajectory.

Model	SEA Filter (s)	Latency (s)	Overhead (%)
FLUX (2D FFT)	0.058	9.4	0.6
HunyuanVideo (3D FFT)	0.362	90.8	0.4

Table 5: Runtime overhead of SEA filtering per sample, averaged over

10

runs.

Resolution	SEA Filter (s)	Total Latency (s)	Overhead (%)
480p	0.668	161.5	0.41
720p	1.539	561.1	0.27

Table 6: Runtime overhead of SEA filtering per sample on Wan2.1-14B-T2V at different output resolutions.

Models	Subject Consistency	Background Consistency	Temporal Flickering	Motion Smoothness	Dynamic Degree	Aesthetic Quality	Imaging Quality	Object Class
TeaCache ( $\delta$ =0.12)	95.59%	95.99%	99.14%	98.77%	62.50%	60.92%	62.07%	86.31%
TaylorSeer ( $\mathcal{S}$ =2)	95.75%	96.20%	99.09%	98.83%	63.89%	60.93%	62.73%	83.47%
SeaCache ( $\delta$ =0.19)	95.77%	96.28%	99.15%	98.88%	62.50%	60.55%	62.01%	85.28%
TeaCache ( $\delta$ =0.2)	95.57%	96.04%	99.18%	98.76%	62.50%	60.28%	60.28%	86.47%
TaylorSeer ( $\mathcal{S}$ =3)	95.67%	96.18%	99.07%	98.86%	63.89%	60.64%	63.25%	82.20%
SeaCache ( $\delta$ =0.35)	95.78%	96.35%	99.20%	98.92%	61.11%	60.00%	61.02%	82.59%

Table 7: VBench metrics in HunyuanVideo .

Models	Subject Consistency	Background Consistency	Temporal Flickering	Motion Smoothness	Dynamic Degree	Aesthetic Quality	Imaging Quality	Object Class
TeaCache ( $\delta$ =0.09)	95.89%	97.09%	98.30%	97.37%	81.94%	62.48%	67.88%	80.46%
TaylorSeer ( $\mathcal{S}$ =2)	95.78%	96.90%	98.37%	97.47%	88.89%	62.14%	68.08%	82.75%
SeaCache ( $\delta$ =0.2)	95.96%	97.05%	98.20%	97.41%	84.72%	62.31%	68.01%	81.17%
TeaCache ( $\delta$ =0.15)	96.04%	97.02%	98.21%	97.35%	83.33%	62.25%	67.47%	80.22%
TaylorSeer ( $\mathcal{S}$ =3)	95.32%	96.54%	98.21%	97.48%	84.72%	60.85%	67.83%	78.32%
SeaCache ( $\delta$ =0.35)	96.03%	97.00%	98.12%	97.39%	81.94%	61.71%	67.66%	79.75%

Table 8: VBench metrics in Wan2.1 1.3B .

Method ( $\approx 50\%$ )	Rank $\downarrow$	Method ( $\approx 30\%$ )	Rank $\downarrow$
TeaCache ( $\delta$ =0.12)	2.03	TeaCache ( $\delta$ =0.20)	2.16
TaylorSeer ( $\mathcal{S}$ =2)	2.06	TaylorSeer ( $\mathcal{S}$ =3)	2.09
SeaCache ( $\delta$ =0.19)	1.91	SeaCache ( $\delta$ =0.35)	1.75

Table 9: Comparison of avg. rank on VBench in HunyuanVideo .

Method ( $\approx 50\%$ )	Rank $\downarrow$	Method ( $\approx 30\%$ )	Rank $\downarrow$
TeaCache ( $\delta$ =0.09)	2.13	TeaCache ( $\delta$ =0.15)	1.53
TaylorSeer ( $\mathcal{S}$ =2)	1.91	TaylorSeer ( $\mathcal{S}$ =3)	2.34
SeaCache ( $\delta$ =0.30)	1.97	SeaCache ( $\delta$ =0.35)	2.13

Table 10: Comparison of avg. rank on VBench in Wan2.1 1.3B .

HunyuanVideo			Wan2.1 1.3B
Method ( $\approx 50\%$ )	Single-scale score $\uparrow$	Multi-scale score $\uparrow$	Method ( $\approx 50\%$ )	Single-scale score $\uparrow$	Multi-scale score $\uparrow$
TeaCache ( $\delta$ =0.12)	2.72	2.76	TeaCache ( $\delta$ =0.09)	2.97	3.03
TaylorSeer ( $\mathcal{S}$ =2)	2.92	2.95	TaylorSeer ( $\mathcal{S}$ =2)	1.90	1.95
SeaCache ( $\delta$ =0.19)	3.98	3.99	SeaCache ( $\delta$ =0.30)	3.93	3.95
Method ( $\approx 30\%$ )	Single-scale score $\uparrow$	Multi-scale score $\uparrow$	Method ( $\approx 30\%$ )	Single-scale score $\uparrow$	Multi-scale score $\uparrow$
TeaCache ( $\delta$ =0.20)	2.11	2.16	TeaCache ( $\delta$ =0.15)	2.44	2.49

Table 11: CompressedVQA [ 60 ] scores on HunyuanVideo and Wan2.1 1.3B under single-scale and multi-scale settings.

	Method	Refresh Ratio	PSNR	LPIPS	SSIM
Distillation (LightX2V [ 11 ] )	Vanilla	25% (4 steps)	11.444	0.475	0.405
	TeaCache	25% (4 steps)	11.762	0.480	0.420
	SeaCache	25% (4 steps)	11.926	0.465	0.432
Efficient Attn. (Jenga [ 75 ] )	Vanilla	50% (25 steps)	15.154	0.357	0.604
	TeaCache	50% (25 steps)	19.463	0.191	0.744
	SeaCache	48% (24 steps)	24.453	0.097	0.852

Table 12: Compatibility with fast inference works on Wan2.1-T2V: LightX2V [ 11 ] (14B, 16-step) and Jenga [ 75 ] (1.3B, 50-step), evaluated under comparable refresh ratio budgets.

Method	R.R.	PSNR $\uparrow$	LPIPS $\downarrow$	Method	R.R.	PSNR $\uparrow$	LPIPS $\downarrow$
FLUX.1-dev (50-step)
MagCache ( $\delta$ =0.04) [ 43 ]	52%	29.96	0.056	MagCache ( $\delta$ =0.15) [ 43 ]	34%	24.73	0.126
SeaCache ( $\delta$ =0.215)	52%	30.37	0.053	SeaCache ( $\delta$ =0.4)	34%	24.97	0.123
MagCache ( $\delta$ =0.07) [ 43 ]	44%	27.89	0.079	MagCache ( $\delta$ =0.35) [ 43 ]	28%	22.51	0.179
SeaCache ( $\delta$ =0.27)	44%	28.09	0.072	SeaCache ( $\delta$ =0.55)	28%	23.01	0.172
Wan2.1 1.3B T2V (50-step)

Table 13: Comparison with MagCache at matched refresh ratios (R.R.) by tuning the cache threshold

\delta

. We report full-reference quality against the uncached outputs on FLUX.1-dev and Wan2.1 1.3B -T2V. SeaCache shows higher PSNR and lower LPIPS than MagCache [ 43 ] at the same refresh ratio.

Figure 11: Qualitative comparison between MagCache and SeaCache at matched refresh ratios on FLUX.1-dev and Wan2.1 1.3B T2V. At the same refresh ratio, SeaCache better preserves the uncached trajectory while reducing refresh operations.

Figure 12: Additional qualitative comparison of SeaCache and baselines on FLUX at refresh ratios of approximately $30\%$ and $50\%$ .

Figure 13: Additional T2V qualitative comparison of SeaCache and baselines at refresh ratios of approximately $30\%$ and $50\%$ .

Diffusion Model Caching

Diffusion model caching reuses features, activations, or attention states across denoising steps to reduce inference cost while preserving output quality.

See all 112 papers

SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching
Oct 2, 2026Zhendong Mi, Pu Zhao, Ziyu Hu +4Diffusion Model Caching Diffusion Model Inference Acceleration
Stevens Institute of Technology · Northeastern University · Technische Universität Darmstadt
EpaCache: Error-Propagation-Aware Caching for Accelerating Diffusion-Based Visual Generation
Aug 29, 2026Yuhan Liu, Zongwei Hong, Jinglun Li +3Diffusion Model Caching Diffusion Model Inference Acceleration
JIIOV Technology
OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference
Jul 31, 2026Zhikang Xie, Xichen Ye, Yifan Wu +5Diffusion Model Caching Diffusion Model Inference Acceleration
College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · 3Shanghai Key Laboratory of Intelligent Information Processing

Oct 2, 2026J/K move · Enter open · S save

SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching

Zhendong Mi, Pu Zhao, Ziyu Hu, Xiaodong Yu+3

Stevens Institute of Technology · Northeastern University · Technische Universität Darmstadt

Diffusion Model Caching Diffusion Model Inference Acceleration World Model Learning

Open paper AskPDF arXiv

Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Transformer evaluations during denoising. Existing caching methods mainly exploit temporal redundancy at the feature or token level, leaving the underlying mathematical structure of diffusion features largely unexplored. In this work, we reveal that world-model features exhibit highly stable singular subspaces across nearby denoising steps, while their singular values follow predictable evolution patterns. Building on this observation, we propose SpectralCache, a training-free spectral caching framework that reuses stable singular subspaces and estimates only low-dimensional singular values through linear extrapolation. We further exploit the spectral consistency between neighboring full-computation features to skip selected expensive backbone evaluations via singular value scaling. Extensive experiments on representative world models demonstrate that SpectralCache consistently improves inference efficiency while preserving generation quality. On HunyuanWorld-Voyager-13B, SpectralCache achieves 5.22x acceleration while maintaining a WorldScore of 65.90 for static scenes, substantially outperforming existing training-free caching methods in inference efficiency.

Abstract

Diffusion Model Caching

Explore similar work

SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching

EpaCache: Error-Propagation-Aware Caching for Accelerating Diffusion-Based Visual Generation

OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference