Relational Feature Caching for Accelerating Diffusion Transformers

Byunggwan Son; Jeimin Jeon; Jeongwoo Choi; Bumsub Ham

cs.CVFeb 23, 2026

Relational Feature Caching for Accelerating Diffusion Transformers

Authors: Byunggwan Son, Jeimin Jeon, Jeongwoo Choi, Bumsub Ham

Organizations: Yonsei University · Korea Institute of Science and Technology (KIST)

Abstract

Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features of computationally expensive modules at certain timesteps, and exploiting them for subsequent steps to reduce redundant computations. Recent forecasting-based caching approaches employ temporal extrapolation techniques to approximate the output features with cached ones. Although effective, relying exclusively on temporal extrapolation still suffers from significant prediction errors, leading to performance degradation. Through a detailed analysis, we find that 1) these errors stem from the irregular magnitude of changes in the output features, and 2) an input feature of a module is strongly correlated with the corresponding output. Based on this, we propose relational feature caching (RFC), a novel framework that leverages the input-output relationship to enhance the accuracy of the feature prediction. Specifically, we introduce relational feature estimation (RFE) to estimate the magnitude of changes in the output features from the inputs, enabling more accurate feature predictions. We also present relational cache scheduling (RCS), which estimates the prediction errors using the input features and performs full computations only when the errors are expected to be substantial. Extensive experiments across various DiT models demonstrate that RFC consistently outperforms prior approaches significantly. Project page is available at https://cvlab.yonsei.ac.kr/projects/RFC

Figures & tables

Panel of Figure 1 — Figure 1: Feature analysis and comparison between existing approaches (FORA ( Selvaraju et al., 2024 ) , TaylorSeer ( Liu et al., 2025b ) ), and our method (RFC) using DiT-XL/2 ( Peebles and Xie, 2023 ) . (a-b) Min-max normalized $L_{2}$ distances of output and input features, measured between consecutive timesteps. While the variations of feature changes are irregular, those of input and output remain closely aligned with each other. (c) The prediction errors across different modules. We measure the relative $L_{1}$ error between output features with and without applying caching methods and average the values over the timesteps. (d) Quantitative results on ImageNet ( Deng et al., 2009 ) evaluated in terms of FLOPs and sFID ( Nash et al., 2021 ) .

Panel of Figure 2 — Figure 2: Empirical analyses in DiT-XL/2 ( Peebles and Xie, 2023 ) . (a) RSD of $s_{k}(t-k)$ with varying $t$ . (b) Relative $L_{1}$ errors of the output and input features, i.e ., $\mathcal{E}_{O}(t)$ and $\mathcal{E}_{I}(t)$ , respectively. Please see the text for details.

Methods	NFC	FLOPs (T) $\downarrow$	Latency (s) $\downarrow$	FID $\downarrow$	sFID $\downarrow$	FID2FC $\downarrow$	sFID2FC $\downarrow$	IS $\uparrow$
Full-Compute	50	23.74	7.61	2.32	4.32	-	-	241.25
FORA ( $N=4$ ) ( Selvaraju et al., 2024 )	14	6.66	2.37	4.33	7.38	1.90	6.32	215.33
ToCa ( $N=4$ ) ( Zou et al., 2025 )	17	8.73	3.64	3.03	5.02	0.86	3.56	229.01
DuCa ( $N=4$ ) ( Zou et al., 2024 )	17	7.66	2.94	3.39	4.95	1.19	4.22	224.03
TaylorSeer ( $N=4,m=2$ ) ( Liu et al., 2025b )	14	6.66	2.84	2.55	5.30	0.44	2.17	232.26
RFC ( $m=2$ )	14.01	6.67	2.86	2.52	4.60	0.30	1.33	231.00

Table 1: Quantitative comparisons on class-conditional image generation for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . Numbers in bold indicate the best performance, while underscored ones are the second best among similar computational costs.

N

is the interval between full computations, and

m

refers to the order of the Taylor expansion.

Methods	NFC	FLOPs (T) $\downarrow$	Latency (s) $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	IR $\uparrow$	CLIP $\uparrow$
Full-Compute	50	$2813.50$	$26.210$	-	-	-	0.9655	17.0720
FORA ( $N=3$ ) ( Selvaraju et al., 2024 )	17	$957.33$	$10.489$	17.4967	0.7236	0.4209	0.9111	16.9515
ToCa ( $N=5$ ) ( Zou et al., 2025 )	14	$1056.26$	$18.591$	18.0333	0.7269	0.3490	0.9457	17.0063
TaylorSeer ( $N=4,m=1$ ) ( Liu et al., 2025b )	14	$788.59$	$8.863$	19.5722	0.7695	0.3226	0.9204	17.0659
TaylorSeer ( $N=4,m=2$ )	14	$788.59$	$8.985$	19.7733	0.7706	0.3175	0.9407	17.0527
RFC ( $m=1$ )	14.02	$789.82$	$9.148$	20.2707	0.7905	0.2934	0.9448	17.0288

Table 2: Quantitative comparisons on text-to-image generation for FLUX.1 dev ( Batifol et al., 2025 ) on DrawBench ( Saharia et al., 2022 ) . Numbers in bold indicate the best performance, while underscored ones are the second best among similar computational costs.

N

is the interval between full computations, and

m

refers to the order of the Taylor expansion.

Methods	NFC	FLOPs (T) $\downarrow$	Latency (s) $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	VBench $\uparrow$
Full-Compute	50	$7520.00$	$263.480$	-	-	-	81.40
FORA ( $N=6$ ) ( Selvaraju et al., 2024 )	9	$1359.19$	$55.339$	15.5221	0.4349	0.2743	78.51
TaylorSeer ( $N=5,m=1$ ) ( Liu et al., 2025b )	10	$1510.21$	$60.051$	16.3606	0.5126	0.1985	80.77
TaylorSeer ( $N=6,m=1$ )	9	$1359.19$	$55.872$	15.5321	0.4606	0.2448	79.52
RFC ( $m=1$ )	8.96	$1354.65$	$57.786$	18.5408	0.6352	0.1329	80.83
FORA ( $N=8$ ) ( Selvaraju et al., 2024 )	7	$1058.45$	$48.340$	14.8878	0.4114	0.2959	77.50

Table 3: Quantitative comparisons on text-to-video generation for HunyuanVideo ( Kong et al., 2024 ) on VBench ( Huang et al., 2024 ) . Numbers in bold indicate the best performance among similar computational costs.

N

is the interval between full computations, and

m

refers to the order of the Taylor expansion.

Figure 3: Qualitative comparisons of (left) class-conditional image generation for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) , and (right) text-to-image generation for FLUX.1 dev ( Batifol et al., 2025 ) on DrawBench ( Saharia et al., 2022 ) .

Figure 4: Qualitative comparisons of text-to-video generation for HunyuanVideo ( Kong et al., 2024 ) on VBench ( Huang et al., 2024 ) . Please see the supplementary material for the actual video clips.

Methods	NFC	FID $\downarrow$	sFID $\downarrow$	FID2FC $\downarrow$	sFID2FC $\downarrow$
TaylorSeer ( Liu et al., 2025b )	14	2.65	5.60	0.57	2.77
RFE	14	2.52	5.18	0.43	2.02
RCS	14	2.52	4.76	0.36	1.88
RFC (RFE + RCS)	14	2.51	4.66	0.31	1.41
TaylorSeer ( Liu et al., 2025b )	11	2.87	5.85	0.73	3.53
RFE	11	2.69	5.22	0.55	2.55

Table 4: Quantitative comparison of different components for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . All results are obtained using the first-order approximation (

m=1

Methods	NFC	FID2FC $\downarrow$	sFID2FC $\downarrow$	Latency (s) $\downarrow$
Linear	14	0.73	3.40	2.85
$w(t)=0.8$	14	0.73	3.36	2.84
$w(t)=1.0$	14	0.57	2.77	2.84
$w(t)=1.2$	14	0.52	2.51	2.84
RFE	14	0.43	2.02	2.86
Linear	11	0.93	4.08	2.52

Table 5: Quantitative comparison of different strategies for estimating the change in output features for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . All results are obtained using the first-order approximation (

m=1

Methods	NFC	FID2FC $\downarrow$	sFID2FC $\downarrow$	Latency (s) $\downarrow$
Input distance	14.03	0.50	2.53	2.62
All modules	14.02	0.37	1.85	2.79
RCS	14.00	0.36	1.88	2.63
Input distance	11.15	0.72	3.47	2.23
All modules	11.00	0.61	3.03	2.41
RCS	11.01	0.62	3.09	2.26

Table 6: Quantitative comparison of different strategies for the cache scheduling using DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . All results are obtained using the first-order approximation (

m=1

Panel of Figure 5 — Figure 5: Feature analyses using DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . (a) A linearity analysis of diffusion features. The solid lines present the linearity between the input and output features, while the dashed lines show the linearity between the timesteps and output features. (b-c) The directional consistency of the input and output features.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Analyses using FLUX.1 dev ( Batifol et al., 2025 ) on DrawBench ( Saharia et al., 2022 ) (upper row) and HunyuanVideo ( Kong et al., 2024 ) on VBench ( Huang et al., 2024 ) (lower row). (a-b) Min-max normalized $L_{2}$ distances of output and input features, measured between consecutive timesteps. (c) RSD of $s_{k}(t-k)$ in Eq. ( 5 ) with varying $t$ . (d) Relative $L_{1}$ errors of the output and input features, i.e ., $\mathcal{E}_{O}(t)$ and $\mathcal{E}_{I}(t)$ in Eq. ( 21 ), respectively.

Methods	NFC	FID $\downarrow$	sFID $\downarrow$	FID2FC $\downarrow$	sFID2FC $\downarrow$
Full-Compute	50	2.32	4.32	-	-
TaylorSeer ( Liu et al., 2025b )	6	13.57	13.19	10.68	15.19
RFC	6	4.31	5.39	1.83	4.80
TaylorSeer ( Liu et al., 2025b )	5	45.81	27.64	42.29	31.46
RFC	5	5.43	6.29	2.81	6.13
TaylorSeer ( Liu et al., 2025b )	4	160.85	135.55	157.53	137.65

Appendix

Table 7: Quantitative comparison under extremely high acceleration ratios for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . All results are obtained using the second-order approximation (

m=2

Methods	NFC	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Full-Compute	6	-	1.0000	0.0000
Step reduction	4	26.5377	0.8855	0.1164
TaylorSeer ( Liu et al., 2025b )	4	29.3435	0.9242	0.0736
RFC	4	32.7436	0.9275	0.0635
Step reduction	3	25.0214	0.8580	0.1511
TaylorSeer ( Liu et al., 2025b )	3	22.2134	0.7491	0.3687

Appendix

Table 8: Quantitative comparison on distilled model, FLUX.1 schnell ( Batifol et al., 2025 ) with 6 denoising steps, on DrawBench ( Saharia et al., 2022 ) . All results are obtained using the first-order approximation (

m=1

Figure 7: Qualitative comparisons of class-conditional image generation for DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) .

Figure 8: Qualitative comparisons of text-to-image generation for FLUX.1 dev ( Batifol et al., 2025 ) on DrawBench ( Saharia et al., 2022 ) .

Figure 9: Qualitative comparisons of text-to-video generation for HunyuanVideo ( Kong et al., 2024 ) on VBench ( Huang et al., 2024 ) .

Model	Order ( $m$ )	$\tau$	NFC
DiT-XL/2 ( Peebles and Xie, 2023 )	2	0.19	14.01
		0.38	11.01
		0.48	10.04
		0.88	8.02
		1.20	7.04
FLUX.1 dev ( Batifol et al., 2025 )	1	0.30	14.02

Appendix

Table 9: Settings of

\tau

for various models.

Figure 10: Trade-off between generation quality and computational cost on ImageNet ( Deng et al., 2009 ) using DiT-XL/2 ( Peebles and Xie, 2023 ) under varying values of $\tau$ . The plot shows the change in performance ( i.e ., sFID2FC) as $\tau$ increases, with the corresponding FLOPs indicated by the size and color of each circle.

Figure 11: Clip scores of text-to-image generation for FLUX.1 dev ( Batifol et al., 2025 ) on DrawBench ( Saharia et al., 2022 ) .

Methods	NFC	FLOPs (T) $\downarrow$	FID $\downarrow$	sFID $\downarrow$	FID2FC $\downarrow$	sFID2FC $\downarrow$	IS $\uparrow$
Full-Compute	50	23.74	2.32	4.32	-	-	241.25
Shallow Skip	16	6.66	3.09	4.65	0.75	2.61	220.00
Deep Skip	16	6.66	3.47	5.15	1.10	2.82	211.54
RFC ( $m=2$ )	14	6.66	2.52	4.60	0.30	1.33	231.00
Shallow Skip	8	3.35	11.70	9.13	9.10	11.71	147.88
Deep Skip	8	3.35	10.20	6.22	6.95	6.59	149.10

Appendix

Table 10: Quantitative results for layer-wise caching strategies on ImageNet ( Deng et al., 2009 ) using DiT-XL/2 ( Peebles and Xie, 2023 ) . We reduce the frequency of full computations by half for the first (shallow skip) or last (deep skip) 7 blocks out of 28. We adjust

\tau

to use more full computations for these variants to match FLOPs. All results are obtained using the second-order approximation (

m=2

Methods	NFC	FID2FC $\downarrow$	sFID2FC $\downarrow$
FORA ( Selvaraju et al., 2024 ) (N=5)	11	78.27	69.84
TaylorSeer ( Liu et al., 2025b ) (N=5)	11	18.83	11.83
RFC	11	8.55	7.59
FORA ( Selvaraju et al., 2024 ) (N=6)	10	103.48	89.17
TaylorSeer ( Liu et al., 2025b ) (N=6)	10	37.81	21.35
RFC	10	11.64	8.25

Appendix

Table 11: Quantitative comparison of FORA ( Selvaraju et al., 2024 ) , TaylorSeer ( Liu et al., 2025b ) , and RFC using the DDIM ( Song et al., 2020 ) model trained on LSUN-bedroom ( Yu et al., 2015 ) with 50 timesteps. We use the first-order approximation (

m=1

Panel of Figure 12 — Figure 12: Analysis of non-uniform intervals scheduled by RCS using DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) . We visualize the average interval between full-compute steps for (a) 14 and (b) 11 NFC settings. $i\to i+1$ at the x-axis denotes the $i$ -th and $(i+1)$ -th full computation, and the y-axis shows the average interval. The results are averaged across 1,000 samples.

Methods	NFC	Order	Peak Memory (MB)	FID2FC $\downarrow$	sFID2FC $\downarrow$
FORA ( Selvaraju et al., 2024 )	10	–	3121	5.32	12.87
TaylorSeer ( Liu et al., 2025b )	10	1	3207	1.42	6.17
RFC	10.04	1	3335	0.58	2.79
TaylorSeer ( Liu et al., 2025b )	10	2	3333	1.05	4.86
RFC	10.02	2	3463	0.54	2.43

Appendix

Table 12: Quantitative results of DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) , in terms of NFC, order in the Taylor expansion, GPU peak memory, FID2FC, and sFID2FC.

Methods	Time (Overhead %)
Baseline (TaylorSeer ( Liu et al., 2025b ) )	2.840s
Input feature computation	+ 0.019s (0.67%)
Input feature prediction	+ 0.002s (0.07%)

Appendix

Table 13: Time overhead of RFC components compared to the baseline (TaylorSeer ( Liu et al., 2025b ) ) using DiT-XL/2 ( Peebles and Xie, 2023 ) on ImageNet ( Deng et al., 2009 ) with 14 NFCs. Percentages in parentheses indicate the time increase relative to the baseline.

Diffusion Model Caching

Diffusion model caching reuses features, activations, or attention states across denoising steps to reduce inference cost while preserving output quality.

See all 112 papers

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers
Mar 5, 2026Guandong LiDiffusion Model Caching Diffusion Model Inference Acceleration
iFLYTEK
BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference
Aug 4, 2026Jinlong Yang, Jinke Wu, Lizilin +1Diffusion Transformer Diffusion Model Caching
Sichuan University Chengdu, China · School of Artificial Intelligence, Sichuan University Chengdu, China
Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models
Apr 29, 2026Zhirong Shen, Rui Huang, Jiacheng Liu +6Diffusion Transformer Diffusion Model Caching
Shanghai Jiao Tong University · University of Electronic Science and Technology of China · Shandong University +2

Mar 5, 2026J/K move · Enter open · S save

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers

Guandong Li

iFLYTEK

Diffusion Model Caching Diffusion Model Inference Acceleration Diffusion Transformer Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation, yet their iterative denoising process incurs substantial computational cost during inference. Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions. In this work, we identify three orthogonal axes of non-uniformity in DiT denoising: (1) temporal -- sensitivity to caching errors varies dramatically across the denoising trajectory; (2) depth -- consecutive caching decisions lead to cascading approximation errors; and (3) feature -- different components of the hidden state exhibit heterogeneous temporal dynamics. Based on these observations, we propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC). On FLUX.1-schnell at 512x512 resolution, SpectralCache achieves 2.46x speedup with LPIPS 0.217 and SSIM 0.727, outperforming TeaCache (2.12x, LPIPS 0.215, SSIM 0.734) by 16% in speed while maintaining comparable quality (LPIPS difference < 1%). Our approach is training-free, plug-and-play, and compatible with existing DiT architectures.

Abstract

Diffusion Model Caching

Explore similar work

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers

BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference

Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models