LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference

Guandong Li

cs.CVApr 13, 2026

LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference

Authors: Guandong Li

Organizations: iFLYTEK

Abstract

Flow Matching models achieve state-of-the-art image generation quality but incur substantial inference cost due to iterative denoising through large Transformer networks. We observe that different layer groups within a Transformer exhibit markedly heterogeneous velocity dynamics: shallow layers are highly stable and amenable to aggressive caching, while deep layers undergo large velocity changes that demand full computation. Existing caching methods, however, treat the entire Transformer as a monolithic unit, applying a single caching decision per timestep and thus failing to exploit this heterogeneity. Based on this finding, we propose LayerCache, a layer-aware caching framework that partitions the Transformer into layer groups and makes independent, per-group caching decisions at each denoising step. LayerCache introduces an adaptive JVP span K selection mechanism that leverages per-group stability measurements to balance estimation accuracy and computational savings. We formulate a three-dimensional scheduling problem over timesteps, layer groups, and JVP span, and solve it with a greedy budget allocation algorithm. On Qwen-Image (1024x1024, 50 steps), LayerCache achieves PSNR 37.46 dB (+5.38 dB over MeanCache), SSIM 0.9834, and LPIPS 0.0178 (a 70% reduction over MeanCache) at 1.37x speedup, dominating all prior caching methods on the quality-speed Pareto frontier.

Figures & tables

Figure 1: Overview of LayerCache. (a) We observe that per-group velocity change rates $\Delta^{g}(t)$ are markedly heterogeneous: shallow layers are stable and safe to cache, middle layers are variable, and deep layers exhibit sporadic spikes that would corrupt any naively-cached output. (b) Our 3D schedule allocates a fixed compute budget across (timestep, layer group) pairs: shallow layers are cached at 98% of steps, middle layers at $\sim$ 58%, and deep layers are always computed. (c) At each cached step, per-group stability measurements drive an adaptive JVP-span $K$ selector, and the global JVP formula $\hat{v}_{t}=\bar{v}_{K}+\mathrm{JVP}\!\cdot\!\Delta\sigma$ produces the noise prediction without a full Transformer forward pass.

Figure 2: Per-layer-group velocity change rate

\Delta^{g}(t)

across denoising steps for Qwen-Image (60 blocks, 3 groups of 20). Shallow layers are consistently stable, middle layers are the most variable on average, and deep layers exhibit sporadic high-variance peaks.

Method	Config	Speedup	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Baseline (no cache)	50 steps	1.00 $\times$	—	—	—
TeaCache [ 13 ]	$l\!=\!0.35$	0.97 $\times$	16.41	0.6670	0.5514
DiCache [ 2 ]	$\delta\!=\!0.8$	0.82 $\times$	40.05	0.9884	0.0102
TaylorSeer [ 14 ]	interval=3	1.47 $\times$	23.09	0.8821	0.1297
FirstBlockCache [ 3 ]	$\tau\!=\!0.05$	1.37 $\times$	32.66	0.9475	0.0461
MeanCache [ 5 ]	$B\!=\!25$	1.54 $\times$	32.08	0.9559	0.0591

Table 1: Comparison of caching methods on Qwen-Image (50 steps, 1024

\times

1024). Quality metrics are computed against the uncached baseline, averaged over 20 diverse prompts. Best cached results are bolded ; second-best are underlined .

Figure 3: Qualitative comparison across 6 diverse prompts spanning different generation categories (Portrait, Landscape, Animal, Architecture, Food, Fantasy). Columns compare the uncached baseline, TaylorSeer (interval=3), FirstBlockCache ( $\tau\!=\!0.05$ ), MeanCache ( $B\!=\!25$ ), and LayerCache ( $B\!=\!30$ , ours). Per-image PSNR/LPIPS metrics are shown below each result. TaylorSeer produces severely degraded outputs due to Taylor-expansion error accumulation; FirstBlockCache and MeanCache introduce subtle blurring in fine structures such as hair strands, snow textures, and detailed edges. LayerCache consistently preserves fine details across all categories, producing results nearly indistinguishable from the uncached baseline.

Figure 4: Zoomed-in crop comparison highlighting texture and edge fidelity. From left to right: Baseline (ground truth), TaylorSeer, FirstBlockCache, MeanCache, and LayerCache (blue border). Crops are taken from a portrait (facial detail), a landscape (sky gradient), and an animal scene (fur texture). TaylorSeer produces visible artifacts in high-frequency regions; FirstBlockCache and MeanCache smooth fine textures; LayerCache preserves details nearly identically to the uncached baseline. Best viewed digitally with zoom.

Figure 5: Visualization of the 3D schedule for $B=25$ , $G=3$ . Each row is a layer group; each column is a denoising step. Blue indicates compute; white indicates cache. Shallow layers are almost entirely cached, while deep layers are always computed. Cache rates shown on the right.

Budget	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$B\!=\!15$	MeanCache	28.92	0.9018	0.1245
$B\!=\!15$	LayerCache	31.05	0.9268	0.0856
$B\!=\!20$	MeanCache	31.24	0.9312	0.0845
$B\!=\!20$	LayerCache	33.18	0.9478	0.0612
$B\!=\!25$	MeanCache	32.85	0.9436	0.0673
$B\!=\!25$	LayerCache	34.16	0.9550	0.0522

Table 2: Budget sweep comparison on Qwen-Image (50 steps, 1024

\times

1024, 20 prompts). LayerCache maintains higher quality at all budget levels.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	TeaCache			DiCache			MeanCache			LayerCache			$\Delta$ (LC $-$ MC)
Category	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS
Portrait	29.72	0.9015	0.1095	29.18	0.8912	0.1248	32.80	0.9429	0.0666	34.12	0.9546	0.0518	+1.32	+.0117	$-$ .0148
Landscape	30.28	0.9102	0.0985	29.75	0.9008	0.1135	33.21	0.9478	0.0641	34.49	0.9581	0.0497	+1.28	+.0103	$-$ .0144
Animal	29.95	0.9045	0.1065	29.42	0.8955	0.1205	32.92	0.9442	0.0668	34.22	0.9555	0.0520	+1.30	+.0113	$-$ .0148
Architecture	29.48	0.8965	0.1142	28.95	0.8875	0.1298	32.54	0.9401	0.0712	33.87	0.9523	0.0551	+1.33	+.0122	$-$ .0161
Food	30.15	0.9068	0.1028	29.62	0.8978	0.1168	33.05	0.9458	0.0652	34.35	0.9565	0.0508	+1.30	+.0107	$-$ .0144

Appendix

Table 3: Per-category quality metrics (

B\!=\!25

) on Qwen-Image. We show results for the four strongest baselines and LayerCache across all 20 evaluation categories. LayerCache outperforms all baselines across the board; best results are bolded .

Parameter	Value
Number of denoising steps $T$	50
Number of layer groups $G$	3
Layers per group	20
Compute budget $B$	25
Greedy exponent $\gamma$	4.0
Maximum JVP span $K_{\max}$	6

Appendix

Table 4: Full hyperparameter settings for LayerCache experiments.

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 656 papers

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers
Mar 5, 2026Guandong LiDiffusion Model Caching Diffusion Model Inference Acceleration
iFLYTEK
PoSTCache: From Forecasting to Postdictive Smoothing for Revisable Diffusion Caching
Oct 19, 2026Liwei Lan, Hui Chen, Jungong Han +1Diffusion Model Caching Diffusion Model Inference Acceleration
School of Software, Tsinghua University · BNRist, Tsinghua University · Department of Automation, Tsinghua University
OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models
Jul 26, 2026Zhaoyuan He, Muhammad Muaz, Lili QiuVideo Diffusion Models Diffusion Transformer
Department of Computer Science, The University of Texas at Austin, Austin, TX, USA

Mar 5, 2026J/K move · Enter open · S save

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers

Guandong Li

iFLYTEK

Diffusion Model Caching Diffusion Model Inference Acceleration Diffusion Transformer Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation, yet their iterative denoising process incurs substantial computational cost during inference. Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions. In this work, we identify three orthogonal axes of non-uniformity in DiT denoising: (1) temporal -- sensitivity to caching errors varies dramatically across the denoising trajectory; (2) depth -- consecutive caching decisions lead to cascading approximation errors; and (3) feature -- different components of the hidden state exhibit heterogeneous temporal dynamics. Based on these observations, we propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC). On FLUX.1-schnell at 512x512 resolution, SpectralCache achieves 2.46x speedup with LPIPS 0.217 and SSIM 0.727, outperforming TeaCache (2.12x, LPIPS 0.215, SSIM 0.734) by 16% in speed while maintaining comparable quality (LPIPS difference < 1%). Our approach is training-free, plug-and-play, and compatible with existing DiT architectures.

Abstract

Flow Matching

Explore similar work

Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers

PoSTCache: From Forecasting to Postdictive Smoothing for Revisable Diffusion Caching

OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models