Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse

Hao Liu; Ye Huang; Chenghuan Huang; Zhenyi Zheng; Jiangsu Du; Ziyang Ma; Jing Lyu; Yutong Lu

cs.CVApr 6, 2026

Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse

Authors: Hao Liu, Ye Huang, Chenghuan Huang, Zhenyi Zheng, Jiangsu Du, Ziyang Ma, Jing Lyu, Yutong Lu

Organizations: Sun Yat-sen University, China · Independent Researcher

Abstract

Video Diffusion Transformer (DiT) models are a dominant approach for high-quality video generation but suffer from high inference cost due to iterative denoising. Existing caching approaches primarily exploit similarity within the diffusion process of a single request to skip redundant denoising steps. In this paper, we introduce Chorus, a caching approach that leverages similarity across requests to accelerate video diffusion model serving. Chorus achieves up to 45% speedup on industrial 4-step distilled models, where prior intra-request caching approaches are ineffective. Particularly, Chorus employs a three-stage caching strategy along the denoising process. Stage 1 performs full reuse of latent features from similar requests. Stage 2 exploits inter-request caching in specific latent regions during intermediate denoising steps. This stage is combined with Token-Guided Attention Amplification to improve semantic alignment between the generated video and the conditional prompts, thereby extending the applicability of full reuse to later denoising steps.

Figures & tables

Figure 1: The video generation illustration of Chorus on a distilled 4-step Wan2.1 model. Chorus reuses cached latent states from a semantically similar request. It significantly accelerates inference while maintaining accurate semantic generation.

Figure 2: The DiT model architecture and inference process.

Figure 3: Illustration of intra- and inter-request caching reuse, with denoising timesteps explicitly expanded.

Figure 5: The illustration of key amplification in TGAA.We dynamically amplify key vectors of condition tokens (e.g., “African”, “wild”) that best capture the semantic difference in cross-attention operator to guide video latents.

Figure 6: Attention map of cross-attention. Cross-attention is more influential in the early denoising stages, while its effect diminishes later, with nearly all attention scores concentrating on the EOS token. We visualize step 2 (early-stage denoising) and step 48 (late-stage denoising).

Figure 7: The illustration of selective region denoising.

Method	CLIP-SCORE↑	vbench-q↑	Latency(s)↓	Hit Latency(s)↓	Speedup↑	Speedup(hit)↑	Hit Rate
Wan2.1-14B-T2V-distilled (81 frames, 480P, 4steps, no cfg)
baseline	31.9970	0.9000	64.15	-	1.00x	1.00x	-
NIRVANA(0)	31.3038	0.8993	56.75	46.37	1.13x	1.38×	42.6%
Chorus(0)	31.5006	0.9005	55.91	44.51	1.15x	1.44×	42.6%
NIRVANA(1k)	31.0635	0.8991	53.79	46.93	1.19x	1.37×	58.9%
Chorus(1k)	31.3081	0.9006	52.09	44.15	1.23x	1.45x	58.9%

Table 1: Quantitative comparison on Wan2.1 text-to-video models. (0)/(1k) denote cold-start and warm-start caches, respectively. Hit Latency represents the average latency of requests with cache hit. The matching threshold (CLIP similarity) for both distilled models and 50-step models are 0.75 and 0.65 respectively. We highlight the best and second best entries.

Method	CLIP-SCORE $\uparrow$	vbench-q $\uparrow$	Latency(s) $\downarrow$	Speedup $\uparrow$
baseline	31.7581	0.8995	38.51	1.000x
Stage-1 only	30.5059	0.8988	28.20	1.365 $\times$
Stage-1 + TGAA (output)	30.5883	0.9014	28.04	1.373 $\times$
Stage-1 + TGAA (key)	30.7958	0.8987	28.20	1.365 $\times$
Stage-1 + TGAA	30.7987	0.9013	28.20	1.365 $\times$

Table 2: Ablation study of TGAA. We fix Stage-1 reuse and toggle TGAA components (output vs. key amplification). Speedup is relative to the no-reuse baseline.

Method	Compute(%) $\downarrow$	CLIP-SCORE $\uparrow$	vbench-q $\uparrow$	Latency(s) $\downarrow$	Speedup $\uparrow$
Stage-1 + TGAA (w/o SRD)	100.00	30.7987	0.9013	28.20	1.37 $\times$
Stage-1 + TGAA + SRD(12,20)	90.91	30.7653	0.9013	27.13	1.42 $\times$
Stage-1 + TGAA + SRD(8,14)	83.80	30.7469	0.9014	26.90	1.43 $\times$
Stage-1 + TGAA + SRD(0,0)	38.42	30.3223	0.9015	24.13	1.60 $\times$

Table 3: Ablation study of SRD . The

(m,n)

controls mask expansion and the token compute fraction. Lower compute generally improves end-to-end speed with a small quality trade-off. Speedup is relative to the no-reuse baseline.

Figure 9: Cache dynamics under cold start. Cache hit rate and average latency as the cache grows from an empty initialization.

Figure 10: Sensitivity of TGAA amplification factors. (a) Over-amplified keys hurt motion. (b) Over-amplified outputs add artifacts. (c) Under-amplification fails to steer semantics.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Results on distilled Wan2.1-T2V-14B

Diffusion Model Caching

Diffusion model caching reuses features, activations, or attention states across denoising steps to reduce inference cost while preserving output quality.

See all 109 papers

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration
Oct 7, 2026Yuxiang Xiong, Ruiyan Wang, Wenqiang Wang +5Diffusion Model Caching Diffusion Model Inference Acceleration
Shanghai Jiao Tong University · Alibaba Cloud Computing · Alibaba Token Hub, Alibaba Group
Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep
Mar 25, 2026Tianyi Liu, Ye Lu, Linfeng Zhang +5Diffusion Model Inference Acceleration Video Diffusion Models
Nanyang Technological University · Shanghai Jiao Tong University · The Hong Kong Polytechnic University
Heuristics are Not Enough: Calibrated and Corrected Caching for Video Diffusion Acceleration
Oct 12, 2026Jiyue Jiang, Zhiyu Huo, Yuhan Chen +3Diffusion Model Caching Diffusion Model Acceleration
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Theory Lab, 2012 Labs, Huawei Technology Co. Ltd

Oct 7, 2026J/K move · Enter open · S save

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration

Yuxiang Xiong, Ruiyan Wang, Wenqiang Wang, Teng Hu+4

Shanghai Jiao Tong University · Alibaba Cloud Computing · Alibaba Token Hub, Alibaba Group

Diffusion Model Caching Diffusion Model Inference Acceleration Video Diffusion Models

Open paper AskPDF arXiv

Diffusion Transformers (DiTs) achieve remarkable performance in video synthesis, but their iterative denoising process suffers from high inference latency. To address this, caching has emerged as an effective acceleration strategy by capitalizing on inter-step redundancy during denoising. Existing dynamic caching methods typically estimate the error that cache reuse would introduce at each denoising step (step error) to guide cache decisions, whereas our concern is how much quality loss cache reuse would cause in the final generated video (terminal error). We show that step error does not directly correspond to terminal error and that latent information helps capture their relationship, thereby informing cache decisions. Moreover, existing threshold-based methods cannot provide precise speedup control, making it difficult to meet practical requirements for user-specified acceleration targets. To address these limitations, we introduce MORCA, a cache scheduling framework trained through offline-to-online reinforcement learning to make latent-aware reuse/recompute decisions under user-specified acceleration targets. Extensive experiments on different video generation models across multiple target acceleration ratios demonstrate that MORCA achieves better generation fidelity than state-of-the-art caching methods under comparable computational budgets. Code is available at https://github.com/x10ngyx/MORCA.

Abstract

Diffusion Model Caching

Explore similar work

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration

Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep

Heuristics are Not Enough: Calibrated and Corrected Caching for Video Diffusion Acceleration