Carnator: Fast Text-to-Video Generation with Generation-Native Compatibility-Guided Cross-Request Reuse

Xingkun Yin; Xuebin Tang; Mingkun Xu; Hongyang Du

cs.AISep 26, 2026

Carnator: Fast Text-to-Video Generation with Generation-Native Compatibility-Guided Cross-Request Reuse

Authors: Xingkun Yin, Xuebin Tang, Mingkun Xu, Hongyang Du

Organizations: Department of Electrical and Computer Engineering, University of Hong Kong, Hong Kong SAR, China · Guangdong Institute of Intelligence Science and Technology, Zhuhai, China

Abstract

Video diffusion transformers produce high-quality videos, yet iterative denoising incurs substantial inference latency, limiting interactive and large-scale serving. Most existing acceleration methods focus on individual requests, thereby restricting efficiency gains to redundancy within a single generation trajectory. Recent cross-request reuse offers an additional source of savings, but existing approaches often infer reusability from coarse semantic similarity. This conflates semantic relatedness with generation-level computational compatibility, so aggressive reuse may accept incompatible historical computation while conservative reuse leaves substantial acceleration unrealized. We present Carnator, a cross-request acceleration framework that addresses this challenge by extracting and using generation-native compatibility evidence directly from the model's evolving internal states. Specifically, Carnator performs a lightweight early probe to construct an Early Signature from internal diffusion states, assessing reuse validity through risk-aware compatibility decisions. The same evidence characterizes reuse scope by localizing target-specific computation and guiding joint reuse of historical latent trajectories and sparse attention connectivity. Across three text-to-video backbones, Carnator consistently achieves higher cache-hit end-to-end acceleration than the evaluated cross-request baselines despite more selective cache acceptance, reaching up to 2.17 $\times$ speedup while maintaining competitive generation quality.

Figures & tables

Figure 1: Failure of semantic-similarity-based reuse. A semantically similar historical request yields competitive CLIP-T and VBench-Q scores, yet naive reuse preserves the incompatible non-snowy background and violates the target request. Carnator detects the incompatibility and falls back to dense generation.

Figure 2: Overview of Carnator . Historical requests are first retrieved through text-level recall and filtered using generation-native Early Signatures. For an accepted cache, Carnator jointly reuses historical latent trajectories and sparse attention connectivity while preserving target-specific computation.

Figure 3: Early generation states provide informative signals for cross-request compatibility. Similar request pairs exhibit stronger alignment in early denoising representations, while easy and hard negative pairs show increasingly distinct prompt-conditioned residuals and latent dynamics.

Figure 4: Joint latent-attention reuse in Carnator .

Method	CLIP-T $\uparrow$	VBench-Q $\uparrow$	DiT Hit Latency (s) $\downarrow$	E2E Hit Latency (s) $\downarrow$	DiT Speedup (Hit) $\uparrow$	E2E Speedup (Hit) $\uparrow$	E2E Speedup $\uparrow$	Hit Rate
Wan2.2-TI2V-5B
Dense	0.3372	0.6331	265.23	293.88	1.00 $\times$	1.00 $\times$	1.00 $\times$	N/A
NIRVANA-VID	0.2955	0.6457	204.21	256.09	1.30 $\times$	1.15 $\times$	1.12 $\times$	80.67%
Chorus	0.3005	0.6392	175.48	264.47	1.51 $\times$	1.11 $\times$	1.09 $\times$	80.67%
Chorus II	0.2986	0.6400	201.58	222.44	1.32 $\times$	1.32 $\times$	1.24 $\times$	80.67%
CARNATOR	0.2913	0.6382	159.65	187.30	1.66 $\times$	1.57 $\times$	1.36 $\times$	72.67%

Table 1: End-to-end performance on the VidProM workload across three video diffusion models. Within each model, all methods process the same request sequence under the same generation configuration. For cross-request methods, cache misses fall back to dense generation.

Retrieval / Gate	Hit Rate	CLIP-T $\uparrow$	VBench-Q $\uparrow$	E2E Hit Latency (s) $\downarrow$	Speedup $\uparrow$
Text-Level Retrieval	77.33%	0.2725	0.6301	185.6	1.58 $\times$
+ Early Signature	72.67%	0.2913	0.6382	187.3	1.57 $\times$

Table 2: Ablation of reuse-validity assessment on the VidProM workload. Speedup is calculated with respect to dense generation.

Variant	Latency (s) $\downarrow$	Speedup $\uparrow$	CLIP-T $\uparrow$	VBench-Q $\uparrow$
Full Computation	266.26	1.00 $\times$	0.3391	0.6432
Latent Reuse Only	206.06	1.29 $\times$	0.3237	0.6436
Attention Reuse Only	160.91	1.65 $\times$	0.3291	0.6291
Joint Latent-Attention Reuse	155.85	1.71 $\times$	0.3281	0.6433

Table 3: Ablation of latent and attention reuse in CARNATOR .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Latent-state discrepancy for similar, easy-negative, and hard-negative request pairs across video frames.

Figure 6: Blockwise attention mass and sparse connectivity for two related text-to-video requests. Similar requests preserve substantial structural similarity in their attention patterns despite changes in object identity.

Figure 7: Qualitative result of Carnator

Figure 8: Heat Map of Evidence Region $\mathcal{R}_{\mathrm{evidence}}$ under the first ten denoising steps.

Figure 9: Qualitative comparison of CHAI across video diffusion architectures. From left to right, we show the OpenSora 1.2 baseline, CHAI on OpenSora 1.2, the Wan2.2 baseline, and a direct adaptation of CHAI to Wan2.2. CHAI preserves generation quality on its native OpenSora architecture, while the direct Wan2.2 adaptation exhibits noticeable visual degradation.

Region-update layers	Window	Latency (s) $\downarrow$	Speedup $\uparrow$	CLIP-T $\uparrow$	VBench-Q $\uparrow$
None (dense)	—	273.61	1.00 $\times$	0.3330	0.6699
5	13–17	160.67	1.70 $\times$	0.2681	0.6447
11	10–20	156.67	1.75 $\times$	0.2676	0.6415
15	8–22	150.58	1.82 $\times$	0.2700	0.6440
20	6–25	150.97	1.81 $\times$	0.2803	0.6497
25	3–27	148.97	1.84 $\times$	0.2749	0.6268

Appendix

Table 4: Layer-window ablation on Wan2.2-TI2V-5B. All reuse variants use the same accepted cache. Latency includes the dense probe, and speedup is relative to dense inference.

$\bm{\tau}^{\star}$	Coverage	Accepted $n$	Unsafe $u$	Empirical Risk	$U_{\alpha}(u,n)$
$(1.62,\,2.17,\,1.00,\,7.58)$	71.43%	50	0	0.00%	4.50%

Appendix

Table 5: Risk-aware threshold selection and certification results. The target unsafe-reuse rate is

\delta=5\%

at a one-sided confidence level of

90\%

Diffusion Model Inference Acceleration

Diffusion model inference acceleration reduces the time and compute needed to generate or reconstruct outputs while preserving quality.

See all 485 papers

CHAI: CacHe Attention Inference for text2video
Feb 18, 2026Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta +1T2V Generation Efficient Video Generation
Georgia Institute of Technology.
Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generation
Jun 23, 2026Hao Liu, Chenghuan Huang, Hao Liu +6Image-to-Video Generation Video Diffusion Models
Sun Yat-sen University · WeChat HPC, Tencent Inc. · WeChat Vision, Tencent Inc.
MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration
Oct 7, 2026Yuxiang Xiong, Ruiyan Wang, Wenqiang Wang +5Diffusion Model Caching Diffusion Model Inference Acceleration
Shanghai Jiao Tong University · Alibaba Cloud Computing · Alibaba Token Hub, Alibaba Group

Feb 18, 2026J/K move · Enter open · S save

CHAI: CacHe Attention Inference for text2video

Joel Mathew Cherian, Ashutosh Muralidhara Bharadwaj, Vima Gupta, Anand Padmanabha Iyer

Georgia Institute of Technology.

T2V Generation Efficient Video Generation Diffusion Model Inference Acceleration Video Diffusion Models Diffusion Model Caching

Open paper AskPDF arXiv

Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality.

Abstract

Diffusion Model Inference Acceleration

Explore similar work

CHAI: CacHe Attention Inference for text2video

Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generation

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration