TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Jiahui Hu; Weile Mo; Peiliang Cai; Jiacheng Liu; Jinkui Ren; Xiantao Zhang; Xiaobing Tu; Linfeng Zhang

cs.CVOct 11, 2026

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Authors: Jiahui Hu, Weile Mo, Peiliang Cai, Jiacheng Liu, Jinkui Ren, Xiantao Zhang, Xiaobing Tu, Linfeng Zhang

Organizations: Shanghai Jiao Tong University · South China University of Technology · Alibaba Group

Abstract

Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to $6.71\times$ KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.

Figures & tables

Figure 1: Magnitudes of KV caches and their prediction residuals on Self-Forcing. Top: the original KV cache magnitudes at representative layers and heads. Bottom: the corresponding residual magnitudes after applying TempoKV. Prediction reduces the dynamic range of both keys and values, suppressing outliers across latent frames and channels.

Figure 2: TempoKV Framework. (a) Offline Calibration: a clean BF16 KV trace is collected from 8 calibration prompts, and predictors are fitted on it by linear regression. (b) Online Inference: $\widetilde{K}_{t,s}$ is predicted from the reconstructed K history, and $\widetilde{V}_{t,s}$ from the aligned $\widehat{K}_{t,s}$ together with the reconstructed V history. Only the low-bit residual $R_{t,s}$ is stored.

Figure 3: Asymmetric predictability of video KV cache. (a) Temporal history predicts keys well, while values benefit substantially from the aligned key. (b) This advantage holds across all layers. (c) The gain remains stable over long generation.

Figure 4: Qualitative comparison of long-horizon generation with Self-Forcing. While KIVI, QuaRot, and RTN exhibit progressively stronger visual degradation over time, TempoKV maintains greater temporal consistency, with results remaining close to the BF16 reference.

Method	Compression Ratio	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Background Consistency $\uparrow$	Subject Consistency $\uparrow$	Dynamic Degree $\uparrow$	Imaging Quality $\uparrow$	Aesthetic Quality $\uparrow$	Motion Smoothness $\uparrow$	Visual Quality $\uparrow$
BF16	1.00 $\times$	$\infty$	1.00	0.00	0.00	95.91	95.81	58.25	61.18	53.46	97.08	76.95
INT2 KV Cache, 480p
RTN	5.33 $\times$	10.32	0.42	0.58	143.05	95.76	95.05	15.05	42.05	41.22	96.73	64.31
KIVI	5.33 $\times$	10.57	0.33	0.66	168.80	95.29	95.09	12.43	42.75	43.03	97.43	64.34
QuaRot	5.33 $\times$	9.51	0.38	0.64	174.07	95.83	94.18	39.77	36.35	49.48	95.83	68.57
QVG	6.61 $\times$	16.21	0.70	0.22	14.24	95.99	95.78	54.62	58.58	52.42	97.47	75.81

Table 1: Causal Forcing results on MovieGen at

832\times 480

and 717 frames. Best and second-best results within each precision setting are shown in bold and underline , respectively.

Method	Compression Ratio	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Background Consistency $\uparrow$	Subject Consistency $\uparrow$	Dynamic Degree $\uparrow$	Imaging Quality $\uparrow$	Aesthetic Quality $\uparrow$	Motion Smoothness $\uparrow$	Visual Quality $\uparrow$
BF16	1.00 $\times$	$\infty$	1.00	0.00	0.00	96.36	97.88	42.39	70.39	39.77	98.61	74.23
INT2 KV Cache, 480p
RTN	5.33 $\times$	12.03	0.48	0.57	227.40	95.83	96.71	12.32	43.36	38.15	98.04	64.06
KIVI	5.31 $\times$	14.11	0.53	0.48	150.31	95.82	97.50	12.79	51.63	38.64	98.32	65.78
QuaRot	5.33 $\times$	11.68	0.34	0.60	139.98	89.07	78.92	36.34	62.76	38.57	96.12	66.96
QVG	6.61 $\times$	18.34	0.70	0.18	15.17	95.25	97.05	39.40	66.48	39.69	98.49	72.73

Table 2: Self-Forcing results on MovieGen at

832\times 480

and 717 frames. Best and second-best results within each precision setting are shown in bold and underline , respectively.

Method	Compression Ratio	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Background Consistency $\uparrow$	Subject Consistency $\uparrow$	Dynamic Degree $\uparrow$	Imaging Quality $\uparrow$	Aesthetic Quality $\uparrow$	Motion Smoothness $\uparrow$	Visual Quality $\uparrow$
BF16	1.00 $\times$	$\infty$	1.00	0.00	0.00	95.70	97.77	49.33	72.45	40.09	98.53	75.65
INT2 KV Cache, 480p
RTN	5.33 $\times$	14.21	0.57	0.36	96.48	95.73	97.55	30.20	56.86	38.46	97.60	69.40
KIVI	5.33 $\times$	15.33	0.63	0.34	62.02	94.96	94.39	30.67	64.07	38.84	96.63	69.93
QuaRot	5.33 $\times$	13.35	0.54	0.42	36.48	94.58	95.18	41.00	66.91	39.56	97.20	72.40
QVG	6.47 $\times$	15.75	0.66	0.25	24.29	95.61	97.66	47.33	69.54	39.95	98.52	74.77

Table 3: LongCat-Video-13B results on MovieGen at

832\times 480

and 717 frames. Best and second-best results within each precision setting are shown in bold and underline , respectively.

Figure 5: Quality–efficiency tradeoffs. Visual Quality is plotted against (a) peak VRAM, (b) KV-cache compression ratio, and (c) generation latency. TempoKV remains close to BF16 quality while providing the best balance among compression, runtime memory, and generation latency.

Figure 6: Quality preservation over long video generation. (a) Imaging Quality across increasing video lengths, where TempoKV remains close to the BF16 baseline and consistently outperforms existing quantization methods. (b–c) PSNR and SSIM comparisons further show that TempoKV better preserves quality under aggressive KV compression.

Figure 7: Predictor ablation. (a–b) Reconstruction error of the K and V caches on different effective bits. (c) Per-dimension difference between the cross predictor and the temporal-only predictor.

Table 11

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Predictability of the key and value caches on LongCat-Video-13B, measured by cosine similarity between predicted and original caches. (a) Mean over all layers, with and without the aligned cache as input. (b) Per layer. (c) Per latent frame.

Bits (K/V)	Direction	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
2 / 2	K $\to$ V	19.85	0.7822	0.1303	10.04
2 / 2	V $\to$ K	17.66	0.6394	0.2575	75.73
4 / 2	K $\to$ V	23.17	0.8539	0.0729	5.23
4 / 2	V $\to$ K	17.71	0.6426	0.2562	73.80
5 / 3	K $\to$ V	24.17	0.8815	0.0597	4.13
5 / 3	V $\to$ K	20.74	0.7886	0.1139	17.24

Appendix

Table 6: Cross predictor is directional. Predicting V from the reconstructed K outperforms the reverse at every bit allocation and on every metric. The better direction is shown in bold .

Anchor bits	Comp. $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Subject	Motion	Aesthetic	Imaging
Anchor bits	Comp. $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Consistency $\uparrow$	Smoothness $\uparrow$	Quality $\uparrow$	Quality $\uparrow$
2	6.739 $\times$	16.83	0.6095	0.2742	65.88	96.68	98.28	39.30	62.61
3	6.723 $\times$	19.53	0.7304	0.1471	23.31	96.96	98.43	39.69	67.35
4	6.707 $\times$	19.85	0.7832	0.1303	10.04	97.04	98.48	39.79	67.81
5	6.692 $\times$	19.79	0.7858	0.1243	10.09	97.04	98.49	39.83	68.02
6	6.676 $\times$	19.28	0.7818	0.1244	10.12	97.04	98.50	39.79	67.87

Appendix

Table 7: Ablation on anchor precision. Anchor precisions below 4 bits degrade reconstruction quality, whereas increasing the precision beyond 4 bits provides little additional benefit.

Method	Compression	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Background Consistency $\uparrow$	Subject Consistency $\uparrow$	Dynamic Degree $\uparrow$	Imaging Quality $\uparrow$	Aesthetic Quality $\uparrow$	Motion Smoothness $\uparrow$	Visual Quality $\uparrow$
BF16	1.00 $\times$	$\infty$	1.00	0.00	0.00	95.68	97.71	47.67	74.71	41.60	98.29	76.28
RTN	3.20 $\times$	15.65	0.72	0.17	15.23	95.28	97.61	43.12	74.33	40.59	98.32	74.87
KIVI	3.20 $\times$	13.13	0.61	0.32	59.83	95.60	94.81	26.67	65.02	39.14	96.39	69.61
QuaRot	3.20 $\times$	16.48	0.74	0.15	13.56	95.73	97.72	45.67	74.32	40.58	98.33	75.40
QVG	3.59 $\times$	15.43	0.66	0.23	18.94	95.58	97.51	47.00	74.32	40.60	98.18	75.53
QVG-Pro	2.75 $\times$	15.32	0.68	0.20	19.60	95.82	97.52	46.00	74.57	40.63	98.28	75.47

Appendix

Table 8: Quality comparison on LongCat-Video at 720p under INT4 quantization. Best and second-best results are shown in bold and underline , respectively.

Panel of Figure 9 — Figure 9: Calibration of TempoKV. Left: A small calibration set is sufficient for TempoKV. Reconstruction error drops rapidly with the first few calibration prompts and nearly saturates thereafter on both Self-Forcing and Causal Forcing, with little additional gain beyond a handful of prompts. Right: TempoKV is robust to calibration-set distribution. Switching between action and scene prompts changes reconstruction error by at most $0.77\%$ relative to the matched setting.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Visual Quality $\uparrow$
Causal Forcing, 480p
Forcing-KV	9.32	0.38	0.58	40.07	76.70
TempoKV (k2v2)	17.23	0.76	0.17	9.15	77.04
TempoKV (k4v2)	22.85	0.79	0.13	6.34	77.11
TempoKV (k5v3)	23.21	0.80	0.11	4.03	77.12
Self-Forcing, 480p

Appendix

Table 9: Comparison with token pruning. Across both Causal Forcing and Self-Forcing, TempoKV improves fidelity in PSNR, SSIM, LPIPS, and FID, while also achieving higher overall Visual Quality.

Figure 10: Qualitative comparison on LongCat-Video-13B at INT4. TempoKV maintains superior temporal coherence and visual fidelity at a higher compression ratio.

Figure 11: Qualitative comparison on Causal Forcing at INT2. TempoKV maintains superior temporal coherence and visual fidelity at a higher compression ratio.

KV-Cache Quantization

Key-value cache quantization compresses transformer attention state to reduce memory use and bandwidth during long-context inference.

See all 100 papers

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
May 10, 2026Yicheng Ji, Zhizhou Zhong, Jun Zhang +7Diffusion Model Caching KV-Cache Compression
ZJU · Video Rebirth · HKUST +1
CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches
Oct 7, 2026Albert Ge, Akash Haridas, Utkarsh Saxena +5KV-Cache Compression KV-Cache Quantization
Advanced Micro Devices Inc. (AMD) · University of Wisconsin-Madison
Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation
May 28, 2026Jiayi Luo, Qiyan Liu, Tengyang Wang +8KV Caching Long-Horizon Video Generation
BUAA · ZGCA · PKU +3

May 10, 2026J/K move · Enter open · S save

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models

Yicheng Ji, Zhizhou Zhong, Jun Zhang, Qin Yang+6

ZJU · Video Rebirth · HKUST+1

Diffusion Model Caching KV-Cache Compression Autoregressive Video Generation Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Autoregressive (AR) video diffusion models adopt a streaming generation framework, enabling long-horizon video generation with real-time responsiveness, as exemplified by the Self Forcing training paradigm. However, existing AR video diffusion models still suffer from significant attention complexity and severe memory overhead due to the redundant key-value (KV) caches across historical frames, which limits scalability. In this paper, we tackle this challenge by introducing KV cache compression into autoregressive video diffusion. We observe that attention heads in mainstream AR diffusion models exhibit markedly distinct attention patterns and functional roles that remain stable across samples and denoising steps. Building on our empirical study of head-wise functional specialization, we divide the attention heads into two categories: static heads, which focus on transitions across autoregressive chunks and intra-frame fidelity, and dynamic heads, which govern inter-frame motion and consistency. We then propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads. While maintaining output quality, our method achieves a generation speed of over 29 frames per second on a single NVIDIA H200 GPU along with 30% cache memory reduction, delivering up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution, and further scaling to 2.82x speedup at 1080P resolution. Code and demo videos are provided at https://zju-jiyicheng.github.io/Forcing-KV-Page.

Abstract

KV-Cache Quantization

Explore similar work

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models

CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches

Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation