NanoKV: High-Ratio KV Cache Compression for Autoregressive Video Generation

Jiacheng Lu; Haoyi Zhu; Sipei Yi; Enze Xie; Cheng Zhuo; Yu Li

cs.CVNov 4, 2026

NanoKV: High-Ratio KV Cache Compression for Autoregressive Video Generation

Authors: Jiacheng Lu, Haoyi Zhu, Sipei Yi, Enze Xie, Cheng Zhuo, Yu Li

Organizations: Zhejiang University · NVIDIA

Abstract

Autoregressive video models rely on historical key-value (KV) caches to preserve scene consistency over extended rollouts, making KV storage and repeated attention over history a growing memory and computational bottleneck. Aggressive compression can degrade generation quality, while reconstructing full-dimensional KV leaves attention computation largely unchanged. We introduce NanoKV, a frozen-backbone framework for generation-aware high-ratio KV compression. NanoKV exploits redundancy across tokens and attention heads by storing shared content in cluster centers and encoding joint head-channel residuals into low-bit coefficients. Crucially, NanoKV uses different fitting objectives for Keys and Values according to their distinct roles in attention: Key decoders are optimized to preserve downstream block outputs, whereas Value encoders and decoders are fitted for faithful reconstruction through the deployed quantization path. Building on the same compressed representation, the optional NanoKV-Fast variant reuses the stored coefficients for mixed-dimensional attention, reducing computation without modifying the video backbone. Across three autoregressive video backbones, NanoKV consistently improves the quality-storage trade-off. On HY-WorldPlay, NanoKV achieves $26.09\times$ effective KV compression, nearly twice that of KVTC, while improving PSNR by 1.0 dB. NanoKV-Fast retains $20.53\times$ effective compression and achieves an approximately $1.15\times$ end-to-end speedup over KVTC.

Figures & tables

Figure 1: NanoKV achieves high-ratio historical KV compression while preserving generated video quality across three autoregressive video backbones. Displayed PSNR is measured against the matched Original rollout. NanoKV reaches up to 26.09 $\times$ effective KV compression and reduces measured peak GPU memory by up to 35.9%.

Figure 2: Comparison of NanoKV(-Fast) with representative KV compression and attention-access baselines. (a) VideoMLA modifies KV projections and adapts the model for shared latent-space attention. (b) KVTC preserves the original projections but reconstructs full-dimensional KV before attention. (c) NanoKV preserves the original projections and frozen backbone, using generation-aware, role-specific K/V fitting; optional NanoKV-Fast reuses the coefficients for mixed-dimensional attention. $h_{1},\ldots,h_{H}$ denote attention heads.

Figure 3: Motivation for generation-aware KV compression. (a) Joint head–channel PCA retains the most KV variance at matched

8\times

dimensional compression. (b) Compression-induced output errors accumulate with network depth, while block-output alignment reduces mean relative error by 15.3%. Errors are measured relative to Original; lines and shading show the mean and standard deviation across 16 chunks. Results are on HY-WorldPlay.

Figure 4: Overview of NanoKV for generation-aware high-ratio KV compression. (a) Joint head–channel encoding stores historical KV as coefficients and shared centers, read through full-dimensional (NanoKV) or mixed-dimensional (NanoKV-Fast) decoding and attention. (b) The encoder compresses residuals into INT4 coefficients; the decoder reconstructs them and adds back centers. (c) Key decoders use block-output alignment; Value encoders and decoders use quantization-aware fitting. (d) A fixed policy routes history to full- or low-dimensional attention. The latter reduces Queries and restores output dimensions before LSE merging.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	VBench $\uparrow$	Total KV $\downarrow$ (GiB)	KV CR $\uparrow$	E2E Time / Speedup $\uparrow$
HY-WorldPlay	–	–	–	0.8129	20.96	1.00 $\times$	437.63 s
Quant VideoGen	17.58	0.5379	0.3887	0.7744	3.14	6.69 $\times$	0.86 $\times$
TurboQuant	17.95	0.5459	0.3523	0.7918	4.18	5.01 $\times$	0.90 $\times$
KVTC (32 $\times$ )	18.56	0.5796	0.2900	0.7936	1.55	13.50 $\times$	0.92 $\times$
[][] NanoKV (32 $\times$ )	19.56	0.6152	0.2399	0.8057	0.80	26.09 $\times$	0.96 $\times$
[][] NanoKV-Fast (32 $\times$ )	19.04	0.5931	0.2787	0.7981	1.02	20.53 $\times$	1.06 $\times$

Table 1: Quality, compression, and efficiency comparison of NanoKV and its Fast variant with baseline methods. Parentheses denote nominal coefficient compression ratios.

Configuration	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	VBench $\uparrow$
Joint PCA + INT4	16.38	0.4953	0.4077	0.7753
$+$ Shared centers	18.38	0.5754	0.3047	0.7949
$+$ Value quantization-aware fitting	18.54	0.5839	0.2922	0.8007
[][] $+$ Key block-output alignment (full)	20.20	0.6692	0.1968	0.8171

Table 2: Component analysis on HY-WorldPlay at nominal

16\times

compression.

Figure 6: Preserving quality over continuous generation. On HY-WorldPlay at nominal

32\times

, NanoKV maintains the highest (a) reference PSNR and (b) VBench among evaluated compression methods throughout generation; NanoKV-Fast’s gap grows later in the rollout.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Change	Pixel error	$\Delta$ Time	Peak (GiB)
HY-WorldPlay	post- $\to$ pre-RoPE	–	–	OOM ( $\approx 114$ , est.) $\to$ 71.75
	KV lifecycle optimization	0	$-1.60\%$	71.75 $\to$ 55.92
LingBot-World 2.0	post- $\to$ pre-RoPE	0	$-2.07\%$	59.33 $\to$ 59.33
Matrix-Game 3.5	KV lifecycle optimization	0	$+1.69\%$	62.56 $\to$ 41.38

Appendix

Table 3: Cache implementation: output consistency, runtime, and peak allocated memory.

Figure 7: PyTorch peak allocated GPU memory (GiB) for all methods on the fixed timing input.

Method	Implementation / coding	Grouping and protection	Resident precision
QVG	Official Quant-VideoGen backend; INT2 residuals	Block size 64; 256 centers each for K/V; two K-means iterations; one residual stage; no extra raw history	BF16 centers and scales (A100 adaptation)
TurboQuant	Our implementation of the paper’s algorithms; K: 2-bit + 1-bit residual; V: 3-bit	Per-head 128D transforms; no extra raw history	FP32 transforms/codebooks; BF16 norms
KVTC	Our video port; PCA and DP mixed-bit allocation; packed INT2/INT4/FP8, with zero-bit omissions	Consecutive layer groups; four sink and 128 recent tokens	BF16 PCA bases, scales and shifts; FP32 means

Appendix

Table 4: Evaluated baseline implementations. Protection here denotes additional uncompressed historical tokens, not the model’s common history window.

Model	Budget	Rank	Centers	K learning rate	Inner updates
HY-WorldPlay	$16\times$	512	64	$5\cdot 10^{-6}$	5
HY-WorldPlay	$32\times$	256	64	$5\cdot 10^{-6}$	5
LingBot-World 2.0	$32\times$	640	64	$2.5\cdot 10^{-6}$	1
Matrix-Game 3.5	$32\times$	384	64	Not used	Not used

Appendix

Table 5: Codec and Key-alignment settings. Rank is per layer and per K/V branch; centers are fitted separately for K and V.

Model	Nominal coefficient CR	$d_{h}=32$	$64$	$96$	$128$
HY-WorldPlay	16 $\times$	448	208	32	176
HY-WorldPlay	32 $\times$	440	192	24	208
LingBot-World 2.0	32 $\times$	926	159	304	211
Matrix-Game 3.5	32 $\times$	0	432	0	288

Appendix

Table 6: Head–layer counts at each deployed attention dimension.

Model	History routed to full dimension
HY-WorldPlay	Four recent frames; one anchor per 16 older frames, or per eight on layers 15, 17, 19, 20, 22, 23, 26, 30, 31, 36, 46, 48. Full-layer fallback: 27–35, 40, 45 at $16\times$ ; additionally 39 and 47 at $32\times$ .
LingBot-World 2.0	First three chunks; one sink frame and two recent frames thereafter. Layer 11 uses a full-dimensional prefix for all eight chunks.
Matrix-Game 3.5	Five recent cached latent frames; sensitive head–layer pairs retain 128 dimensions. No additional sink or periodic anchors use full-dimensional reads.

Appendix

Table 7: Full-dimensional routing in the deployed NanoKV-Fast configurations. Layer indices are zero-based; frame counts refer to latent frames.

Low-dimensional maps	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Analytical initialization	17.21	0.5463	0.3451
+ Attention adaptation	18.59	0.5920	0.2837

Appendix

Table 8: NanoKV-Fast low-dimensional attention adaptation on the HY-WorldPlay

32\times

panel. Higher PSNR/SSIM and lower LPIPS are better.

Model	Layers	Heads	Head dimension	Video frames
HY-WorldPlay	54	16	128	253
LingBot-World 2.0	40	40	128	125
Matrix-Game 3.5	30	24	128	169

Appendix

Table 9: Model dimensions and evaluation video lengths.

Figure 8: Persistent KV storage (GiB) for full-dimensional attention configurations, split into cached data and resident operators.

Method	HY-WorldPlay	Matrix-Game 3.5	LingBot-World 2.0
Original	58.70	42.41	60.40
Quant VideoGen	40.90	34.00	46.39
TurboQuant	42.27	33.34	47.16
KVTC (32 $\times$ )	43.80	32.21	47.94
NanoKV (32 $\times$ )	38.56	31.96	44.56
NanoKV-Fast (32 $\times$ )	39.46	32.13	44.89

Appendix

Table 10: NVML-measured peak GPU memory (GiB), complementing the PyTorch allocated peaks in Figure 7 .

Model	Budget	Historical share	History MACs saved $\uparrow$	All visual-attention MACs saved $\uparrow$
HY-WorldPlay	$16\times$	85.19%	39.36%	33.53%
HY-WorldPlay	$32\times$	85.19%	37.93%	32.31%
LingBot-World 2.0	$32\times$	71.43%	34.32%	24.51%
Matrix-Game 3.5	$32\times$	72.00%	20.83%	15.00%

Appendix

Table 11: Denoising visual-attention arithmetic of NanoKV-Fast relative to NanoKV. The overall saving is the product of the historical share and the saving within history.

Figure 9: VBench over generation for (a) HY-WorldPlay at

16\times

and (b) LingBot-World 2.0 at

32\times

Figure 10: Reference fidelity over generation for (a) HY-WorldPlay at

16\times

and (b) LingBot-World 2.0 at

32\times

Figure 11: HY-WorldPlay examples under four camera actions, showing matched first, middle, and final frames. Rows compare Original, quantization baselines, KVTC, NanoKV, and NanoKV-Fast; both nominal compression budgets are included where labeled. Panels (c)–(d) continue on the next page.

Panel of Figure 12 — Figure 12: LingBot-World 2.0 examples under four camera actions. Matched first, middle, and final frames compare Original, QVG, TurboQuant, KVTC, NanoKV, and NanoKV-Fast. Full frames show scene consistency through camera motion and return.

Panel of Figure 13 — Figure 13: Matrix-Game 3.5 examples under four camera actions. Matched first, middle, and final frames compare Original, QVG, TurboQuant, KVTC, NanoKV, and NanoKV-Fast. All rows use the same input image and action sequence within each panel.

KV-Cache Compression

Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.

See all 285 papers

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation
Oct 11, 2026Jiahui Hu, Weile Mo, Peiliang Cai +5KV-Cache Quantization Video Diffusion Models
Shanghai Jiao Tong University · South China University of Technology · Alibaba Group
CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches
Oct 7, 2026Albert Ge, Akash Haridas, Utkarsh Saxena +5KV-Cache Compression KV-Cache Quantization
Advanced Micro Devices Inc. (AMD) · University of Wisconsin-Madison
Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
May 10, 2026Yicheng Ji, Zhizhou Zhong, Jun Zhang +7Diffusion Model Caching KV-Cache Compression
ZJU · Video Rebirth · HKUST +1

Oct 11, 2026J/K move · Enter open · S save

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Jiahui Hu, Weile Mo, Peiliang Cai, Jiacheng Liu+4

Shanghai Jiao Tong University · South China University of Technology · Alibaba Group

KV-Cache Quantization Video Diffusion Models Autoregressive Video Generation

Open paper AskPDF arXiv

Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to

6.71\times

KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.

Abstract

KV-Cache Compression

Explore similar work

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models