KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Study

Suraj Ranganath; Vaishak Menon; Anish Patnaik

cs.LGMar 29, 2026

KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Study

Authors: Suraj Ranganath, Vaishak Menon, Anish Patnaik

Organizations: University of California, San Diego

Abstract

Self-forcing video generation extends a short-horizon video model to longer rollouts by repeatedly feeding generated content back in as context. This scaling path immediately exposes a systems bottleneck: the key-value (KV) cache grows with rollout length, so longer videos require not only better generation quality but also substantially better memory behavior. We present a comprehensive empirical study of KV-cache compression for self-forcing video generation on a Wan2.1-based Self-Forcing stack. Our study covers 33 quantization and cache-policy variants, 610 prompt-level observations, and 63 benchmark-level summaries across two evaluation settings: MovieGen for single-shot 10-second generation and StoryEval for longer narrative-style stability. We jointly evaluate peak VRAM, runtime, realized compression ratio, VBench imaging quality, BF16-referenced fidelity (SSIM, LPIPS, PSNR), and terminal drift. Three findings are robust. First, the strongest practical operating region is a FlowCache-inspired soft-prune INT4 adaptation, which reaches 5.42-5.49x compression while reducing peak VRAM from 19.28 GB to about 11.7 GB with only modest runtime overhead. Second, the highest-fidelity compressed methods, especially PRQ_INT4 and QUAROT_KV_INT4, are not the best deployment choices because they preserve quality at severe runtime or memory cost. Third, nominal compression alone is not sufficient: several methods shrink KV storage but still exceed BF16 peak VRAM because the current integration reconstructs or retains large BF16 buffers during attention and refresh stages. The result is a benchmark harness, analysis workflow, and empirical map of which KV-cache ideas are practical today and which are promising research directions for better memory integration. Code, data products, and the presentation dashboard are available at https://github.com/suraj-ranganath/kv-quant-longhorizon/.

Figures & tables

Benchmark	Method	Comp.	Peak VRAM	Runtime	Img.	SSIM	Drift
Moviegen	BF16	1.00	19.28	58.6	0.739	1.000	0.739
Moviegen	FlowCache Soft-Prune INT4	5.49	11.71	75.0	0.739	0.544	0.738
Moviegen	FlowCache Prune INT4	5.50	11.71	72.2	0.727	0.457	0.726
Moviegen	PRQ INT4	1.60	20.69	160.0	0.739	0.824	0.739
Moviegen	QuaRot KV INT4	3.20	19.98	236.6	0.738	0.724	0.738
Moviegen	RTN INT4 Recent2	2.43	21.37	68.9	0.736	0.732	0.735

Table 2: Representative operating points across both benchmarks. FLOWCACHE_SOFT_PRUNE_INT4 is the strongest practical deployment choice, whereas PRQ_INT4 and QUAROT_KV_INT4 define a high-fidelity but systems-expensive region.

Figure 1: Global systems-quality landscape on MovieGen . FlowCache-inspired prune/soft-prune methods dominate the practical low-VRAM region, PRQ and QuaRot occupy the high-fidelity region, and spatially mixed methods collapse despite plausible motivation.

Figure 2: Global systems-quality landscape on StoryEval . The qualitative structure mirrors MovieGen : the practical winner remains in the FlowCache-inspired soft-prune region, while the highest-fidelity compressed methods sit at much higher runtime or peak-memory cost.

Figure 3: Pareto/frontier analysis on MovieGen . Methods that survive the quality-preserving compression frontier are not the same methods that survive the systems-efficiency frontier, which is why deployment winners and research-quality winners diverge.

Figure 4: Pareto/frontier analysis on StoryEval . The same separation persists under narrative-style rollout stability, reinforcing that the design-space conclusions are not artifacts of one benchmark.

Figure 5: Curated six-method qualitative comparisons for two MovieGen examples and one StoryEval example.

Figure 6: Representative per-prompt traces from available 10-second runs. Left: allocated VRAM over normalized generation time for BF16 and selected quantized methods. Right: BF16-equivalent versus compressed KV footprint for the RTN policy variants. These traces show that some methods genuinely compress the cache but still incur higher measured peaks because the current integration reconstructs or updates large BF16 buffers transiently.

Panel of Figure 7 — Figure 7: Complementary summary views of systems and long-horizon behavior across methods. Left: peak VRAM versus realized compression highlights which methods convert KV savings into actual memory relief. Right: temporal drift comparison highlights which methods preserve rollout stability as context accumulates.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Family	Implementation detail	Runtime / code path
BF16	BF16	No quantizer object; the causal KV cache is stored and read back in native BF16 throughout generation.	No quantized cache-policy hook; acts as the uncompressed reference path. Code: 01_generate.py; causal_model.py
RTN_INT2	RTN	RTNQuantizer with symmetric blockwise quantization of both K and V, block_size=16, 2-bit packed storage plus per-block scales.	Quantize-on-write every step; dequantize the active cache back to dense BF16 before attention. Code: rtn.py; 01_generate.py; causal_model.py
RTN_INT4	RTN	RTNQuantizer with symmetric blockwise quantization of both K and V, block_size=16, 4-bit packed storage plus per-block scales.	Quantize-on-write every step; dequantize the active cache back to dense BF16 before attention. Code: rtn.py; 01_generate.py; causal_model.py
RTN_K2_V4	RTN	RTNQuantizer mixed-precision variant with 2-bit keys and 4-bit values on the same symmetric blockwise path.	Same per-step quantize/dequantize path as standard RTN. Code: rtn.py; 01_generate.py
RTN_INT4 RECENT2	RTN	RTN INT4 quantizer for the old prefix; the newest 2 frame-blocks are carved out and stored separately as BF16 recent_k/recent_v state.	Per-step quantize-on-write with recent_blocks=2; on read, the quantized prefix is dequantized and the BF16 recent tail is copied back in. Code: rtn.py; 01_generate.py; causal_model.py
RTN_INT4 REFRESH	RTN	RTN INT4 quantizer, but used only during the clean-context refresh pass rather than every denoising step.	quantize_cadence=refresh_only; denoising keeps live BF16 cache state, then the timestep-zero context rerun writes quantized cache state. Code: rtn.py; 01_generate.py; causal_inference.py

Appendix

Table 3: Code-level implementation notes for all 33 evaluated methods. Shared defaults across the harness include blockwise KV quantization at the causal cache boundary with block size 16 when a quantizer is active.

Figure 8: MovieGen generation-quality summary from the repository assets. This plot complements the main-paper realism/fidelity discussion by showing the same benchmark slice in a compact static view.

Figure 9: MovieGen memory-footprint summary from the repository assets. This view complements the main-paper peak-VRAM and compression analysis with the dashboard-style static summary used in presentation material.

Figure 10: MovieGen performance summary from the repository assets. We include it separately so runtime and deployment-facing behavior remain legible in print.

Method	Comp.	Peak VRAM	Runtime	Img.	SSIM	LPIPS	PSNR	Drift
FlowCache Prune INT2	7.78	11.11	69.9	0.637	0.467	0.483	15.26	0.633
FlowCache Soft-Prune INT2	6.82	11.71	76.1	0.662	0.482	0.440	15.84	0.658
FlowCache Prune INT4	5.50	11.71	72.2	0.727	0.457	0.412	15.30	0.726
FlowCache Soft-Prune INT4	5.49	11.71	75.0	0.739	0.544	0.297	17.67	0.738
FlowCache Native Soft-Prune INT4	5.49	11.74	63.6	0.726	0.411	0.475	13.26	0.724
RTN INT2	5.33	19.98	87.1	0.567	0.451	0.475	15.04	0.562

Appendix

Table 4: Moviegen benchmark summary for all methods. BF16 is the reference baseline row. Img. denotes imaging quality, and Drift denotes drift-last imaging quality.

Method	Comp.	Peak VRAM	Runtime	Img.	SSIM	LPIPS	PSNR	Drift
FlowCache Prune INT2	7.68	11.14	70.2	0.516	0.492	0.506	14.26	0.516
FlowCache Soft-Prune INT2	6.72	11.76	74.4	0.532	0.497	0.495	14.18	0.536
FlowCache Prune INT4	5.43	11.75	72.4	0.682	0.465	0.490	inf	0.680
FlowCache Soft-Prune INT4	5.42	11.76	75.2	0.680	0.518	0.416	inf	0.679
FlowCache Native Soft-Prune INT4	5.42	11.78	64.2	0.657	0.436	0.523	11.94	0.657
RTN INT2	5.33	19.98	86.1	0.464	0.459	0.526	13.57	0.471

Appendix

Table 5: Storyeval benchmark summary for all methods. BF16 is the reference baseline row. Img. denotes imaging quality, and Drift denotes drift-last imaging quality.

KV-Cache Quantization

Key-value cache quantization compresses transformer attention state to reduce memory use and bandwidth during long-context inference.

See all 101 papers

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation
Oct 11, 2026Jiahui Hu, Weile Mo, Peiliang Cai +5KV-Cache Quantization Video Diffusion Models
Shanghai Jiao Tong University · South China University of Technology · Alibaba Group
Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation
May 13, 2026Jiayu Chen, Junbei Tang, Wenbiao Zhao +6Self-Attention Long-Horizon Video Generation
Peking University · South China University of Technology · Xinjiang University +2
FrameFold: Response-Guided KV Cache Consolidation for Autoregressive Long Video Generation
Oct 10, 2026Muyu Liu, Mingming Liu, Hong Gu +2Autoregressive Video Generation Long Video Generation
ShanghaiTech University · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China

Oct 11, 2026J/K move · Enter open · S save

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Jiahui Hu, Weile Mo, Peiliang Cai, Jiacheng Liu+4

Shanghai Jiao Tong University · South China University of Technology · Alibaba Group

KV-Cache Quantization Video Diffusion Models Autoregressive Video Generation

Open paper AskPDF arXiv

Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to

6.71\times

KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.

Abstract

KV-Cache Quantization

Explore similar work

TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation

Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation

FrameFold: Response-Guided KV Cache Consolidation for Autoregressive Long Video Generation