Autoregressive video diffusion models maintain a growing attention key-value (KV) cache, making memory a key bottleneck for long video generation. Recently, clustering-based quantization has emerged as an effective method to compress video KV caches by grouping similar key and value vectors then quantizing residual errors. However, when later frames attend to the compressed history, small reconstruction errors can compound as the video grows. Beyond the within-layer similarity exploited by clustering, we find strong dependencies across cache layers in video models. This motivates Coral-KV - Calibrated Offline, Residual-coded, and Adapted Locally - a KV-cache compression framework that uses cache states in each layer to predict keys and values in subsequent layers, then quantizes the remaining prediction residuals. We construct compact block-diagonal-plus-low-rank (block-DPLR) predictors: block-diagonal components model channel relationships within each attention head, while a global low-rank term models dependencies between heads. Coral-KV exposes a flexible fidelity-memory trade-off: it can reuse its predictors across videos or refit lightweight block-diagonal predictors to the current video for higher fidelity. Across extensive evaluations on both HY-WorldPlay and MAGI-1, our reusable configuration improves PSNR over QVG-4 by 0.32 dB and 4.95 dB, respectively, while using 48% and 37% less cache memory. Against QVG-Pro, our video-adapted configuration improves MAGI-1 PSNR by 3.77 dB while using 33.7% less cache memory. Coral-KV also maintains higher reconstruction fidelity over extended generation and reduces temporal discontinuities while retaining comparable generation latency.
Figures & tables
Figure 1: CORAL-KV improves the memory–quality frontier. (a) average PSNR versus total compressed KV storage across 32 prompts per model, using the same storage accounting as in Table 1 ; stars mark our selected reusable codec. (b) Qualitative comparisons of BF16, QVG-1, QVG-Pro, and reusable CORAL-KV on matched frames.
Figure 2: Cross-layer prediction and quantization-boundary effects. (a) Linear maps predict current-layer keys from previous-layer keys and current-layer values from previous-layer values. Adding current-layer keys improves value prediction. The key and augmented value maps explain over 80% of the variance on both models. (b) We compare reconstruction MSE from predictors and centroids on the same keys and values before residual quantization. Bars show relative MSE normalized to the centroid result. Predictors significantly reduce reconstruction MSE compared with centroids. (c) QVG fits video-specific centroids online when a cache block reaches its token limit, whereas CORAL-KV reuses offline-calibrated predictors. The yellow box highlights facial distortion in QVG’s first frame generated from compressed history; visible degradation also appears in the clothing and ski pole.
Figure 3: CORAL-KV reconstruction. At layer ℓ>0 , reconstructed cache states from layer ℓ−1 feed the predictors. The key predictor maps Kℓ−1 to Kℓ ; adding the quantized key residual gives Kℓ . The value predictor maps [Kℓ,Vℓ−1] to Vℓ ; adding the quantized value residual gives Vℓ . Both predictors use a block-diagonal component and a rank- r low-rank component.
Figure 4: Predictor-structure ablation. Block-DPLR outperforms block-diagonal and parameter-matched low-rank predictors. Reusable block-DPLR shares offline predictors across videos. Video-adapted block-DPLR keeps its low-rank factors fixed and refits block-diagonal terms for each filled cache block, improving fidelity at higher memory cost. The dense control is also fitted offline.
Figure 5: (a–b) Median absolute excess kurtosis across layers and two prompts; lower is closer to a unit Gaussian. Hn transforms each non-overlapping group of n residual channels independently. Wider transforms have model-dependent effects beyond H64 . (c) Comparison at one shared scale per 64 values. “Variation” is the standard deviation divided by the mean of the squared magnitudes of the four 16-value subblocks; lower values indicate more uniform magnitudes.
Fidelity
Total memory (GB) ↓
Latency
Method
PSNR ↑
SSIM ↑
LPIPS ↓
1 video
16 videos
Overhead ↓
HY-WorldPlay (285 frames)
BF16
–
–
–
46.71
747.42
0.0%
QVG-1
15.70
0.496
0.513
7.99
127.91
+7.8%
QVG-4
16.66
0.540
0.438
13.36
213.82
+15.3%
QVG-Pro
17.16
0.559
0.408
14.46
231.34
+17.7%
Table 1: Coral-KV improves fidelity with less total cache memory in matched comparisons against existing baselines on both models. HY-WorldPlay memory counts all completed cache blocks in the 285-frame video; its fidelity evaluations retain the most recent 48 latent frames. Shared predictors are counted once at 16 videos. Latency is generation-time overhead relative to BF16; its measurement protocol appears in Appendix F.3 .
Method
Cache ↓
QBD ↓
VBench score ↑
Subject
Background
Imaging
Aesthetic
Motion
Flicker
HY-WorldPlay
BF16
46.71
1.000
89.153
92.821
69.434
61.860
98.208
96.073
QVG-1
7.99
3.346
84.399
91.032
57.982
59.959
98.508
97.263
QVG-4
13.36
1.386
88.436
92.395
66.869
61.932
98.328
96.683
QVG-Pro
14.46
1.158
89.317
92.932
68.040
61.965
98.274
96.440
Table 2: Coral-KV improves boundary stability with less cache memory and broadly similar VBench scores relative to QVG-Pro on both models. Cache memory (GB) is the single-video total from Table 1 . All results use 355 prompts.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Resolution
Frames
Denoising steps
First compression- dependent frame
Attention implementation
HY-WorldPlay
704×1280
285
4
29
AITER
MAGI-1
720×720
192
16
24
native GQA
Appendix
Table 3: Evaluation configurations. Attention implementations are held fixed across methods within each model.
Split
Prompts
Permitted use
Calibration
32
Fit offline predictors and quantizer parameters
Selection
32
Select configurations and report ablations
Evaluation
355
Held-out comparisons and confidence intervals
Appendix
Table 4: Disjoint prompt splits and their permitted uses.
Figure 6: Cross-layer prediction across depth. The full layerwise measurements summarized in Figure 2 (a). Lines and shading show the mean and range over three held-out cache dumps.
Model
State
Predictor
Offline centroids
Online centroids
HY-WorldPlay
Keys
0.0669
0.2522
0.1626
Values
0.2618
0.6403
0.4445
MAGI-1
Keys
0.0080
0.0270
0.0162
Values
0.3806
0.6959
0.5226
Appendix
Table 5: Relative MSE at equal BF16 parameter storage. All arms reconstruct the same held-out KV tokens before residual quantization. Lower is better. Centroid assignments require additional storage.
Figure 7: Boundary stability over long-horizon generation. BF16-normalized QBD at all eight cache-quantization boundaries on the 32-prompt HY-WorldPlay selection set. Lines are prompt means and bands are 95% bootstrap intervals. Lower is better.
Model (prompts)
Variant
Memory (GB)
Coral-KV
DensityKV
Recent BF16
HY-WorldPlay (16)
Reusable
4.05
16.47
11.76
11.52
HY-WorldPlay (16)
Video-adapted
4.52
16.78
11.67
11.75
MAGI-1 (64)
Reusable
0.99
31.45
28.18
27.31
MAGI-1 (64)
Video-adapted
1.15
32.97
28.67
27.38
Appendix
Table 6: Coral-KV has higher BF16-relative PSNR than both token-selection controls at matched cache memory. Mean RGB PSNR (dB; higher is better) on 16 HY-WorldPlay and 64 MAGI-1 paired prompts. Memory is the retained compressed-history budget; each control’s K/V and any selection metadata fit within it. These cohorts are separate from the 355-prompt evaluation in Table 1 .
Total memory (GB)
Method
1 block
5 blocks
9 blocks
QVG-1
0.89
4.44
7.99
QVG-4
1.48
7.42
13.36
QVG-Pro
1.61
8.03
14.46
Coral-KV (reusable)
1.21
4.06
6.90
Coral-KV (video-adapted)
1.53
5.63
9.74
Appendix
Table 7: Reusable Coral-KV reduces the memory cost of retaining full video history. Each block contains eight latent frames, or 7,040 visual tokens; nine blocks cover the 285-frame HY-WorldPlay video. The totals include QVG codebooks or the reusable predictor parameters.
Model
LR rank
LR/DPLR MB
Δ PSNR (dB)
Δ SSIM
WorldPlay
281
501.4/502.5
−0.519[−0.867,−0.179]
−0.0125[−0.0247,−0.0005]
MAGI-1
204
69.1/69.3
−2.778[−3.240,−2.344]
−0.0328[−0.0435,−0.0235]
Appendix
Table 8: Equal-memory structure control on the 32-prompt selection set. Deltas are standalone low-rank minus block-DPLR r128; brackets are paired 95% CIs.
Figure 8: Layer-zero anchor ablation on the 32-prompt selection set. Bars show the paired effect of replacing the INT4 anchor with BF16; positive values favor BF16. Error bars are paired 95% bootstrap confidence intervals over 32 prompts.
Figure 9: EDEN-2 codebook and quantizer selection. (a) The 16 EDEN-2 reconstruction points over representative H64 -transformed residual pairs from MAGI-1. (b–c) On the 32-prompt selection set, crossing H16 / H64 with scalar/EDEN-2 assignment at a fixed 64-value scale group shows the highest PSNR for H64 /EDEN-2 on both models.
Method
Residual codec
PSNR ↑
Storage (GB) ↓
HY-WorldPlay (285 frames)
QVG-1 (online)
Scalar INT2
15.941
8.117
QVG-1 (offline codebooks)
Scalar INT2
15.299
6.657
Coral-KV (reusable)
Scalar INT2
16.162
6.889
QVG-1 (online)
H64 /EDEN-2
16.249
8.470
QVG-1 (offline codebooks)
H64 /EDEN-2
14.910
7.010
Appendix
Table 9: At similar total compressed-state storage, reusable prediction improves PSNR over offline-trained codebooks with either residual quantizer. Online QVG-1 fits codebooks for each cache block; the offline-codebook variant reuses calibration-fitted codebooks. Storage includes anchors, residual codes and scales, assignments, and codebooks or predictors. HY totals nine generated cache blocks, although fidelity generation retains the most recent 48 latent frames; MAGI-1 retains four 12,150-token blocks. Bold marks the highest PSNR within each model and residual quantizer.
Calibration prompts
2
8
16
32
HY-WorldPlay
17.58
18.02
18.59
17.94
MAGI-1
37.22
37.89
38.06
38.39
Appendix
Table 10: Effect of calibration-set size. Rank-128 block-DPLR predictors fitted on 2, 8, 16, or 32 calibration prompts (one video per prompt) are evaluated on the same 32-prompt selection set. PSNR (dB) is from a separate calibration-size experiment; MAGI-1 uses 96 frames.
Figure 10: MAGI-1 fidelity over 192 decoded frames. Across all 355 held-out prompts, (a) prompt-mean PSNR and 95% bootstrap bands are measured against matched BF16 generation for QVG-Pro and the two reported CORAL-KV configurations; (b) shows paired CORAL-minus-QVG-Pro PSNR differences and intervals. Dotted lines mark cache boundaries; the dashed line begins the continuation beyond the standard 96-frame horizon. Curves are unsmoothed.
Figure 11: Long-range reconstruction examples. MAGI-1 (top; frames 95, 143, and 191) and HY-WorldPlay (bottom; frames 47, 110, and 244). Rows compare BF16, QVG-Pro, and video-adapted CORAL-KV; tile labels report full-frame PSNR against BF16.
Figure 12: Long-range reconstruction examples. MAGI-1 (top; frames 95, 143, and 191) and HY-WorldPlay (bottom; frames 47, 110, and 244). Rows compare BF16, QVG-Pro, and video-adapted CORAL-KV; tile labels report full-frame PSNR against BF16.
Figure 13: HY-WorldPlay first-boundary reconstruction. BF16, QVG, and CORAL-KV at frames 28–30. The yellow box marks QVG’s largest local fidelity gap at frame 29; tile labels give PSNR versus BF16.
Figure 14: MAGI-1 first-boundary reconstruction. BF16, QVG, and CORAL-KV at frames 23–25. The yellow box marks QVG’s largest local fidelity gap at frame 24; tile labels give PSNR versus BF16.
Autoregressive (AR) video diffusion models enable streaming generation of long videos but suffer from significant memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to 6.71× KV cache compression with the best overall quality--efficiency tradeoff. Our code will be released on GitHub.
Jiahui Hu, Weile Mo, Peiliang Cai +5
Shanghai Jiao Tong University · South China University of Technology · Alibaba Group
Autoregressive (AR) video diffusion models adopt a streaming generation framework, enabling long-horizon video generation with real-time responsiveness, as exemplified by the Self Forcing training paradigm. However, existing AR video diffusion models still suffer from significant attention complexity and severe memory overhead due to the redundant key-value (KV) caches across historical frames, which limits scalability. In this paper, we tackle this challenge by introducing KV cache compression into autoregressive video diffusion. We observe that attention heads in mainstream AR diffusion models exhibit markedly distinct attention patterns and functional roles that remain stable across samples and denoising steps. Building on our empirical study of head-wise functional specialization, we divide the attention heads into two categories: static heads, which focus on transitions across autoregressive chunks and intra-frame fidelity, and dynamic heads, which govern inter-frame motion and consistency. We then propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads. While maintaining output quality, our method achieves a generation speed of over 29 frames per second on a single NVIDIA H200 GPU along with 30% cache memory reduction, delivering up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution, and further scaling to 2.82x speedup at 1080P resolution. Code and demo videos are provided at https://zju-jiyicheng.github.io/Forcing-KV-Page.
Chunk-wise autoregressive video diffusion models rely on a KV cache of previously generated chunks to avoid redundant computation, but this cache quickly becomes a memory bottleneck as videos grow longer. Methods that quantize the KV cache to low bitwidths reduce memory pressure but degrade video quality. We show that a key driver of this degradation is a systematic bias in attention weights: due to the convexity of the exponential in softmax attention, quantization noise inflates the contribution of cached keys, a phenomenon we call the Jensen bias. This effect causes quantized keys to steal attention mass from the unquantized current chunk. We derive a per-attention-score correction that removes this bias in expectation, computed on the fly from the quantization step sizes of the cached keys and the query norm. Using a second-order Taylor approximation, the additional computational overhead is negligible, and no additional memory is needed alongside the cache. Evaluated on MAGI-1, SkyReels-V2, and HY-WorldPlay at INT2 quantization, our correction recovers most of the quality lost to aggressive quantization, reaching near-BF16 video quality, and can outperform INT4 quantization while using 50% less memory.