DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency

Zeqi Xiao; Qingle Liu; Kaiwen Zhang; Yifan Zhou; Zihan Ding; Xingang Pan

cs.CVSep 30, 2026

DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency

Authors: Zeqi Xiao, Qingle Liu, Kaiwen Zhang, Yifan Zhou, Zihan Ding, Xingang Pan

Organizations: Nanyang Technological University · Tsinghua University · Princeton University

Abstract

Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compression strategies discard history using fixed windows or select tokens through local attention and similarity signals, without directly measuring whether a chunk contributes information beyond the retained context. We introduce DeCoPrune, a training-free method that treats cache compression as a denoising-consistency problem. We find that tokens with larger discrepancies between intermediate clean predictions and final denoised values tend to carry visual evidence less predictable from the retained context. DeCoPrune uses this model-intrinsic signal to retain high-discrepancy tokens in the long-term cache while pruning low-discrepancy tokens. To evaluate information retention, we introduce CMBench, comprising 58 approximately one-minute generated or real-world context episodes and 116 Reappear or Revisit continuation tasks requiring recall of earlier events or objects. Experiments with LingBot World v2 show that DeCoPrune achieves a DINO score of 0.6701 on a 0-1 scale, with an 85.43% reduction in cumulative historical KV token counts and a 4.14-fold continuation-generation speedup over FullKV. Its head-specialized variant reaches 0.6783 at an 86.19% pruning ratio, approaching FullKV's 0.6803 score and exceeding the evaluated compression baselines at similar budgets. These results indicate that denoising consistency can support long-range information retention while reducing autoregressive inference cost. Our project homepage is https://decoprune.github.io. The code is available at https://github.com/DeCoPrune/CMBench, and the benchmark at https://huggingface.co/datasets/Aoraku/CMBench.

Figures & tables

Figure 1: Overview of CMBench and DeCoPrune . (a) A representative CMBench episode showing which historical tokens DeCoPrune retains and reuses during continuation. (b) Consistency–pruning-ratio trade-off across pruning methods. At comparable pruning ratios, our method achieves higher DINO consistency than the evaluated compression baselines.

Figure 2: Denoising consistency and context redundancy. (a) At an intermediate denoising state $t=t^{\prime}$ , a local step-to-final extrapolation (red dashed) can deviate from the final denoised state without context ( $A\!\to\!B$ ). Relevant context concentrates the conditional outcome, aligning the extrapolation more closely with the final state ( $C\!\to\!D$ ). Our policy treats content that is consistently predictable from existing context as redundant, pruning low-discrepancy tokens while retaining high-discrepancy tokens. (b) Curves report the MSE between the $x_{0}$ prediction at each intermediate denoising step and the final $x_{0}$ . The Causal Forcing, LongLive, and Self-Forcing plots compare runs with and without the available context KV: first-frame KV for Causal Forcing and Self-Forcing, and preceding-chunk KV for LongLive. For LingBot World v2, which does not support text-to-video generation, both runs use an initial frame and rotate the camera: “with context” means that the content revealed after rotation appeared earlier in the context, whereas “without context” means that it did not. In all four comparisons, the with-context condition has lower error across the plotted steps, consistent with our criterion.

Figure 3: DeCoPrune overview. A probe and final prediction from the same denoising trajectory yield a token-retention mask. The mask physically compacts historical KVs after the recent-window delay.

Figure 4: Overview of CMBench . Top: a one-minute generated context assembled from six prompted clips; three target events yield three continuation tasks. Bottom: the corresponding continuation prompts and evaluation pipeline. The reference target and its generated counterpart are localized and segmented, then compared using DINO similarity.

Method	DINO $\uparrow$	PR $\uparrow$	FPS $\uparrow$	Speedup $\uparrow$	Temporal	Motion	Aesthetic	Image
Method	DINO $\uparrow$	PR $\uparrow$	FPS $\uparrow$	Speedup $\uparrow$	Flickering $\uparrow$	Smoothness $\uparrow$	Quality $\uparrow$	Quality $\uparrow$
FullKV	0.6803	0.00%	1.568	1.00 $\times$	0.9499	0.9705	0.4633	0.7125
Streaming ( Xu et al., 2026c ; Yang et al., 2025 )	0.4592	94.37%	9.591	6.12 $\times$	0.9474	0.9698	0.4769	0.7103
DummyForcing ( Guo et al., 2026 )	0.4461	98.03%	7.393	4.72 $\times$	0.9425	0.9687	0.4677	0.6933
ForcingKV ( Ji et al., 2026 )	0.5313	80.62%	5.288	3.37 $\times$	0.9437	0.9661	0.4610	0.7018
TempDiff ( Hwang et al., 2024 ; Fu et al., 2025 )	0.6229	86.36%	6.443	4.11 $\times$	0.9425	0.9634	0.4677	0.7161

Table 1: Main comparison on CMBench and VBench . DINO is reported on a 0–1 scale; results are averaged over three random seeds. FPS and speedup over FullKV measure continuation generation only, excluding prefix processing.

Figure 5: Qualitative comparison under the matched settings used in Tables 1 and 3 . We highly recommend viewing the video comparisons on the supplementary webpage to better appreciate temporal consistency and visual detail.

Panel of Figure 6 — Figure 6: Ablation and metric analysis on 13 independent cases. (a) Threshold $\gamma$ is swept from light to dark at each probe-step index; the marked operating point is $s^{*}=2$ ( $\tau^{*}=899$ ), $\gamma=0.10$ . (b) VBench subject/background consistency and CMBench DINO versus PR.

Table 8

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Effect of RoPE re-indexing across context positions. The heatmaps show the distribution of CMBench cases over context-prompt time and DINO similarity for DeCoPrune–HS (top) and FullKV (bottom), without re-indexing (left) and with re-indexing (right). Cell values and color intensity indicate the number of cases. In the separate diagnostic evaluation of Table 3 , re-indexing shifts the score distribution upward for both methods, increasing the mean DINO similarity from 0.6686 to 0.7156 for DeCoPrune–HS and from 0.4385 to 0.6804 for FullKV .

Figure 8: Full-context comparison across autoregressive video backbones. We compare ground-truth reference frames with FullKV generations from LingBot World v2 , Causal Forcing , Self-Forcing , and LongLive on three representative object-retrieval cases that require no camera-control input. Despite retaining the complete context, the three alternative backbones frequently fail to recover the target object, whereas the LingBot World v2 generations more closely reproduce the targets in these examples.

Backbone	Mean DINO $\uparrow$
LingBot World v2 ( Gao et al., 2026 )	0.6803
Causal Forcing ( Zhu et al., 2026 )	0.2627
Self-Forcing ( Huang et al., 2025a )	0.2609
LongLive ( Yang et al., 2025 )	0.2608

Appendix

Table 4: Full-context performance of candidate backbones on CMBench . Mean DINO (0–1) is computed over the same completed subset for all backbones.

Method	PR $\uparrow$	DINO $\uparrow$
FullKV	0.00%	0.6106
DeCoPrune	57.25%	0.5880
Streaming	53.57%	0.5162
ForcingKV	54.17%	0.5302
DummyForcing	55.60%	0.5157

Appendix

Table 5: LongLive continuation with 10-second contexts on 18 cases. DINO is reported on a 0–1 scale.

Method	Synthetic Video	Real Video
FullKV	0.6873	0.6621
DeCoPrune	0.6909	0.6412
TempDiff ( Hwang et al., 2024 ; Fu et al., 2025 )	0.6484	0.5212
ForcingKV ( Ji et al., 2026 )	0.5422	0.3839
Streaming ( Xu et al., 2026c ; Yang et al., 2025 )	0.4592	0.3274
DummyForcing ( Guo et al., 2026 )	0.4549	0.3111

Appendix

Table 6: Evaluation by context-video source. Mean DINO scores (0–1) on synthetic-video and real-video contexts using one random seed.

Explore similar work

May 10, 2026cs.CV

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models

Autoregressive (AR) video diffusion models adopt a streaming generation framework, enabling long-horizon video generation with real-time responsiveness, as exemplified by the Self Forcing training paradigm. However, existing AR video diffusion models still suffer from significant attention complexity and severe memory overhead due to the redundant key-value (KV) caches across historical frames, which limits scalability. In this paper, we tackle this challenge by introducing KV cache compression into autoregressive video diffusion. We observe that attention heads in mainstream AR diffusion models exhibit markedly distinct attention patterns and functional roles that remain stable across samples and denoising steps. Building on our empirical study of head-wise functional specialization, we divide the attention heads into two categories: static heads, which focus on transitions across autoregressive chunks and intra-frame fidelity, and dynamic heads, which govern inter-frame motion and consistency. We then propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads. While maintaining output quality, our method achieves a generation speed of over 29 frames per second on a single NVIDIA H200 GPU along with 30% cache memory reduction, delivering up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution, and further scaling to 2.82x speedup at 1080P resolution. Code and demo videos are provided at https://zju-jiyicheng.github.io/Forcing-KV-Page.

Yicheng Ji, Zhizhou Zhong, Jun Zhang +7

ZJU · Video Rebirth · HKUST +1

Autoregressive Video Diffusion Models Video Diffusion Models Autoregressive Diffusion Key-Value Cache Compression Diffusion Models

Ask PDF

May 18, 2026cs.CV

Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion

Recent advances in autoregressive video diffusion have enabled sequential and streaming video generation. However, long-horizon generation requires increasingly large KV caches, making efficient compression without sacrificing quality challenging. Existing methods mostly select historical frames based on attention scores, but their context decisions remain coarse. When multiple frames are generated in the same chunk, these methods often apply a shared history selection to the whole chunk, score historical frames solely by attention, and assign head-wise budgets either uniformly or by attention-pattern heuristics rather than explicit head-importance estimation. We show that frames within the same generated chunk can depend on distinct historical frames, that the same historical frame can receive different attention scores as its relative temporal distance to the current frames changes, and that masking different heads induces unequal generation degradation. Motivated by these findings, we propose \textbf{Focused Forcing}, a training-free KV selection method that focuses cached history along both generated-frame and head dimensions. For each generated frame, Focused Forcing preserves the most relevant and distinctive historical frames by combining attention scores with diversity scores of historical frames, while assigning larger budgets to heads with higher estimated importance. Across multiple autoregressive generation paradigms, Focused Forcing achieves up to

\textbf{1.48}\times

end-to-end acceleration without training, while \textbf{improving visual quality and text alignment}. \textit{Our code will be released on GitHub.}

Peiliang Cai, Evelyn Zhang, Jiacheng Liu +8

1SJTU · 2SDU · 3HUST +4

Autoregressive Video Diffusion Models Keyframe Selection Key-Value Cache

Ask PDF

Jun 11, 2026cs.CV

TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment

Autoregressive video diffusion models provide a natural formulation for streaming and variable-length video generation by conditioning newly generated frames on previously generated content. However, extending these models to minute-level generation remains challenging: the limited KV-cache budget prevents the model from retaining the full history, while repeatedly conditioning on self-generated frames induces a context distribution shift that accumulates over time, leading to visual artifacts, quality degradation, and temporal drift. In this paper, we propose TetherCache, a training-free and plug-and-play cache management strategy for drift-resistant long video generation. TetherCache organizes the cache into sink, memory, and recent regions, and introduces two complementary mechanisms. First, GRAB (Gated Recall with Attention-Diversity Balancing) selects long-range memory frames using a gated score that combines attention-based relevance with temporal diversity, preserving informative yet diverse historical context under a fixed cache budget. Second, TAME (Trusted Alignment via Memory Editing) lightly edits newly recalled memory tokens by aligning their statistics to a trusted context distribution, reducing the pollution caused by drifted historical features. Built on Self-Forcing, TetherCache consistently improves long-video generation quality on VBench-Long across 30s, 60s, and 240s settings. In particular, for 240s generation, it substantially improves overall and semantic scores while reducing quality drift from 7.84 to 1.33, demonstrating its effectiveness for stable long-horizon autoregressive video diffusion.

Yu Meng, Xiangyang Luo, Letian Li +5

Tsinghua University · D-INFK, ETH Zürich

Autoregressive Video Diffusion Models Autoregressive Video Generation Interactive Video Generation Kv-Cache Management Value Alignment

Ask PDF