PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

Xiaofeng Mao; Shaohao Rui; Kaining Ying; Bo Zheng; Chuanhao Li; Mingmin Chi; Kaipeng Zhang

cs.CVMar 26, 2026

PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

Authors: Xiaofeng Mao, Shaohao Rui, Kaining Ying, Bo Zheng, Chuanhao Li, Mingmin Chi, Kaipeng Zhang

Organizations: Alaya Studio, Shanda AI Research Tokyo · Fudan University · Shanghai Innovation Institute

Abstract

Autoregressive video diffusion models have demonstrated remarkable progress, yet they remain bottlenecked by intractable linear KV-cache growth, temporal repetition, and compounding errors during long-video generation. To address these challenges, we present PackForcing, a unified framework that efficiently manages the generation history through a novel three-partition KV-cache strategy. Specifically, we categorize the historical context into three distinct types: (1) Sink tokens, which preserve early anchor frames at full resolution to maintain global semantics; (2) Mid tokens, which achieve a massive spatiotemporal compression (32x token reduction) via a dual-branch network fusing progressive 3D convolutions with low-resolution VAE re-encoding; and (3) Recent tokens, kept at full resolution to ensure local temporal coherence. To strictly bound the memory footprint without sacrificing quality, we introduce a dynamic top- $k$ context selection mechanism for the mid tokens, coupled with a continuous Temporal RoPE Adjustment that seamlessly re-aligns position gaps caused by dropped tokens with negligible overhead. Empowered by this principled hierarchical context compression, PackForcing can generate coherent 2-minute, 832x480 videos at 16 FPS on a single H200 GPU. It achieves a bounded KV cache of just 4 GB and enables a remarkable 24x temporal extrapolation (5s to 120s), operating effectively either zero-shot or trained on merely 5-second clips. Extensive results on VBench demonstrate state-of-the-art temporal consistency (26.07) and dynamic degree (56.25), proving that short-video supervision is sufficient for high-quality, long-video synthesis. https://github.com/ShandaAI/PackForcing

Figures & tables

Figure 1: Our framework enables the generation of high-quality, temporally coherent videos up to 120 seconds.

Figure 2: Overview of PackForcing. (a) The three-partition KV management organizes the denoising context into sink tokens (full resolution), mid tokens (compressed and dynamically selected), and recent / current tokens (full resolution with full-to-reduced exchange). (b) & (d) The dual-branch compression module fuses a High-Resolution (HR) branch (a progressive 4-stage 3D CNN) with a Low-Resolution (LR) branch (pooling followed by VAE re-encoding) via element-wise addition to minimize token count. (c) To efficiently bound the memory footprint, dynamic context selection is applied to retain only the top-

K

informative mid tokens. To resolve the ensuing position gaps caused by dropped tokens, a Temporal RoPE Adjustment is utilized to re-align and enforce continuous positional indices across the historical contexts.

Figure 3: Attention patterns in causal video generation. (a) Density map shows attention distributed across the entire history. (b) Importance scores $\phi_{j}=\sum_{i}\mathbf{q}_{i}^{\top}\mathbf{k}_{j}/\sqrt{d}$ reveal sparse information. (c) Near-flat late-stage importance (mean $=$ $0.499$ ) rules out FIFO eviction. (d) High Jaccard distance ( $0.75$ ) and position diversity ( $>0.85$ ) show rapid, diverse block retrieval, motivating compression.

Method		Dyn. Deg. $\uparrow$	Mot. Smth. $\uparrow$	Over. Cons. $\uparrow$	Img. Qual. $\uparrow$	Aest. Qual. $\uparrow$	Subj. Cons. $\uparrow$	Back. Cons. $\uparrow$
60 s generation
CausVid Yin et al. (2025)	48.43	98.04	23.36	65.69	60.63	84.53	89.84
LongLive Yang et al. (2025)	44.53	98.70	25.73	69.06	63.30	92.00	92.97
Self-Forcing Huang et al. (2025a)	35.93	98.26	24.92	66.62	57.15	80.41	86.95
Rolling Forcing Liu et al. (2025)	33.59	98.70	25.73	71.06	61.43	91.62	93.00
Deep Forcing Yi et al. (2025)	53.67	98.56	21.75	67.75	58.88	92.55	93.80

Table 1: Quantitative comparison on 60 s and 120 s benchmarks (7 VBench Huang et al. (2024) metrics). Best results are highlighted in bold .

Method	0–10 s	10–20 s	20–30 s	30–40 s	40–50 s	50–60 s	Overall
CausVid Yin et al. (2025)	32.65	31.78	31.47	31.13	30.81	30.79	31.44
LongLive Yang et al. (2025)	33.95	33.38	33.14	33.51	33.45	33.36	33.46
Self-Forcing Huang et al. (2025a)	33.89	33.23	31.66	29.99	28.37	27.12	30.71
Rolling Forcing Liu et al. (2025)	33.85	33.39	32.94	32.78	32.49	32.25	32.95
Deep Forcing Yi et al. (2025)	33.47	33.29	32.38	32.28	32.26	32.27	32.33
PackForcing (ours)	34.04	33.99	33.70	33.37	33.24	32.90	33.54

Table 2: CLIP Score comparison on long video generation (60s).

Figure 4: Qualitative comparison of 120 s generation. Sampled frames across seven timestamps under the same prompt. PackForcing consistently maintains strict subject identity and high visual fidelity throughout the entire sequence. In contrast, baseline methods suffer from progressive degradation over time: Self-Forcing exhibits color shifts by 60 s and eventual collapse, CausVid loses background details, and LongLive generates severely restricted motion. Furthermore, Rolling-Forcing struggles with significant subject inconsistencies, while DeepForcing loses the main subject at 100s.

Sink Size			Overall CLIP $\big\uparrow$	Dyn. Deg. $\big\uparrow$	Mot. Smth. $\big\uparrow$	Over. Cons. $\big\uparrow$	Img. Qual. $\big\uparrow$	Aest. Qual. $\big\uparrow$	Subj. Cons. $\big\uparrow$	Back. Cons. $\big\uparrow$
0	31.24	20.31	98.89	23.11	72.87	59.37	74.72	86.37
2	34.85	49.69	98.57	26.78	72.59	65.85	91.37	93.68
4	34.85	50.16	98.57	26.99	72.59	65.84	91.37	93.73
8	35.09	49.84	98.68	26.29	73.18	66.46	93.11	94.53
16	35.39	35.16	98.82	26.71	73.05	66.34	93.84	94.92

Table 3: Quantitative ablation results of sink size. We consolidate overall CLIP scores and VBench metrics across varying sink sizes. Setting

N_{\text{sink}}{=}8

achieves the optimal balance between dynamic motion and semantic consistency.

Figure 5: Qualitative ablation results on 60 s generation. Removing sink tokens leads to progressive semantic drift, whereas disabling either the RoPE adjustment or dynamic context selection introduces severe frame reset artifacts.

Table 9

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	KV Compress	Bounded Memory	RoPE Adjustment	Long-range Memory
CausVid Yin et al. (2025)	✗	✗	—	✗
Self-Forcing Huang et al. (2025a)	✗	✗	—	✗
Rolling-F. Liu et al. (2025)	✗	✓	✓	✗
LongLive Yang et al. (2025)	✗	✓	N/A	partial
DeepForcing Yi et al. (2025)	✗	✓	✓	partial
PackForcing (Ours)	✓	✓	✓	✓

Appendix

Table 6: Design comparison of causal video generation methods. PackForcing uniquely integrates learned compression, bounded memory, and RoPE adjustment to ensure persistent long-range memory during extended generation.

Sink Size		0–20s	20–40s	40–60s	60–80s	80–100s	100–120s	Overall
0	34.72	33.24	31.51	30.44	29.01	28.51	31.24
2	35.32	35.45	34.64	34.24	34.63	34.80	34.85
4	35.33	35.45	34.64	34.24	34.63	34.79	34.85
8	35.59	35.16	35.04	35.14	34.81	34.81	35.09
16	35.54	35.34	35.21	35.42	35.40	35.46	35.39

Appendix

Table 7: Detailed temporal CLIP Score breakdown. Evaluated across 20-second intervals for different sink sizes.

N_{\text{sink}}\geq 8

demonstrates robust prevention of temporal degradation.

RoPE Corr.	CLIP (0–20 s) $\uparrow$	CLIP (40–60 s) $\uparrow$	$\Delta\,\downarrow$
✗	33.95	31.42	2.53
✓	34.02	33.07	0.95

Appendix

Table 8: Ablation on RoPE Corr.

Method	KV Cache	Peak GPU	Speed
Full cache	${\sim}138$ GB	OOM	—
Window-only	3.1 GB	24 GB	18 FPS
PackForcing (FIFO)	4.0 GB	26 GB	16 FPS
PackForcing (Part.)	4.2 GB	27 GB	15 FPS

Appendix

Table 9: Memory and speed for 120 s generation (

832{\times}480

, 16 FPS, single A100-80GB).

Long Video Generation

Long video generation creates extended, coherent video sequences while managing memory, motion, identity, and narrative continuity.

See all 65 papers

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation
Mar 22, 2026Zengqun Zhao, Yanzuo Lu, Ziquan Liu +3Long-Horizon Video Generation Autoregressive Video Generation
Queen Mary University of London · Imperial College London · Huawei R&D UK
Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity
May 14, 2026Jiahao Tian, Yiwei Wang, Gang Yu +1Diffusion Transformer Autoregressive Diffusion
AGI Lab, Westlake University · University of California at Merced · StepFun
In-Distribution Forcing for Long Video Generation at Test Time
Oct 2, 2026Jeongwoo Shin, Youngyoon Choi, Sangwoo Jo +5Long Video Generation Video Diffusion Models
Seoul National University · Korea University · Sungkyunkwan University +1

Mar 22, 2026J/K move · Enter open · S save

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

Zengqun Zhao, Yanzuo Lu, Ziquan Liu, Jifei Song+2

Queen Mary University of London · Imperial College London · Huawei R&D UK

Long-Horizon Video Generation Autoregressive Video Generation Memory-Augmented Video Generation Temporal Consistency in Video Generation KV-Cache Management

Open paper AskPDF arXiv

Autoregressive video diffusion has recently emerged as a promising paradigm for long-video generation, enabling causal synthesis beyond the temporal limits of bidirectional models. Existing forcing-based training strategies reduce exposure bias by conditioning models on their own predictions during rollout, yet minute-scale generation remains challenging due to progressive temporal degradation and constrained motion evolution. In this work, we study the role of temporal KV memory during long-horizon autoregressive inference. Our analysis shows that simply retaining more historical frames does not consistently improve generation quality; instead, both the quantity and temporal placement of memory strongly affect motion dynamics. These findings suggest that temporal memory should be treated as structured context rather than a homogeneous chronological buffer. Motivated by this observation, we introduce Relax Forcing, a training-free memory mechanism for autoregressive video diffusion. Relax Forcing decomposes temporal context into three functional components: Sink frames that provide global stability, Tail frames that preserve short-term continuity, and dynamically selected History frames that supply mid-range motion structure. History frames are selected using a relaxation-based criterion that encourages alignment with global anchors while suppressing redundancy with recent context. This role-aware sparse memory design mitigates error accumulation during long-horizon rollout while preserving motion evolution and reducing attention overhead. Experiments on VBench-Long show that Relax Forcing improves long-video generation, achieving stronger motion dynamics and higher overall scores than existing autoregressive baselines. These results indicate that structured temporal memory is an effective and complementary direction for scalable long-video generation.

Abstract

Long Video Generation

Explore similar work

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity

In-Distribution Forcing for Long Video Generation at Test Time