Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

Zengqun Zhao; Yanzuo Lu; Ziquan Liu; Jifei Song; Jiankang Deng; Ioannis Patras

cs.CVMar 22, 2026

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

Authors: Zengqun Zhao, Yanzuo Lu, Ziquan Liu, Jifei Song, Jiankang Deng, Ioannis Patras

Organizations: Queen Mary University of London · Imperial College London · Huawei R&D UK

Abstract

Autoregressive video diffusion has recently emerged as a promising paradigm for long-video generation, enabling causal synthesis beyond the temporal limits of bidirectional models. Existing forcing-based training strategies reduce exposure bias by conditioning models on their own predictions during rollout, yet minute-scale generation remains challenging due to progressive temporal degradation and constrained motion evolution. In this work, we study the role of temporal KV memory during long-horizon autoregressive inference. Our analysis shows that simply retaining more historical frames does not consistently improve generation quality; instead, both the quantity and temporal placement of memory strongly affect motion dynamics. These findings suggest that temporal memory should be treated as structured context rather than a homogeneous chronological buffer. Motivated by this observation, we introduce Relax Forcing, a training-free memory mechanism for autoregressive video diffusion. Relax Forcing decomposes temporal context into three functional components: Sink frames that provide global stability, Tail frames that preserve short-term continuity, and dynamically selected History frames that supply mid-range motion structure. History frames are selected using a relaxation-based criterion that encourages alignment with global anchors while suppressing redundancy with recent context. This role-aware sparse memory design mitigates error accumulation during long-horizon rollout while preserving motion evolution and reducing attention overhead. Experiments on VBench-Long show that Relax Forcing improves long-video generation, achieving stronger motion dynamics and higher overall scores than existing autoregressive baselines. These results indicate that structured temporal memory is an effective and complementary direction for scalable long-video generation.

Figures & tables

Figure 1: Long-horizon autoregressive video generation over one minute. Baseline Self Forcing suffers from severe temporal drift and visual degradation as errors accumulate. Adding an attention sink stabilises early frames but limits motion diversity. Rolling Forcing improves short-term coherence but still exhibits inconsistent patterns. In contrast, Relax Forcing dynamically selects informative history while suppressing redundancy, achieving stable identity preservation and sustained scene evolution.

Figure 2: Analysis of temporal memory in long-video extrapolation. (a–c): Varying the number of Sink, History, and Tail frames shows that increasing conditioning memory size does not consistently improve generation quality and may constrain motion dynamics. (d): Under a fixed memory budget, selecting history frames from different temporal positions leads to noticeable variations in motion dynamics while leaving image quality largely unchanged.

Figure 3: Overview of Relaxed KV Memory. Instead of retaining dense chronological history, temporal memory is decomposed into three functional components: Sink for global anchors, History for intermediate motion structure, and Tail for recent continuity. During generation, candidate historical frames are dynamically selected to remain aligned with Sink while avoiding redundancy with Tail. The selected memory is then integrated through a relaxed KV formulation with adjusted relative positional encoding, enabling the model to leverage non-contiguous temporal context while preserving long-range consistency during autoregressive rollout.

Figure 4: Temporal memory roles during long-horizon rollout. Different memory configurations lead to distinct failure modes over time. Without memory, generation becomes fragmented. Sink-only conditioning preserves appearance but limits motion evolution. History-only conditioning enables motion variation but weakens identity consistency. Tail-only conditioning improves short-term continuity but destabilises long-term rollout, while excessive Tail leads to generation collapse. These results highlight the heterogeneous roles of temporal memory in balancing stability and motion dynamics.

Figure 5: Complementary roles of temporal memory components during long-horizon rollout. Removing individual components leads to distinct failure modes. Without Sink, the generation exhibits temporal drift. Without History, motion evolution becomes limited and repetitive. Without Tail, short-term continuity is weakened, resulting in rigid motion. A balanced combination of Sink, History, and Tail maintains both stability and dynamic evolution.

Methods	Training Free	Throughput (FPS)	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Average
	30 seconds
CausVid [ 36 ] [CVPR’25]	$\times$	15.78	97.92	96.77	59.77	66.36	98.08	47.21	77.69
Self Forcing [ 13 ] [NeurIPS’25]	$\times$	15.78	97.34	96.47	59.44	68.58	98.63	36.62	76.18
Rolling Forcing [ 20 ] [ICLR’26]	$\times$	15.75	98.07	96.84	60.75	70.73	98.74	32.71	76.31
LongLive [ 31 ] [ICLR’26]	$\times$	18.16	97.97	96.83	61.51	69.07	98.76	45.55	78.28
Deep Forcing [ 35 ] [ICML’26]	$\surd$	15.79	97.34	96.48	60.68	69.31	98.27	57.56	79.94

Table 1: Quantitative comparison of long-video generation on VBench-Long.

Methods	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Average
Baseline (Self Forcing)	97.22	96.39	59.18	68.43	98.39	39.38	76.50
+ Full Attention	97.25	96.60	59.22	66.43	98.29	45.03	77.14
+ Attention Sink	97.18	96.40	59.80	68.58	98.32	54.37	79.11
+ Relaxed KV Memory (Ours)	96.99	96.12	60.13	68.50	97.80	65.67	80.87

Table 2: Effect of different memory conditioning strategies.

Component	Size	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Overall
Sink	0	95.84	95.26	58.92	67.56	97.68	54.93	78.36
	1	96.85	95.98	59.29	68.66	97.80	63.54	80.36
	2	96.99	96.13	60.12	68.50	97.80	65.62	80.86
	3	97.46	96.46	60.87	69.12	98.13	58.55	80.10
History	0	96.98	96.10	60.33	68.53	97.81	61.15	80.15
	1	96.99	96.13	60.12	68.50	97.80	65.62	80.86

Table 3: Analysis of memory allocation. We independently vary the number of Sink, History, and Tail frames while keeping the other components fixed.

Methods	Imaging Quality $\uparrow$	Dynamic Degree $\uparrow$	Overall $\uparrow$	Drift $\downarrow$	Repetition $\downarrow$	Balance $\downarrow$
Baseline (Self Forcing)	68.58	36.62	76.18	2.70	81.34	1.000
Random Sampling	68.55	62.47	80.42	2.15	83.59	0.805
Fixed-Positional Sampling	68.51	64.22	80.59	2.18	83.16	0.768
Attention-Based Sampling	68.80	63.18	80.54	2.17	83.20	0.765
First-Half Sampling	68.83	59.76	80.27	1.68	87.87	1.000
Relax Forcing (Ours)	68.50	65.67	80.87	2.13	83.33	0.745

Table 4: Comparison of different history selection strategies in Relaxed KV Memory.

Methods	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Average
Self Forcing	97.34	96.47	59.44	68.58	98.63	36.62	76.18
+ Relax Forcing w/o Hybrid RoPE	96.71	95.96	59.92	67.84	98.03	60.06	79.76
+ Relax Forcing w/ Hybrid RoPE	96.99	96.13	60.18	68.51	97.80	65.65	80.88

Table 5: Ablation results of Hybrid RoPE in Relax Forcing.

Metric	Baseline (21f)	Relax Forcing (7f)	Speedup
Flash Attention	444.2 ms	168.1 ms	2.64 $\times$
KV Select + RoPE	125.5 ms	114.1 ms	1.10 $\times$
Candidate Scoring	0.0 ms	3.4 ms	–
KV Cache Management	13.0 ms	49.5 ms	0.26 $\times$
Total Self-Attention	678.5 ms	430.8 ms	1.58 $\times$
Block Generation Time	848 ms	627 ms	1.35 $\times$

Table 6: Latency profiling comparison between the dense baseline and Relax Forcing. The baseline performs self-attention over 21 frames, while Relax Forcing reduces the effective attention length to 7 frames using structured KV memory.

Figure 6: Human preference study. Relax Forcing is preferred over competing methods in the user evaluation.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

History Position	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Overall
None	96.98	96.10	60.33	68.53	97.81	61.15	80.15
0	97.12	96.22	60.66	68.65	97.88	61.60	80.36
1	96.99	96.13	60.41	68.58	97.80	63.34	80.54
2	97.06	96.18	60.44	68.55	97.85	62.47	80.42
3	97.07	96.19	60.46	68.61	97.85	62.00	80.36
4	96.93	96.08	60.15	68.49	97.76	63.43	80.47

Appendix

Table C.1: Analysis on history frame selection positions while fixing the memory configuration to Sink=2 and Tail=1.

Candidates Number	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Average
2	96.98	96.12	60.16	68.52	97.79	65.55	80.85
3	97.00	96.12	60.17	68.57	97.80	65.39	80.84
4	96.99	96.13	60.12	68.50	97.80	65.62	80.86
5	97.00	96.14	60.00	68.48	97.79	65.30	80.78
7	96.94	96.09	60.14	68.46	97.77	63.44	80.47
9	96.96	96.09	60.04	68.38	97.76	63.97	80.53

Appendix

Table D.1: Effect of the size of candidate history frames used for Relaxed KV selection.

Figure D.1: Sensitivity to the redundancy weight

\lambda

in Relaxed KV Memory. Performance remains stable across a moderate range of

\lambda

, while an excessively large redundancy penalty reduces motion dynamics and overall performance. We use

\lambda=2.0

by default.

Method	Subject Consistency	Background Consistency	Aesthetic Quality	Imaging Quality	Motion Smoothness	Dynamic Degree	Average
Abrupt scene cuts / multi-shot editing, 30 seconds
Self Forcing	95.11	95.35	59.70	67.12	97.73	64.07	79.85
Rolling Forcing	96.90	96.01	63.12	72.17	98.38	45.07	78.61
Deep Forcing	95.40	95.47	61.01	66.56	97.48	67.73	80.61
Relax Forcing	95.32	95.21	60.92	69.06	97.17	88.73	84.40
Abrupt scene cuts / multi-shot editing, 60 seconds

Appendix

Table D.2: Results on three challenging long-video generation settings.

Length	Configuration	FlashAttention (ms)	% of Diffusion	Diffusion Time (ms)	Attention Speedup
30s	Relaxed KV	7,273	29.8%	24,467	–
30s	Dense-21	17,848	53.5%	33,358	2.45 $\times$
60s	Relaxed KV	14,689	29.0%	50,781	–
60s	Dense-21	37,185	54.2%	68,588	2.53 $\times$
120s	Relaxed KV	30,000	27.9%	107,738	–
120s	Dense-21	75,612	54.6%	138,516	2.52 $\times$

Appendix

Table F.1: FlashAttention runtime comparison across different generation lengths.

Figure H.1: Qualitative comparison of long-video generation under different methods.

Figure H.2: Qualitative comparison of long-video generation under different methods.

Figure H.3: Qualitative comparison of long-video generation under different methods.

Figure H.4: Qualitative comparison of long-video generation under different methods.

Figure H.5: Qualitative comparison of long-video generation under different methods.

Long-Horizon Video Generation

Long-horizon video generation creates coherent, controllable videos over extended durations despite limited context and accumulating errors.

See all 162 papers

Aug 27, 2026

Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion

Scaling video generation to long durations reveals a critical bottleneck: current models lack robust long-term memory. This deficiency can be studied along two critical aspects: object permanence, the ability to precisely reproduce the appearance of objects upon re-entry; and memory capacity, the ability to process ultra-long context and use information from distant history. Robust long-term memory requires both: object permanence without sufficient context handling limits the temporal scope, while long context length without permanence fails to maintain identity. To address this, we present Ring Forcing, an autoregressive video diffusion framework designed to robustly construct and precisely utilize long-term memory. Our ring-structured training strategy enforces retrieval from distant history, effectively reconciling the trade-off between strict historical adherence and generative diversity. To expand memory capacity, we introduce a compression and timestep composition strategy. Under fixed sequence length constraints, this method extends the effective historical span to minutes-long durations and achieves a comprehensive receptive field over the entire history. Furthermore, we present a sparse RoPE mechanism to enable flexible, scalable memory adaptation while fully exploiting pre-trained priors. Extensive experiments demonstrate that Ring Forcing achieves superior minutes-long coherence and object permanence, significantly outperforming state-of-the-art methods.

Bowen Xue, Brandon Y. Feng, Chenguo Lin +6

Stanford University, Stanford, CA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA · Peking University, Beijing, China +2

Long-Horizon Video Generation Memory-Augmented Video Generation Autoregressive Video Generation

Ask PDF

Sep 17, 2026

Recency Forcing: Bridging the Long-Horizon Gap in Autoregressive Video Generation

Autoregressive (AR) video generation degrades over long horizons due to an overlooked train-inference discrepancy we term KV eviction mismatch: models train on short clips where all context frames reside in the KV cache, but at inference, memory constraints force distant frames to be evicted from the KV cache - removing context the model was conditioned on. Rather than simulating eviction via context truncation - which discards temporal information the model still needs and degrades motion coherence - we keep the context but while progressively reducing the influence of distant frames, making their eventual eviction negligible. To guide this design, we introduce the positional response

R( Δ, \, t_{\text{denoise}})

, a perturbation-based sensitivity measure revealing that context influence decays steeply with temporal distance and varies systematically across denoising steps. Motivated by this analysis, we propose Recency Forcing, which applies a non-positive, timestep-dependent bias, termed Temporal Response Bias (TRB), on pre-softmax attention logits derived directly from

R

, closing the train-inference gap without modifying context length or training objectives. We further introduce Biased Attention Reparameterization (BAR), an exact reformulation that moves the bias outside the softmax, making TRB a standard FlashAttention call at zero overhead. Recency Forcing operates in both training-free mode and training-based mode. Experiments on VBench and VBench-Long demonstrate state-of-the-art long-horizon generation quality at no additional inference cost.

Tri Cao, Hung Nguyen, Phong Nguyen +1

Qualcomm AI Research

Long-Horizon Video Generation KV-Cache Management Autoregressive Video Generation Temporal Consistency in Video Generation

Ask PDF

Mar 26, 2026

PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

Autoregressive video diffusion models have demonstrated remarkable progress, yet they remain bottlenecked by intractable linear KV-cache growth, temporal repetition, and compounding errors during long-video generation. To address these challenges, we present PackForcing, a unified framework that efficiently manages the generation history through a novel three-partition KV-cache strategy. Specifically, we categorize the historical context into three distinct types: (1) Sink tokens, which preserve early anchor frames at full resolution to maintain global semantics; (2) Mid tokens, which achieve a massive spatiotemporal compression (32x token reduction) via a dual-branch network fusing progressive 3D convolutions with low-resolution VAE re-encoding; and (3) Recent tokens, kept at full resolution to ensure local temporal coherence. To strictly bound the memory footprint without sacrificing quality, we introduce a dynamic top-

k

context selection mechanism for the mid tokens, coupled with a continuous Temporal RoPE Adjustment that seamlessly re-aligns position gaps caused by dropped tokens with negligible overhead. Empowered by this principled hierarchical context compression, PackForcing can generate coherent 2-minute, 832x480 videos at 16 FPS on a single H200 GPU. It achieves a bounded KV cache of just 4 GB and enables a remarkable 24x temporal extrapolation (5s to 120s), operating effectively either zero-shot or trained on merely 5-second clips. Extensive results on VBench demonstrate state-of-the-art temporal consistency (26.07) and dynamic degree (56.25), proving that short-video supervision is sufficient for high-quality, long-video synthesis. https://github.com/ShandaAI/PackForcing

Xiaofeng Mao, Shaohao Rui, Kaining Ying +4

Alaya Studio, Shanda AI Research Tokyo · Fudan University · Shanghai Innovation Institute

Long Video Generation Video Diffusion Models Autoregressive Video Generation KV-Cache Compression Temporal Consistency in Video Generation

Ask PDF

Abstract

Long-Horizon Video Generation

Explore similar work

Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion

Recency Forcing: Bridging the Long-Horizon Gap in Autoregressive Video Generation

PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference