Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion

Bowen Xue; Brandon Y. Feng; Chenguo Lin; Yuchen Lin; Yujia Zeng; Lvmin Zhang; Maneesh Agrawala; Honglei Yan

cs.CVAug 27, 2026

Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion

Authors: Bowen Xue, Brandon Y. Feng, Chenguo Lin, Yuchen Lin, Yujia Zeng, Lvmin Zhang, Maneesh Agrawala, Honglei Yan,

Organizations: Stanford University, Stanford, CA, USA · Massachusetts Institute of Technology, Cambridge, MA, USA · Peking University, Beijing, China · University of California, Berkeley, Berkeley, CA, USA ·

Abstract

Scaling video generation to long durations reveals a critical bottleneck: current models lack robust long-term memory. This deficiency can be studied along two critical aspects: object permanence, the ability to precisely reproduce the appearance of objects upon re-entry; and memory capacity, the ability to process ultra-long context and use information from distant history. Robust long-term memory requires both: object permanence without sufficient context handling limits the temporal scope, while long context length without permanence fails to maintain identity. To address this, we present Ring Forcing, an autoregressive video diffusion framework designed to robustly construct and precisely utilize long-term memory. Our ring-structured training strategy enforces retrieval from distant history, effectively reconciling the trade-off between strict historical adherence and generative diversity. To expand memory capacity, we introduce a compression and timestep composition strategy. Under fixed sequence length constraints, this method extends the effective historical span to minutes-long durations and achieves a comprehensive receptive field over the entire history. Furthermore, we present a sparse RoPE mechanism to enable flexible, scalable memory adaptation while fully exploiting pre-trained priors. Extensive experiments demonstrate that Ring Forcing achieves superior minutes-long coherence and object permanence, significantly outperforming state-of-the-art methods.

Figures & tables

Figure 2: Attention allocation of Predicted Frames to History Frames. (a) Standard Linear Training suffers from myopic attention bias, failing to retrieve effective information from distant history. (b) Ring-Structured Training (Ours) enables the model to extract more information from the past, achieving precise retrieval of historical frames.

Figure 3: Overview of the Ring-Structured Training Strategy. We construct a sequence ring by concatenating a linear raw video with its reversed counterpart. By unrolling the ring into a conditioning sequence, the target frames are naturally embedded into the distant history as the Hidden GT, which essentially forces the autoregressive generator to learn explicit long-range retrieval. To prevent trivial shortcuts, a Random Head Crop is applied to truncate boundary information leakage, while a Random Context Drop is used to balance history adherence and generation diversity.

Figure 4: Timestep-composed long-context conditioning.

Figure 5: Qualitative results on the Appear–Disappear–Reappear (A-D-R) benchmark. Our model demonstrates robust object permanence by retrieving identity information from distant history, ensuring consistent subject reconstruction even after long temporal gaps where baseline models typically fail.

Model	Objective Metrics					User Study
Model	Aesthetic	Dynamics	Naturalness	Motion Smoothness	Instruction Following	Consistency	Overall Quality
LongLive	5.622	1.056	3.031	0.991	24.805	4.12	2.25
LongCat	5.808	2.196	3.875	0.991	25.341	3.59	4.19
FramePack	5.728	1.790	3.656	0.991	25.014	3.71	4.04
Ours	5.822	2.261	3.922	0.986	25.471	4.35	4.22
Base model †	5.825	1.974	4.016	0.984	25.136	N/A

Table 2: Quantitative comparison on the General long-video generation benchmark (1 minute).

Figure 6: Qualitative results of 60-second long video generation. The visualization demonstrates the model’s capability to maintain overall consistency and visual quality throughout a 1-minute duration in general scenarios.

Figure 7: Effectiveness of Timestep Composition Strategy.

$p_{drop}$	Ring Mode (Reconstruction)			Standard Mode
$p_{drop}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Diversity Score $\uparrow$
0.0	25.02	0.73	0.12	3.71
0.2	24.67	0.72	0.12	3.93
0.4	24.22	0.71	0.13	4.11
0.6	24.02	0.71	0.13	4.13
0.8	23.02	0.68	0.16	4.21

Table 3: Ablation study on Ring Context Drop probability

p_{drop}

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Standard RoPE	19.05	0.58	0.27
Sparse RoPE (Ours)	24.22	0.71	0.13

Table 4: Comparison of reconstruction capabilities between standard index-based RoPE and our sparse RoPE.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A8 : Qualitative Comparison on the A-D-R Benchmark (1s Gap). While state-of-the-art baselines suffer from immediate identity drift and fail to reconstruct the hidden object (stack of white plates), Ring Forcing accurately retrieves historical information, ensuring strict object permanence.

Figure A9 : Qualitative Comparison on the A-D-R Benchmark (1s Gap). Existing autoregressive models exhibit myopic attention bias, altering the geometry and texture of the subject (cookie) after merely one second of occlusion. In contrast, our method maintains strong attribute retention.

Figure A10 : Qualitative Comparison on the A-D-R Benchmark (1s Gap). Even with simple objects (hot dog), baseline models struggle to maintain consistency. Ring Forcing effectively bridges the temporal gap, accurately recovering the subject’s identity.

Figure A11 : Qualitative Comparison on the A-D-R Benchmark (1s Gap). While baseline models struggle to maintain the structural and textural details of the hidden object (wooden pallet) after a brief occlusion, Ring Forcing successfully retrieves the precise historical information, ensuring strict object permanence.

Figure A12 : Qualitative Comparison on the A-D-R Benchmark (5s Gap). As the temporal divide expands, baseline models experience catastrophic forgetting. Ring Forcing reliably preserves the specific visual features of the subject (yellow shirt) from distant history.

Figure A13 : Qualitative Comparison on the A-D-R Benchmark (5s Gap). Our method successfully reconstructs complex human attributes across a 5-second occlusion, whereas baselines hallucinate entirely new identities, highlighting their inability to perform long-range retrieval.

Figure A14 : Extreme Long-Term Memory Retrieval (15s Gap). Operating well beyond the maximum context limits of current baselines, Ring Forcing accurately recovers diverse objects (pizza, watermelon, acoustic guitar) after a 15-second disappearance, demonstrating highly resilient memory capacity.

Figure A15 : Extreme Long-Term Memory Retrieval (30s Gap). Stress-testing our method with a half-minute visual gap. Ring Forcing exhibits robust temporal consistency and semantic adherence even for dynamic scenes (burning log, basketball, tractor), successfully overcoming the bottleneck of prolonged occlusions.

Figure A16 : Long-Term Object Permanence Evaluation (60s Gap). Demonstrating minute-level memory preservation, Ring Forcing successfully retrieves precise high-frequency details (lawnmower, motorcycle, autumn leaf) across a 60-second temporal gap, further demonstrating the effectiveness of Ring Forcing.

Figure A17 : Qualitative Results on General Long Video Generation (1/5). This example demonstrates the model’s capability to maintain visual quality, stable dynamics, and overall spatiotemporal consistency throughout a 1-minute continuous generation in general scenarios.

Figure A18 : Qualitative Results on General Long Video Generation (2/5). This example demonstrates the model’s capability to maintain visual quality, stable dynamics, and overall spatiotemporal consistency throughout a 1-minute continuous generation in general scenarios.

Figure A19 : Qualitative Results on General Long Video Generation (3/5). This example demonstrates the model’s capability to maintain visual quality, stable dynamics, and overall spatiotemporal consistency throughout a 1-minute continuous generation in general scenarios.

Figure A20 : Qualitative Results on General Long Video Generation (4/5). This example demonstrates the model’s capability to maintain visual quality, stable dynamics, and overall spatiotemporal consistency throughout a 1-minute continuous generation in general scenarios.

Figure A21 : Qualitative Results on General Long Video Generation (5/5). This example demonstrates the model’s capability to maintain visual quality, stable dynamics, and overall spatiotemporal consistency throughout a 1-minute continuous generation in general scenarios.

Long-Horizon Video Generation

Long-horizon video generation creates coherent, controllable videos over extended durations despite limited context and accumulating errors.

See all 162 papers

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation
Mar 22, 2026Zengqun Zhao, Yanzuo Lu, Ziquan Liu +3Long-Horizon Video Generation Autoregressive Video Generation
Queen Mary University of London · Imperial College London · Huawei R&D UK
PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference
Mar 26, 2026Xiaofeng Mao, Shaohao Rui, Kaining Ying +4Long Video Generation Video Diffusion Models
Alaya Studio, Shanda AI Research Tokyo · Fudan University · Shanghai Innovation Institute
Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation
May 15, 2026Mingqiang Wu, Weilun Feng, Zhefeng Zhang +8Video Diffusion Models Long-Horizon Video Generation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · China University of Mining & Technology, Beijing +3

Mar 22, 2026J/K move · Enter open · S save

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

Zengqun Zhao, Yanzuo Lu, Ziquan Liu, Jifei Song+2

Queen Mary University of London · Imperial College London · Huawei R&D UK

Long-Horizon Video Generation Autoregressive Video Generation Memory-Augmented Video Generation Temporal Consistency in Video Generation KV-Cache Management

Open paper AskPDF arXiv

Autoregressive video diffusion has recently emerged as a promising paradigm for long-video generation, enabling causal synthesis beyond the temporal limits of bidirectional models. Existing forcing-based training strategies reduce exposure bias by conditioning models on their own predictions during rollout, yet minute-scale generation remains challenging due to progressive temporal degradation and constrained motion evolution. In this work, we study the role of temporal KV memory during long-horizon autoregressive inference. Our analysis shows that simply retaining more historical frames does not consistently improve generation quality; instead, both the quantity and temporal placement of memory strongly affect motion dynamics. These findings suggest that temporal memory should be treated as structured context rather than a homogeneous chronological buffer. Motivated by this observation, we introduce Relax Forcing, a training-free memory mechanism for autoregressive video diffusion. Relax Forcing decomposes temporal context into three functional components: Sink frames that provide global stability, Tail frames that preserve short-term continuity, and dynamically selected History frames that supply mid-range motion structure. History frames are selected using a relaxation-based criterion that encourages alignment with global anchors while suppressing redundancy with recent context. This role-aware sparse memory design mitigates error accumulation during long-horizon rollout while preserving motion evolution and reducing attention overhead. Experiments on VBench-Long show that Relax Forcing improves long-video generation, achieving stronger motion dynamics and higher overall scores than existing autoregressive baselines. These results indicate that structured temporal memory is an effective and complementary direction for scalable long-video generation.

Abstract

Long-Horizon Video Generation

Explore similar work

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation