Recast the Past: Stable Long Video Generation with Full Context Memory

Yukuo Ma; Cong Liu; Junke Wang; Junqi Liu; Haibin Huang; Zuxuan Wu; Xuelong Li

cs.CVOct 20, 2026

Recast the Past: Stable Long Video Generation with Full Context Memory

Authors: Yukuo Ma, Cong Liu, Junke Wang, Junqi Liu, Haibin Huang, Zuxuan Wu, Xuelong Li

Organizations: Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom

Abstract

Long-horizon video generation requires retaining an ever-expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual-pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench-Long overall score of 80.1 and competitive static-scene revisitation performance on R2M-Bench. It also leads on all four consistency metrics for off-screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.

Figures & tables

Figure 1: A coarse read on fine-grained history. We compare our method with other representative methods. Keyframe-based approaches, such as CaM ( Yu et al., 2025 ) , typically employ an external rule-based selector to retrieve a subset of frames as memory. In contrast, compression-based methods ( Zhang et al., 2025 ) encode history into a compact representation through a separate reconstruction stage. Different from these approaches, ReCam allows the video model itself to learn how to represent history and how to retrieve it via its native attention mechanism.

Figure 2: Overview of ReCam. Historical latents are grouped into local spatiotemporal blocks and projected into memory tokens by a lightweight MLP. The first frame remains a clean visual anchor. Native self-attention connects the historical memory and current chunk, while text cross-attention is applied only to the current chunk.

Method	#Params	Total Score	Subject Consistency	Background Consistency	Motion Smoothness	Dynamic Degree	Imaging Quality	Aesthetic Quality
MAGI-1 ( Teng et al., 2025 )	24B	78.5	98.2	97.2	99.4	21.4	66.4	55.9
FramePack ( Zhang and Agrawala, 2025 )	13B	79.5	98.7	99.2	99.5	16.8	70.9	61.3
SkyReels-V2 ( Chen et al., 2025 )	14B	79.2	96.0	96.0	99.0	53.2	64.8	56.2
MMPL ( Xiang et al., 2025 )	14B	78.8	96.2	95.3	98.8	49.2	66.4	55.8
LongLive 2.0 ( Chen et al., 2026b )	5B	76.5	92.3	92.5	99.1	25.1	70.0	59.8
Helios ( Yuan et al., 2026 )	14B	78.6	97.6	97.3	99.1	28.6	66.4	59.3

Table 1: VBench-Long evaluation results. We report the VBench-Long metrics on videos generated at 480p, 30 seconds. Higher values are better for all dimensions.

		Appearance Fidelity			Scene Identity			Object Identity
Method	#Params	PSNR	SSIM	LPIPS	DINO-sim	BoQ-sim	MVPR-sim	App. Persist	Sem. Persist
HY-WorldPlay ( Sun et al., 2025 )	8B	2.626	0.060	0.158	0.287	0.340	0.157	0.096	0.026
Sana-WM ( Zhu et al., 2026 )	2.6B	0.534	0.014	0.022	0.039	0.019	0.012	0.004	0.002
LingBot-World-Fast ( Gao et al., 2026a )	18B	1.021	0.028	0.057	0.194	0.207	0.162	0.069	0.019
LingBot-World-2-Fast ( Gao et al., 2026b )	18B	1.349	0.037	0.095	0.216	0.232	0.117	0.064	0.014
ReCam (Ours)	14B	1.691	0.045	0.104	0.286	0.288	0.212	0.087	0.026

Table 2: Static-scene revisitation on R2M-Bench. MemoryGain (MG, top) and Normalized Memory Ratio (NMR, bottom); higher is better for both.

Method	Face $\downarrow$			Character $\downarrow$			Background $\downarrow$			Clothes $\uparrow$
	All	Present	Absent	All	Present	Absent	All	Present	Absent	All	Present	Absent
FramePack-F1 ( Zhang and Agrawala, 2025 )	0.578	0.400	0.745	12.54	9.82	14.70	0.043	0.028	0.054	2.44	3.95	1.25
Helios ( Yuan et al., 2026 )	0.748	0.657	0.771	13.91	12.64	14.52	0.063	0.105	0.043	1.82	2.69	1.41
LongLive 2.0 ( Chen et al., 2026b )	0.626	0.553	0.710	13.18	13.03	13.34	0.239	0.254	0.219	2.25	2.73	1.73
Sana-WM Streaming ( Zhu et al., 2026 )	0.642	0.624	0.682	11.67	10.86	12.92	0.073	0.069	0.078	2.39	2.84	1.73
LingBot-World-Fast ( Gao et al., 2026a )	0.591	0.578	0.606	10.84	10.83	10.86	0.079	0.071	0.085	3.03	3.03	3.04

Table 3: Off-screen subject reappearance. For each consistency metric, we report the mean metric over all cases (All), the cases where the subject is visible in the first frame (Present), and the cases where it is absent from the first frame (Absent). Bold indicates the best result in each column.

Figure 3: Qualitative results. From top to bottom: long-horizon generation, static-scene revisitation, and off-screen subject reappearance.

Panel of Table 4 — Table 4: Ablation studies. (a) History block shape and MLP bottleneck dimension. (b) Context design: VBench changes between the first and last two-second windows ( $\Delta=\mathrm{last}-\mathrm{first}$ ); negative values indicate degradation. Bold marks the best result per column.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Overview of the general-purpose video corpus. (a) Percentage of videos in each duration interval. (b) Word cloud of the 60 most frequent words in the captions after filtering out template terms such as “captures” and “environment”.

Figure 8: Probability densities of the training noise level. The reference uses a single shifted logit-normal sample with shift

5

. Our min/max branch takes the larger of two samples with shift

2.5

; the full training mixture additionally includes 10% uniform samples from

[0.95,1]

Context shift $s_{\mathrm{c}}$	Total Score	Subject Consistency	Background Consistency	Motion Smoothness	Dynamic Degree	Imaging Quality	Aesthetic Quality
0	79.59	97.40	96.81	98.02	47.73	69.50	57.69
0.1	79.78	97.58	96.92	98.11	46.88	70.07	57.92
0.5	79.91	97.72	97.04	98.22	46.31	69.88	58.43
1	80.01	97.58	96.99	98.19	46.31	70.77	58.37
2	79.97	97.73	97.01	98.22	44.89	70.68	58.65
5	79.89	97.79	97.05	98.32	42.33	70.71	59.03

Appendix

Table 6: Effect of the context noise shift on VBench scores. The current-chunk shift is fixed at

s_{\mathrm{m}}=5

. Bold marks the best result per column.

Case 6: Burgundy sweater

Appendix

Table 17

Case 3: Snowy ballroom





@makecaption \fnum@figure Static-scene revisitation across the lakeside, library, and snowy ballroom. Columns show $5$ , $10$ , $20$ , and $25$ seconds for every method. Compare columns 1/2 and 3/4 for persistence of scene layout and object appearance after returning to the same target camera pose.

Appendix

Table 20

Image-to-video examples

Appendix

Table 21

Long-Horizon Video Generation

Long-horizon video generation creates coherent, controllable videos over extended durations despite limited context and accumulating errors.

See all 164 papers

RECAP-Forcing: Retaining Content Appearances for Long Video Generation
Aug 27, 2026Haiyang Xu, Zheng Ding, Zhuowen TuLong Video Generation Autoregressive Video Generation
UC San Diego
LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation
Jun 1, 2026Qixin Hu, Shuai Yang, Wei Huang +2Autoregressive Diffusion Long Video Generation
NVIDIA · USC · MIT
Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation
May 15, 2026Mingqiang Wu, Weilun Feng, Zhefeng Zhang +8Video Diffusion Models Long-Horizon Video Generation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · China University of Mining & Technology, Beijing +3

Aug 27, 2026J/K move · Enter open · S save

RECAP-Forcing: Retaining Content Appearances for Long Video Generation

Haiyang Xu, Zheng Ding, Zhuowen Tu

UC San Diego

Long Video Generation Autoregressive Video Generation Memory-Augmented Video Generation Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Long autoregressive video generation faces a fundamental memory challenge: with a finite attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing memory by appearance novelty. A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize memory by retaining the KV cache associated with newly appearing content--such as entering subjects, disoccluded regions, and newly introduced scenes--at the moment it first becomes visible, prioritizing novelty over recency. Memory should scale with the amount of newly introduced content, rather than with video length. This appearance-indexed memory makes long-range consistency an explicit property of the memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a video, when all visible content is novel, an attention sink preserves the initial scene. As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing memory methods.

Abstract

Long-Horizon Video Generation

Explore similar work

RECAP-Forcing: Retaining Content Appearances for Long Video Generation

LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation

Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation