RECAP-Forcing: Retaining Content Appearances for Long Video Generation

Haiyang Xu; Zheng Ding; Zhuowen Tu

cs.CVAug 27, 2026

RECAP-Forcing: Retaining Content Appearances for Long Video Generation

Authors: Haiyang Xu, Zheng Ding, Zhuowen Tu

Organizations: UC San Diego

Abstract

Long autoregressive video generation faces a fundamental memory challenge: with a finite attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing memory by appearance novelty. A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize memory by retaining the KV cache associated with newly appearing content--such as entering subjects, disoccluded regions, and newly introduced scenes--at the moment it first becomes visible, prioritizing novelty over recency. Memory should scale with the amount of newly introduced content, rather than with video length. This appearance-indexed memory makes long-range consistency an explicit property of the memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a video, when all visible content is novel, an attention sink preserves the initial scene. As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing memory methods.

Figures & tables

Figure 2: What should a long-term video memory store? The same rollout, indexed two ways. Top : the man is recorded at his first appearance ( $t_{1}$ ); each later frame depicts something new: a passing car ( $t_{2}$ ), a pedestrian ( $t_{3}$ ), another passer-by ( $t_{4}$ ). Bottom Left : frame-indexed memory re-stores old and new content alike at every step, then compresses or evicts it as time passes, so memory pressure grows with video length. Bottom Right : appearance-indexed memory (RECAP-Forcing) stores content when it newly appears, at canonical temporal position $0$ , memory scales with the amount of newly appearing content, not with its duration.

Figure 3: The optical-flow novelty bank. (a) For every pair of consecutive frames, RAFT flow is aggregated into the per-patch novelty score of Eq. 4 , and the newly appearing patches are selected. Their per-layer key/value vectors are gathered from the KV cache (temporal RoPE phase reset to zero) and added to the bank together with their scores; the bank keeps the top- $K$ ( $K{=}1{,}560$ ) by score. (b) Selected memory patches (green) on two examples: entering subjects and newly visible parts of subjects.

Method	S.C.	B.C.	T.F.	M.S.	D.D.	A.Q.	I.Q.	O.C.	M.O.	H.A.	C.	S.R.	S.	A.S.	T.S.	Ov.C.	Q.	Sem.	Total
Self-Forcing [ 25 ]	98.0	96.8	98.3	98.9	27.5	53.8	69.2	79.5	32.1	77.0	72.8	62.6	12.1	20.6	20.9	19.6	80.4	58.0	75.9
$+$ Sink ( $\lambda{=}1$ )	98.2	97.0	98.3	98.9	30.4	53.3	67.9	89.2	41.1	75.1	78.2	73.6	14.6	20.4	20.3	21.2	80.4	62.3	76.8
$+$ Ours	97.7	96.3	97.4	98.5	58.1	58.4	70.9	91.8	47.3	77.1	94.6	68.4	17.7	18.7	22.8	22.1	82.9	65.5	79.5
Infinite-Forcing [ 10 ]	97.4	95.9	97.1	98.4	61.3	55.8	68.8	93.6	49.9	74.1	81.9	64.0	12.3	18.6	22.9	21.9	82.2	62.9	78.3
$+$ Ours	97.4	96.0	97.2	98.4	71.3	57.5	68.8	97.1	48.9	75.7	96.1	63.4	15.1	18.5	23.0	22.2	83.3	65.4	79.7
LongLive [ 70 ]	98.7	97.0	98.5	99.2	20.3	57.1	71.6	98.2	52.2	75.9	80.4	80.5	14.8	18.1	22.7	22.6	81.2	65.9	78.1

Table 1: Main results on VBench-Long [ 28 ] , all 16 dimensions (

128

prompts

\times\,5

seeds,

60

s; higher is better). RECAP-Forcing (optical-flow memory bank

+

reinforced sink) is applied as the same training-free inference hook to every base model, each followed by its “

+

Ours” row; for Self-Forcing we additionally report a plain, unreinforced sink (

\lambda{=}1

). Helios is 14B, all others are 1.3B. Abbreviations (first letter of each word): S.C. Subject Consistency, B.C. Background Consistency, T.F. Temporal Flickering, M.S. Motion Smoothness, D.D. Dynamic Degree, A.Q. Aesthetic Quality, I.Q. Imaging Quality; semantic suite: O.C. Object Class, M.O. Multiple Objects, H.A. Human Action, C. Color, S.R. Spatial Relationship, S. Scene, A.S. Appearance Style, T.S. Temporal Style, Ov.C. Overall Consistency. Q./Sem./Total

=

official VBench aggregation (min–max normalized; quality

=7

dims with Dynamic Degree half-weighted, semantic

=9

dims, Total

=(4\,\mathrm{Q}+\mathrm{Sem})/5

). All values are percentages.

Method	S.C.	B.C.	T.F.	M.S.	D.D.	A.Q.	I.Q.	O.C.	M.O.	H.A.	C.	S.R.	S.	A.S.	T.S.	Ov.C.	Q.	Sem.	Total
Self-Forcing	98.0	96.8	98.3	98.9	27.5	53.8	69.2	79.5	32.1	77.0	72.8	62.6	12.1	20.6	20.9	19.6	80.4	58.0	75.9
$+$ Deep Forcing [ 73 ]	98.4	93.7	98.2	94.8	32.0	55.9	71.8	98.3	48.5	76.6	82.9	73.1	21.8	18.3	23.0	22.3	78.7	66.1	76.2
$+$ MemRoPE [ 31 ]	98.4	96.7	98.0	99.0	41.2	56.6	73.1	94.2	44.1	76.6	84.8	68.4	15.9	19.1	21.8	21.1	82.4	63.6	78.7
$+$ Rolling Sink [ 34 ]	98.9	97.3	98.7	99.2	23.0	58.3	72.6	97.9	48.2	80.7	90.3	76.5	17.6	18.2	23.2	22.5	81.8	67.1	78.9
$+$ Ours	97.7	96.3	97.4	98.5	58.1	58.4	70.9	91.8	47.3	77.1	94.6	68.4	17.7	18.7	22.8	22.1	82.9	65.5	79.5

Table 2: Comparison of RECAP-Forcing with other training-free methods. Same evaluation settings on VBench-Long as in Tab. 1 .

Reinf. Sink	Memory Bank	S.C.	B.C.	M.S.	D.D.	A.Q.	I.Q.	Q.	Sem.	Total
✗	✗	97.4	95.9	98.4	61.3	55.8	68.8	82.2	62.9	78.3
✓	✗	98.2	96.7	98.8	41.6	57.8	68.9	81.9	65.2	78.6
✗	✓	96.8	95.5	98.1	78.2	55.2	68.5	82.8	63.0	78.8
✓	✓	97.4	96.0	98.4	71.3	57.5	68.8	83.3	65.4	79.7

Table 3: Ablation of the reinforced sink and memory bank.

	Bank Capacity (frame)						Sink-Reinforcement Strength ( $\lambda$ )
Metric	1/4	1/2	1	2	4	8	1	3	5	8	10	12
Q.	83.0	83.2	83.1	83.4	83.3	82.9	82.8	83.0	83.1	82.7	82.5	82.2
Sem.	66.0	65.8	65.7	65.4	65.9	65.1	63.0	64.5	65.7	65.2	64.4	64.1
Total	79.6	79.7	79.7	79.8	79.8	79.3	78.8	79.3	79.7	79.2	78.9	78.6

Table 4: Ablations on bank capacity and sink-reinforcement strength. Bold columns denote the default settings.

Figure 4:

\lambda

Sweep (on

128

prompts

\times\,5

seeds). (a)

\lambda

smoothly sets attention on the sink:

{\sim}24\%

\lambda{=}5

. (b) Subject Consistency is already saturated while Dynamic Degree erodes past

\lambda{\approx}8

Competitor	Consistency	Motion	Overall	$\mathbf{n}$
Self-Forcing [ 25 ]	$85.2\%$ (6T)	$76.9\%$ (7T)	$84.3\%$ (3T)	54
Deep Forcing [ 73 ]	$73.1\%$ (15T)	$76.9\%$ (5T)	$77.8\%$ (4T)	54
MemRoPE [ 31 ]	$84.3\%$ (7T)	$67.6\%$ (5T)	$75.0\%$ (3T)	54
Rolling Sink [ 34 ]	$82.4\%$ (7T)	$76.9\%$ (5T)	$79.6\%$ (6T)	54
All	$\mathbf{81.3\%}$ (35T)	$\mathbf{74.5\%}$ (22T)	$\mathbf{79.2\%}$ (16T)	216

Table 5: Human study results. Win rates of RECAP-Forcing in blind pairwise comparisons against each competitor. “T” denotes the number of HARD TO SAY judgments, which are treated as ties and assigned 0.5 credit to win rates.

Figure 5: Qualitative results on four one-minute prompts. Across all scenes, the baseline loses subject identity, style, scale, or background consistency. In contrast, RECAP-Forcing preserves the subject(s) and the full scene throughout the minute.

Hyperparameter	Value
Sink size	$1$ latent frame
Attention window	$6$ latent frames
Block size	$3$ latent frames
Bank capacity $K$	$1{,}560$ tokens
Sink reinforcement $\lambda$	$5$
Flow cycle-error threshold $\tau_{\mathrm{cyc}}$	$2$ px

Table 6: Hyperparameters of RECAP-Forcing .

Figure 6: Length Sweep (Baseline vs. RECAP-Forcing,

5

–

60

s; error bars show

\pm

std over

5

seeds). (a) Our method maintains higher Dynamic Degree across all lengths. (b) Subject Consistency remains comparable to the baseline.

Configuration	Frame Sim.	Wins	Crop Sim.	Wins
Baseline	0.745	–	0.642	–
$+$ Ours	0.850	76%	0.696	64%

Table 7: Long-horizon Subject Identity (

20

prompts

\times\,5

seeds,

60

s; DINOv2 CLS similarity between an early window and the final

10

s, on the full frame and on the detected subject crop).

Horizon	Method	start	mid	end
$5$ min	Baseline	1.00	0.63	0.61
$5$ min	+ Ours	1.00	0.81	0.81
$60$ s	Baseline	1.00	0.70	0.62
$60$ s	+ Ours	1.00	0.87	0.88

Table 8: Identity retention over time (

20

prompts

\times

5

seeds. “mid”/“end” are

1

5

min for the five-minute rows and

30

60

s for the

60

-second rows). RECAP-Forcing holds identity where the recency baseline drifts.

Configuration	Frame Sim.	Wins	Crop Sim.	Wins
Baseline	0.699	–	0.648	–
$+$ Ours	0.752	73%	0.688	66%

Table 9: Controlled Reappearance . Subject enters, leaves for a

7.5

s gap (

5\times

the attention window) with the sliding window cleared at every shot boundary, then re-conditioned.

	S.C.	B.C.	M.S.	D.D.	A.Q.	I.Q.	Q.	Sem.	Total
No Bank (Baseline)	97.4	95.9	98.4	61.3	55.8	68.8	82.2	62.9	78.3
Random History	97.1	96.9	98.5	62.4	56.4	69.2	82.4	58.7	77.7
Recent History	97.4	96.1	98.4	62.0	56.0	69.0	82.3	64.2	78.7
DINOv2 Diversity	97.6	96.1	98.6	60.6	57.6	69.4	82.9	65.6	79.4
Flow Novelty (Ours)	97.4	96.0	98.4	71.3	57.5	68.8	83.3	65.4	79.7

Table 10: Ablation of Bank Admission Rules .

Length	Frames (blocks)	Baseline	Ours	Overhead
$1$ min	0 240 ( 0 80)	0 185 s	0 288 s	$1.56\times$
$2$ min	0 480 (160)	0 268 s	0 450 s	$1.68\times$
$5$ min	1200 (400)	1435 s	2239 s	$1.56\times$

Table 11: Generation Time for a single video of resolution of

832\times 480

on a 80GB GH200 GPU. RECAP-Forcing adds a

\sim

1.6

\times

overhead that does not grow with length.

Config	Camera Flow	Object Flow	Object Fraction	Total Flow	Drift Tail
Baseline	1.204	0.710	0.380	1.914	37
$+$ Ours	0.897	0.659	0.408	1.556	24

Table 12: Camera–Object Motion Decomposition RECAP-Forcing reduces camera motion while largely preserving object motion, raising the object share. Drift tail counts videos whose mean total flow exceeds

2\times

the config median.

Long Video Generation

Long video generation creates extended, coherent video sequences while managing memory, motion, identity, and narrative continuity.

See all 66 papers

Recast the Past: Stable Long Video Generation with Full Context Memory
Oct 20, 2026Yukuo Ma, Cong Liu, Junke Wang +4Long-Horizon Video Generation Memory-Augmented Video Generation
Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom
Memento: Reconstruct to Remember for Consistent Long Video Generation
Jun 12, 2026Xuan Wei, Longbin Ji, Guan Wang +5Long Video Generation Memory-Augmented Video Generation
Xiamen University · ERNIE Team, Baidu Inc.
Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation
Mar 22, 2026Zengqun Zhao, Yanzuo Lu, Ziquan Liu +3Long-Horizon Video Generation Autoregressive Video Generation
Queen Mary University of London · Imperial College London · Huawei R&D UK

Oct 20, 2026J/K move · Enter open · S save

Recast the Past: Stable Long Video Generation with Full Context Memory

Yukuo Ma, Cong Liu, Junke Wang, Junqi Liu+3

Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom

Long-Horizon Video Generation Memory-Augmented Video Generation Autoregressive Video Generation Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Long-horizon video generation requires retaining an ever-expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual-pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench-Long overall score of 80.1 and competitive static-scene revisitation performance on R2M-Bench. It also leads on all four consistency metrics for off-screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.

Abstract

Long Video Generation

Explore similar work

Recast the Past: Stable Long Video Generation with Full Context Memory

Memento: Reconstruct to Remember for Consistent Long Video Generation

Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation