Organizations: University of Science and Technology of China · KlingAI Research · Tsinghua University · The Hong Kong University of Science and Technology · City University of Hong Kong
Long-video generators increasingly retrieve past visual observations to prevent identity drift. The intuition is compelling: more access to history should make future video more consistent. But an image can be faithful to the past and wrong for the present. Does visual memory always improve the generated story? We investigate this question through VMem-Bench, a two-track diagnostic study. Track-A measures which references are selected from 92 real films (16,548 segments), before generation and without generator confounds. Track-B compares four retrieval policies with no-memory and gold-informed last-seen controls on 30 causally generated stories (2,824 segments) using a shared generator. Retrieval increases entity recall from 0.559 to 0.628-0.693 and improves similarity to past appearances, but decreases precision from 0.977 to 0.915-0.938 and forbidden-entity avoidance from 0.968 to 0.845-0.898. It can also reinforce obsolete appearances after the story changes them. Conversely, high long-gap recall can arise without memory when the prompt itself supplies the appearance; an appearance-stripping analysis exposes this confound. These results distinguish historical fidelity from current-story correctness and show why retrieval relevance or final-video consistency alone does not establish memory utility.
Figures & tables
Figure 1: A relevant past can be wrong for the present. Three qualitative failures from earlier Track-B testbed rollouts (not the main quantitative rerun), showing previous appearances, retrieved frames, composed keyframes, and generated clips. Top: a deceased character reappears when a jacket is mentioned. Middle: a white look-alike replaces the requested orange dinghy. Bottom: retrieved frames preserve Axi’s old scarf despite a new instruction specifying a short cape. Symbols indicate required-entity matches (✓), misses ( × ), and forbidden appearances (!). These examples illustrate failure paths; they are not fixed-history causal interventions.
Family
Metric
Satisfied when
Long-gap reappearance
recall
A character or prop returns after at least 30 segments.
Look-alike
accuracy
The identity matches the intended member of a confusable pair.
State change
accuracy
A transformed entity appears in its new state.
Count / multi-instance
accuracy
The rendered count matches the authored count.
Avoidance
avoidance-OK
A removed, destroyed, or deceased entity is absent.
False friend
accuracy
A new look-alike is not collapsed into an existing identity.
Table 1: Track-B memory-stress families. Each rate is a micro-average over that family’s opportunities.
Corpus
Hard-case segments
Dataset
Movies
Segments
Entities
Char.
Prop.
Loc.
Dur. (h)
Re-app.
Multi-inst.
Scene-ret.
State-chg.
BlenderOpenMovies
23
1,364
369
106
149
114
3.84
415
674
106
86
LSMDC
69
15,184
2,434
881
690
863
41.07
5,425
8,596
1,582
179
Total
92
16,548
2,803
987
839
977
44.91
5,840
9,270
1,688
265
Table 2: VMem-Bench Track-A corpus statistics (per dataset). Entities = characters + props + locations; hard-case columns count segments per bucket. BlenderOpenMovies is stylized animation, LSMDC live-action film clips; the two are analyzed separately throughout.
Figure 2: (a) Causal evaluation. Both tracks read references for the current instruction. Track-A scores references before revealing the real next segment, then updates from that segment. Track-B generates and scores a clip, then updates from its own output. (b) Track-B cohort and stress coverage. A segment can carry multiple stress tags.
Figure 3: Overall memory trade-offs and diagnostic slices. Points: policy minus no memory (zero line); whiskers: paired story-bootstrap 95% CIs conditional on one rollout/story. Titles give absolute no-memory baselines. (a,b) Pooled character/prop Recall and Precision; (c) forbidden-entity Avoid-OK. (d,e) Character-return recall; 61/62 long-gap returns change state with prompt-described looks. All five use the same percentage-point scale. (f) Historical DINOv3 cosine on a separate scale, on policy-dependent subsets detected in both appearances; higher need not mean more correct. † Gold-ID Last-seen uses a variable budget; retrieval uses K=3 .
Presence, identity, and count
State correctness
Entity exclusion
All
System
Long-gap
Look-alike
False-fr.
Count
Temporal
State-chg
Persist
Avoid
Indirect
Overall ↑
rec.
acc.
acc.
acc.
rec.
acc.
acc.
avoid.
avoid.
rec.
Shared generator: no-memory baseline, four retrieval policies, Last-seen control
No memory ( K=0 )
0.890
0.952
0.980
0.779
0.867
0.991
0.689
0.964
0.852
0.559
Text-frame ret.
0.819
0.964
0.960
0.803
0.827
0.995
0.802 +
0.889 −
0.694 −
0.628 +
Seg+uniform ret.
0.871
0.959
0.900 −
0.794
0.860
0.991
0.783 +
0.829 −
0.705 −
0.689 +
Table 3: Track-B: generated-video outcomes by stress (30 stories/system; higher is better). Cells are absolute rates. In shared-generator policy rows, blue/+ and red/ − mark paired story-bootstrap 95% CIs above/below the first-row no-memory baseline; unmarked cells have no supported direction. Long-gap pools characters and props; Table 4 separates them. Overall is pooled character/prop recall. The separate released-system block uses different generators and is not a memory-only comparison. Stress-level judge sensitivity: Sec. 4.6 .
(a) Original prompts: observational groups
Entity
Gap
n
No memory
DINO-key ΔR
Policies +/−/?
Character
1–4
430
0.553
+0.195 0.120
4/0/0
≥ 30
62
0.984
-0.032 0.130
0/2/2
Prop
1–4
271
0.635
+0.041 0.110
0/0/4
≥ 30
148
0.851
+0.034 0.190
0/0/4
Table 4: Return recall and prompt information (30 stories). No memory: absolute recall; DINO-key ΔR : difference from it. Gray subscripts: full paired story-bootstrap 95% CI width; bold: CI excludes zero. +/−/? : counts of four retrieval policies with CIs above/below/including zero. Gaps: segments; prompts describe changed looks in 61/62 long-gap character returns. Only (b) changes prompts; see Supplementary Table 17 .
Policy
All returns N=1130
Characters N=486
Props N=644
Changed N=61
No memory
0.35
0.27
0.41
0.19
Text-frame Δ
+0.15
+0.13
+0.16
+0.18
Seg+uniform Δ
+0.21
+0.20
+0.21
+0.20
Seg+DINO-key Δ
+0.20
+0.19
+0.21
+0.20
Seg+frame Δ
+0.21
+0.18
+0.23
+0.15
Last-seen Δ
+0.23
+0.23
+0.22
+0.23
Table 5: Historical appearance continuity. DINOv3 crop cosine: no-memory means and policy deltas ( Δ ). Bold: paired story-bootstrap 95% CI excludes zero. N counts eligible events; detected pairs vary by policy. Changed: characters returning after ≥ 30 segments; similarity can reward obsolete looks.
Figure 4: Track-B: where a failure arises , from a stage-by-stage single-author audit of the four retrieval memories in an earlier rollout (pooled, inclusion-weighted conditional shares, not population failure rates). A missed required entity was already wrong in its previous appearance (writing), present there but not retrieved (selection), retrieved but lost in the keyframe or clip (synthesis), or rendered and missed by the judge. Forbidden-render units are reference-associated or judge false alarms; association does not identify a causal reference effect. Per-stratum shares are in Fig. 16 b.
Refs
Segments
ΔR
ΔP
ΔA
1
1,803
+0.087 0.044
-0.002 0.020
-0.012 0.028
2
842
+0.078 0.063
-0.027 0.020
-0.047 0.032
3
149
+0.038 0.092
-0.031 0.067
-0.052 0.104
Table 6: Last-seen is not budget matched. Deltas from no memory by references used. Gray subscripts: full paired story-bootstrap 95% CI widths; bold: CI excludes zero. Different segments per group preclude causal count comparisons.
System
BlenderOpenMovies
LSMDC
Correctness / coverage
Efficiency / redundancy
Budget
Correctness / coverage
Efficiency / redundancy
Budget
Prec ↑
Rec ↑
F1 ↑
SelEff ↑
RV↓
RS↓
Refs
Prec ↑
Rec ↑
F1 ↑
SelEff ↑
RV↓
RS↓
Refs
Retrieval memories (testbed)
Text-frame ret.
0.692
0.887
0.817
0.323
0.492
0.634
15.47
0.630
0.802
0.670
0.330
0.404
0.543
15.94
Seg+uniform ret.
0.662
0.883
0.796
0.297
0.515
0.650
15.47
0.566
0.769
0.622
0.264
0.454
0.605
15.92
Seg+DINO-key ret.
0.579
0.923
0.752
0.431
0.225
0.488
13.80
0.484
0.849
0.591
0.393
0.143
0.393
14.82
Table 7: Track-A reference evidence at each memory’s own budget (Refs per segment). Precision/Recall/F1: correctness and entity coverage; RV/RS : repetition (App. E ). SelEff =Prec(1−RV) . Underline: highest score/lowest repetition. Refs is not ranked.
Metric
No memory
Seg+DINO-key
Imaging
0.645 0.015
0.652 0.023
Aesthetic
0.573 0.013
0.595 0.024
Subject consistency
0.935 0.004
0.929 0.004
Background consistency
0.941 0.002
0.931 0.003
Per-clip entity recall
0.554 0.068
0.740 0.045
Table 8: Clip quality versus entity recall. Same 2,824 clips per condition; means with story-bootstrap 95% CIs; gray subscripts show full widths (upper minus lower). All metrics are higher-is-better. Intervals condition on one rollout per story. Complete conditions and six quality metrics: Supplementary Table 19 .
Judge / backend
ΔR
ΔP
ΔA
Judge check: Wan2.2, 30 stories
Qwen3-VL
+0.133 0.044
-0.057 0.018
-0.114 0.034
InternVL3.5
+0.123 0.049
-0.028 0.023
-0.050 0.032
Cross-judge support
4/4↑
3/4↓
4/4↓
Backend check: Qwen3-VL, 8 matched stories
Wan2.2
+0.150 0.036
-0.061 0.032
-0.130 0.068
Table 9: Judge and backend sensitivity of Seg+DINO-key. Deltas from no memory; gray subscripts give full paired story-bootstrap 95% CI widths, conditional on one rollout per story. Cross-judge support counts policies with same-sign CIs excluding zero. Backend rows retain FLUX.2 keyframes. Limits: Sec. 4.6 .
Figure 5: Track-A reference F1 versus budget. Each curve keeps a memory’s own top-ranked references up to a growing cap. F1 keeps rising with the cap but with shrinking returns, and the ordering holds at every budget: Text-frame leads throughout, and the released memories stay far below. Panels: BlenderOpenMovies (left) and LSMDC (right).
Figure 6: What is hard on Track-A. Continuity recall by entity type (first six columns) and LSMDC event (last four). Darker orange means higher recall on a fixed 0–1 scale. Released memories, below the horizontal line, have lower recall across displayed slices. DINO-key leads nine slices; Text-frame leads LSMDC props.
Figure 7: How the two tracks are built and frozen. Track-A gold starts from a VLM draft of each real film; deterministic normalization and a VLM review route every blocked unit, plus a stratified sample of the rest, to blocking human review before the full validator re-runs and the gold is frozen. Track-B stories are authored: a human writes only the semantic skeleton, code derives every memory label and the prompt stream, and a story is frozen only if both rebuild byte-identically from the skeleton.
Figure 8: From S1 draft to frozen gold. In two Spring segments, the VLM draft describes the action but leaves entities that are on screen unnamed, and the first draft also borrows an action from the next segment. The frozen instruction names every present entity by its roster name, so a memory can be asked for each of them. The footer counts the same kind of change over the whole corpus.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Assessment
What to report or compare
Interpretation boundary
Information contribution
Include no memory with the same non-memory inputs; record what appearance information the prompt and any baseline context supply. Where possible, contrast matched prompts with and without that information (Table 4 ).
Long absence alone does not show memory was needed. Our stripping test does not establish a reliable retrieval benefit or validate a reading gate.
Temporal validity
Report description-matching presence and required state alongside previous-appearance similarity; retain detection/visibility conditions (Table 5 ).
Old resemblance can reward an obsolete look. More recent is not always more valid: flashbacks may require an earlier state.
Unwanted content
Report precision and forbidden-entity avoidance with recall, using the same story requirements (Table 3 ; Sec. 4.3 ).
A recall gain can coincide with extra entities. Whole-policy differences do not isolate a reference’s causal contribution.
Evidence versus use
Inspect selected-reference coverage/precision separately from generated outcomes; use matched instances if relating the two (Table 7 ).
Our Track-A and Track-B use different stories; do not infer a per-story correlation or downstream benefit from Track-A alone.
Perceptual quality
Retain clip-quality metrics and entity-level outcomes with explicit aggregation and denominators (Table 8 ).
Within-clip stability is not entity correctness. Weak measured correlation does not establish independence.
Uncertainty and controls
State rollout/seed counts, resampling units, judge agreement, budget, and whether the generator is shared (Table 9 ).
Story CIs condition on observed rollouts. Judge-sensitive strata, variable-budget Last-seen, and native-generator rows require separate interpretation.
Appendix
Table 10: A checklist for evaluating visual-memory utility. Actions motivated by existing findings, not new experimental results. The checks distinguish information contribution, temporal validity, and unwanted content while retaining evidence-stage, quality, and uncertainty controls.
Figure 9: Anatomy of a Track-A film ( Spring , Blender Studio, CC BY 4.0). Top: five story segments with the instruction gn the system receives (English excerpts of the Chinese gold). Bottom, on grey: scoring-only gold for ten of the film’s eleven entities over segments 0–39 (the rest are titles and credits): where each entity is on screen, its state events, and the per-segment diagnostic tags. The grey puppy leaves after segment 10 and returns in segment 15, the case shown in Fig. 12 .
Figure 10: How a Track-B script plants memory stresses. Story 0001_lighthouse_keeper (87 five-second segments, 19 scenes) drawn as a timeline. The system receives only the prompt stream (top; three prompts quoted verbatim); everything in the grey panel is scoring-only gold. Each lane is one of the story’s 19 entities: shade gives its authored state, hatching marks segments where it must be absent (after it is destroyed or dies, or while only its look-alike twin is on screen), open circles mark prompts that still name a removed entity, and diamonds mark newly introduced look-alikes; the bottom rows tick the segments tagged with each family of Table 1 , with per-family counts at right.
Cohort property
Value
Stories / segments
30 / 2,824
Segments: mean / median
94.1 / 87.5
Segments: min–max
68–131
Scripted duration
3.92 h
Character entities
152
Prop entities
300
Appendix
Table 11: Coverage of the evaluated Track-B cohort. Only the 30 stories used in all six seed-2026 conditions are counted. Entity counts are story-local records. Right: tagged segments, number of stories containing the tag, and per-story count range. Tags overlap and are not scoring denominators or observed generation success rates.
Work
Historical visual context
Learned conditioning
Reported evaluation focus
OneStory [ 1 ]
Semantically selected frames from earlier shots; compact global memory.
Fine-tuned I2V model with frame selection and an adaptive patch conditioner.
Character/environment consistency, shot quality, and semantic alignment in text- and image-conditioned multi-shot generation.
StoryMem [ 25 ]
Generated-shot keyframes selected semantically and filtered by aesthetic preference; dynamic memory bank.
LoRA-trained memory-to-video model using latent concatenation and negative temporal RoPE shifts.
ST-Bench: cross-shot consistency, global/single-shot prompt following, aesthetic quality, and human preference.
SubjectAnchor [ 17 ]
Subject-related precomputed keyframes traced to historical appearances.
Trained Wan2.2-I2V with subject-aware RoPE and partitioned memory attention.
Subject/prompt alignment, identity relative to first/previous appearances, visual quality, and inclusion/exclusion control.
VMem-Bench (this study)
Real-film evidence replay (Track-A); references from each policy’s generated history (Track-B).
Shared FLUX.2/Wan2.2 pipeline across retrieval policies and controls; no newly trained memory architecture.
Reference coverage/precision before generation; required and forbidden appearances, current state, conditional appearance similarity, prompt stripping, and judge/backend sensitivity.
Appendix
Table 12: Scope of neighboring visual-memory work. Designs and evaluation goals reported in the cited papers, rather than scores under a shared protocol. SubjectAnchor explicitly includes inclusion/exclusion control; our distinction concerns evidence-stage diagnostics, current-state scoring, and prompt-information sensitivity. The final row describes the controlled retrieval study, not a newly trained generator.
Memory family
Internal interface / native diagnostic
What internal-only evaluation cannot establish
Window / KV
frames, latents, or KV; recency / native attention; cache reach
Retention does not imply that intent-relevant evidence crosses the finite conditioning boundary; stale history may remain active.
Larger K can buy recall while increasing context spend or contamination; a good retrieval decision may still fail in synthesis.
Entity / state memory
persistent identity / state records; address, update, filter; record validity
Correct records do not guarantee that the right visual evidence is materialized under budget or realized correctly by the generator.
Appendix
Table 13: Different internal memories, one common interface. The families expose different internal states and native diagnostics, so those quantities are not directly comparable and do not by themselves establish what reaches or survives generation. VMem-Bench maps them to common observables: boundary context Cn before synthesis (Track-A) and generated outcome xn after synthesis (Track-B).
System
Stored as
Read by
Forgets by
Causal
Released / published memories
FramePack [ 26 ]
Frame latents, compressed by importance rank
All history, ordered by recency or similarity
Geometric compression; tail deleted or pooled
Partly (endpoint plans)
MemFlow [ 7 ]
Local KV window + KV bank
Prompt-to-frame attention, top- k
Budget; keeps top- k by prompt relevance
Yes
IAMFlow [ 8 ]
Entity-ID registry + archived frame KV
Set cover of IDs named in prompt
None; archive grows, read capped
Yes
LongLive-RAG [ 4 ]
Evicted latent KV + AE embedding
Cosine to latest latent, top- K
None; every evicted frame kept
Yes
Context-as-Memory [ 23 ]
Raw generated frames
Camera FOV overlap with target view
None; read capped at k
Yes
Appendix
Table 14: How causal video memories read and forget. Every read matches the current instruction, latest frame, camera pose, or attention query by similarity or relevance, and forgetting is either absent, a budget eviction ranked by that same relevance, or age-based compression; no causal system evicts or withholds an entry because the story changed its state. Only IAMFlow and the non-causal VideoMemory keep explicit entity records, and both address them by the entities named in the current instruction or shot.
Movie
Segs.
Mean seg.
KV frames
Pred. GiB
Outcome
0053_Rendezvous_mit_Joe_Black
221
7.3 s
2071
555
completed
sita_sings_the_blues_part1
164
14.9 s
3259
873
OOM
0013_Halloween
128
14.9 s
2442
654
completed
1005_Signs
296
9.8 s
3763
1008
OOM
0017_Pianist
235
13.8 s
4287
1148
OOM
Appendix
Table 15: IAMFlow host-memory growth observed during our Track-A scalability test. The limiting quantity is the number of archived KV frames, which scales with total observed video duration rather than simply with the number of benchmark segments. The predicted GiB column is the archived-KV tensor payload from shape alone; OOM rows exceed the fixed host-memory budget before a full-movie scored output can be produced.
Metric
pos./neg. pairs
AUC
threshold
pos. median
neg. median
DINOv3 crop
385/389
1.000
0.67
0.96
0.19
ArcFace face
201/157
0.997
0.35
0.84
0.11
MegaLoc scene
200/200
1.000
0.66
0.89
0.07
DINOv3 scene
200/200
1.000
0.84
0.98
0.17
VLM pairwise judge
same when both visible: 0.98 (pos.), 0.78 (neg.); neg. not consistent: 0.93
Appendix
Table 16: Calibration of the identity-consistency metrics. Positive controls pair two disjoint frame sets of the same clip; negative controls pair two different entities of the same kind and story. For the embeddings, the threshold is the Youden-optimal cut on these controls, fixed before any condition was scored. The VLM judge answers the same question twice with 0.993–1.000 exact agreement.
Figure 11: Track-B judge verdicts checked against human labels on real units. Each row shows the entity’s previous appearance in the same rollout, three frames of the generated clip, the verdict of the pinned Qwen3-VL-32B judge, and the human label for the clip. In rows 1, 4 and 5 the judge is right about a required entity being present or absent and about a forbidden entity being rendered; 28 of 32 judge-confirmed controls were confirmed. In rows 2 and 3 the judge reports a required entity as missing although it is clearly rendered; such misses make up 27% of sampled required-entity failures and concentrate at long gaps and time references, so Track-B recall there is conservative.
Figure 12: Every reference two memories hand over for Spring segment 15. Text-frame returns 16 near-identical views of the girl in the forest and never the puppy, last seen five segments earlier. Seg+DINO-key samples diverse views, finds the puppy on the cloud ridge (74 s), and covers all four entities, but five of its 15 references show none of them. Letters are the automatic judge’s labels; it also tags some misty-forest frames before 144 s as the nest, which does not change which references count as correct here.
Figure 13: Track-A memory F1 versus archived-video length (same runs as Table 7 ). Retrieval remains comparatively stable as history grows; the released causal-memory paths lose more F1. The far bins contain few films. Left: BlenderOpenMovies; right: LSMDC.
Figure 14: Track-B rollouts on two memory probes: memory brings back a recently seen character, and also a dead one. Rows are systems given the same prompt stream; columns show the entity’s previous appearance and the probe segment (middle frame of each generated clip). In (a) the no-memory generator draws a stranger for “he” while all four retrieval memories restore the ranger; in (b) no memory keeps the dead guard out, but all four retrieval memories, MemFlow, and IAMFlow put him back at his plaque. Badges are the automatic judge’s verdicts for that entity, and Track-B Recall and Avoid-OK pool such verdicts over all 2,824 segments; the released systems use their own generators.
Figure 15: Track-B generated-video scores (same 30 -story pool as Table 3 ). (a) Continuity recall by absence-gap bin; the dotted line is the same generator with no memory and the grey dashed line with last-seen references. Support is N=701/293/210 for near/mid/far. (b) Pooled Recall, Precision, F1, and Avoid-OK.
Kind
Gap (segments)
n
No memory
Text
Uniform
DINO-key
Frame
Last-seen
Character
just seen
2173
0.493
+0.106 0.07
+0.198 0.06
+0.201 0.06
+0.191 0.08
+0.143 0.07
1–4
430
0.553
+0.144 0.13
+0.186 0.12
+0.195 0.12
+0.170 0.12
+0.040 0.12
5–29
63
0.714
-0.048 0.24
+0.143 0.23
+0.111 0.25
+0.064 0.28
-0.016 0.22
≥ 30 (returns changed)
62
0.984
-0.177 0.21
-0.129 0.18
-0.032 0.13
-0.081 0.16
-0.032 0.08
Prop
just seen
1087
0.399
+0.070 0.06
+0.118 0.09
+0.111 0.08
+0.085 0.09
+0.049 0.06
1–4
271
0.635
+0.015 0.09
+0.044 0.11
+0.041 0.11
+0.044 0.09
+0.096 0.11
Appendix
Table 17: Track-B recall by entity kind and absence gap. No-memory recall and each condition minus no memory (paired story-bootstrap 95% CI; gray subscripts show full CI widths (upper minus lower); bold when it excludes zero). Just seen means the entity was on screen in the previous segment. Characters are named only by name after their first appearance; 61 of the 62 character returns after 30 or more segments come back changed, with the new look in the instruction. Props often carry their look in the name. Text, Uniform, DINO-key, and Frame are the four retrieval memories.
Stratum
Eligible N
No memory
Text Δ
Uniform Δ
DINO-key Δ
Frame Δ
Last-seen Δ
DINOv3 crop similarity (trackB-embed-1.0)
All returns
1130
0.35
+0.15
+0.21
+0.20
+0.21
+0.23
Characters
486
0.27
+0.13
+0.20
+0.19
+0.18
+0.23
Props
644
0.41
+0.16
+0.21
+0.21
+0.23
+0.22
Gap 1–4
688
0.30
+0.15
+0.22
+0.21
+0.21
+0.25
Gap 5–29
293
0.36
+0.16
+0.19
+0.19
+0.19
+0.17
Appendix
Table 18: Track-B identity consistency. No-memory value and each condition minus no memory; N counts eligible events, while policy-specific detected/visible subsets determine the analyzed means (bold: paired story-bootstrap 95% CI excludes zero). Embeddings: mean cosine similarity between the previous and current appearance (DINOv3 on the localized entity; MegaLoc on the whole frame for locations), over returns detected on both sides. VLM: share judged the same individual among returns visible in both clips (lenient, Table 16 ). Returns are character and prop re-appearances after at least one segment away; returns changed are characters whose look changed after 30 or more segments, for which a higher similarity means the return looks more like its old look.
Figure 16: What memory changes, and where failures arise. (a) Track-B rate per memory stress for the shared generator with no memory (open circle) and with each retrieval memory (filled), with last-seen references as grey crosses, grouped by the decision each stress tests (Sec. 4 ); the gap rows mix entity kinds (Sec. 4.2 ). A marker right of the circle means memory helps; left means it hurts; the right column gives memory minus no memory (* when all four paired 95% CIs exclude zero). Memory helps for recent returns and persistent states and hurts where an entity must stay out. (b) Human attribution of Track-B failures (App. G ), by stratum: the share already wrong in the entity’s previous appearance, present there but not retrieved, retrieved but lost in the keyframe, or actually rendered and missed by the judge. The last row splits judge-flagged forbidden renders into those carried in by the references (purple) and judge false alarms (light purple). The two aggregate rows are inclusion-weighted.
Condition
Imaging ↑
Aesthetic ↑
Subj. cons. ↑
Bg. cons. ↑
Dynamic ↑
Motion sm. ↑
Clip rec. ↑
Text-frame
0.634 0.024
0.584 0.022
0.926 0.004
0.930 0.004
0.521 0.058
0.9933 0.0005
0.672 0.049
Seg+uniform
0.660 0.022
0.596 0.024
0.929 0.005
0.931 0.003
0.485 0.071
0.9931 0.0006
0.735 0.042
Seg+DINO-key
0.652 0.023
0.595 0.024
0.929 0.004
0.931 0.003
0.493 0.073
0.9931 0.0005
0.740 0.045
Seg+frame
0.652 0.024
0.595 0.023
0.928 0.005
0.931 0.003
0.475 0.060
0.9933 0.0005
0.731 0.035
No memory ( K=0 )
0.645 0.015
0.573 0.013
0.935 0.004
0.941 0.002
0.549 0.041
0.9932 0.0004
0.554 0.068
Appendix
Table 19: Per-clip VBench-style quality on the shared-generator testbed (all 2,824 clips per condition). Each cell gives the mean with its story-clustered bootstrap 95% CI; gray subscripts show full widths (upper minus lower); Clip rec. is the mean per-clip entity recall of the frozen judge on the same clips. Best in each column is underlined .
(a) Minus no memory, segments with a parsable judge output in both arms
Condition
dropped
Recall
Precision
Avoid-OK
1–4
5–29
≥ 30
Text-frame
129
+0.075 0.05
-0.041 0.02
-0.073 0.03
+0.105 0.09
+0.015 0.10
-0.067 0.13
Seg+uniform
138
+0.139 0.05
-0.064 0.03
-0.128 0.04
+0.154 0.10
+0.052 0.09
-0.014 0.14
Seg+DINO-key
143
+0.149 0.05
-0.057 0.02
-0.118 0.04
+0.152 0.10
+0.050 0.10
+0.029 0.11
Seg+frame
142
+0.131 0.06
-0.058 0.02
-0.114 0.05
+0.143 0.10
+0.054 0.09
-0.014 0.11
Last-seen refs
112
+0.087 0.04
-0.013 0.01
-0.028 0.02
+0.066 0.09
-0.004 0.09
+0.000 0.10
Appendix
Table 20: Robustness of the Track-B memory effects. Differences to no memory with paired story-bootstrap 95% CIs; gray subscripts show full CI widths (upper minus lower); bold when the CI excludes zero. (a) Dropping every segment on which either arm’s judge output failed to parse leaves all main effects in place; the three segment memories also gain significantly at 5–29 segments. (b) Last-seen effects are stratified by reference count; groups contain different segments, so this is not a count intervention.
Condition
Recall
Precision
Avoid-OK
All A/ κ
Present A/ κ
Qwen3-VL-32B (pinned judge)
No memory
0.559
0.977
0.968
0.908/0.64
0.754/0.52
Text-frame
+0.068 0.04
-0.039 0.02
-0.070 0.02
0.882/0.64
0.723/0.46
Seg+uniform
+0.130 0.05
-0.062 0.02
-0.123 0.05
0.882/0.67
0.727/0.45
Seg+DINO-key
+0.133 0.04
-0.057 0.02
-0.114 0.03
0.880/0.66
0.713/0.43
Seg+frame
+0.115 0.06
-0.057 0.02
-0.110 0.04
0.881/0.66
0.721/0.44
Appendix
Table 21: Track-B memory effects under a second judge (the same videos, 30 stories, same roster, prompts, and clips). Differences to no memory with paired story-bootstrap 95% CI, gray subscripts show full CI widths (upper minus lower); bold when it excludes zero. Agreement / κ is shown for the full roster and required-present entities.
Condition
Recall
Precision
Avoid-OK
Near-dup
FLUX.2 → Wan2.2 (main pipeline)
No memory (level)
0.563
0.979
0.969
0/759
Text-frame
+0.084 0.05
-0.034 0.03
-0.067 0.06
0/759
Seg+uniform
+0.153 0.06
-0.061 0.04
-0.130 0.08
0/759
Seg+DINO-key
+0.150 0.04
-0.061 0.03
-0.130 0.07
0/759
Seg+frame
+0.115 0.11
-0.067 0.03
-0.136 0.05
0/759
Appendix
Table 22: Track-B memory effects on two MiniMax-H3 realizations (the same 8 stories, 767 segments per condition, seed 2026, same judge). Differences to no memory with paired story-bootstrap 95% CI, gray subscripts show full CI widths (upper minus lower); bold when it excludes zero. Near-dup: adjacent segments whose middle frames nearly coincide (a symptom of the video model copying its references instead of following the instruction).
Required entity missing or wrong
Forbidden entity rendered
Memory
Hist.
Select.
Synth.
Judge
n
Context
Gen.
Judge
n
Text-frame
45
28
1
27
36
60
0
40
10
Seg+uniform
34
35
9
22
38
100
0
0
9
Seg+DINO-key
13
44
18
25
35
56
0
44
9
Seg+frame
25
38
2
34
38
67
0
33
9
Pooled
29
36
7
27
147
71
0
29
37
Appendix
Table 23: Where closed-loop failures arise , by retrieval memory (%, inclusion-weighted). Left: judge-flagged failures of a required entity, by the stage at which it is lost: already wrong in its earlier appearance (Hist., writing), in the history but not retrieved (Select.), in the references but lost in the keyframe or clip (Synth.), or in fact rendered (Judge). Right: judge-flagged forbidden renders: also visible in the references (Context), visible only in the output (Gen.), or not rendered (Judge). These are observational stage labels, not a causal decomposition. Unresolved units excluded; pooled CIs in App. G.1 .
Miss (required entity absent)
Memory
gap 1–4
5–29
≥ 30
time ref.
Wrong
Forbidden
Control
Units
Text-frame
8 (140)
8 (56)
8 (20)
4 (8)
10 (109)
10 (201)
8 (596)
56
Seg+uniform
8 (155)
8 (63)
8 (12)
4 (6)
10 (121)
10 (265)
8 (582)
56
Seg+DINO-key
8 (116)
10 (52)
8 (18)
2 (2)
10 (117)
10 (265)
8 (639)
56
Seg+frame
8 (109)
8 (42)
8 (20)
4 (5)
10 (111)
10 (274)
8 (632)
56
All
32 (520)
34 (213)
32 (70)
14 (21)
40 (458)
40 (1005)
32 (2449)
224
Appendix
Table 24: Frozen author-audit sample. Units per retrieval memory and unit type, with the size of the judge-flagged pool each was drawn from in parentheses; the unequal draws are why shares are inclusion-weighted. Strata with too few candidates fall back to a neighbouring stratum (time references and look-alike errors are pooled with mid-gap misses and state errors). The 224 units are shown with 22 hidden repeats (246 scored items) after five unscored practice items, 251 items in all.
Decision
Observed evidence
Hypothesis to test
Result counting against it
When to read
Prompt stripping changes no-memory presence substantially; the stripped retrieval gain is uncertain (Table 4 ).
Gate access when history adds information beyond the prompt and baseline context, compared with always reading.
Gating loses required appearances or valid-state fidelity without reducing interference; textual detail alone is an inadequate gate.
Which moment
Changed returns can become more similar to their obsolete look (Table 5 ).
Retrieve the instruction-valid entity state, allowing earlier states for flashbacks.
State-matched references do not improve requested-state correctness or damage identity continuity at the same budget.
How much
Last-seen costs increase across unmatched reference-count strata (Table 20 ).
Fewer references reduce interference without losing needed evidence; vary count using explicitly specified additions.
The apparent cost increase disappears under the matched comparison; additional references improve recall without an exclusion cost.
What to forget
Retrieval raises recall while reducing precision and avoidance (Sec. 4.3 ).
Suppress unrequested content in useful references while retaining required identity/state cues.
Suppression fails to improve avoidance or loses useful identity/state evidence; removing more content is not automatically better.
What to write
The earlier single-author audit assigns 29% of sampled flagged misses to stored wrong renderings (App. G ).
Reject or repair incorrect observations before write-back.
Rejecting history discards uniquely useful evidence and worsens later recall/state fidelity compared with retaining it.
Appendix
Table 25: From observations to falsifiable design hypotheses. Proposed interventions have not been implemented or validated here. Fix generation prefix, prompt, generator state, and seed. Match count for state/filtering tests; reading gates and budget tests intentionally vary access/count and must specify removed or added references. For write policies, match the prefix before write-back and follow the resulting trajectories. A seed match alone does not fix diverged histories.
Memory-based long-video generation aims to combine user control with long-range visual consistency. We organize this process into three functional components: a planner, an adapter, and a generator. Representative multi-shot systems couple memory to generator-specific interfaces or use predefined agent workflows, rather than independently training a memory policy for cross-shot objectives. We introduce REEL, a general-purpose visual-memory planner. Through a unified tool interface, REEL learns to retrieve references, create missing visual evidence, and update persistent memory. Backend-specific adapters translate its evidence plans into supported conditioning inputs, allowing the same policy to guide different black-box video generators. To train REEL, we construct two high-quality datasets, REEL-SFT-4k and REEL-RL-1k, covering diverse interaction patterns and prompt types. Training proceeds in three stages: supervised fine-tuning, single-shot Group Relative Policy Optimization (GRPO), and multi-shot GRPO. Multi-shot GRPO carries policy-generated memory across consecutive shots, so later outcomes guide earlier reference selection and memory updates. REEL-Bench tests identity preservation and intended state changes under text-only requests and timed reference-image uploads. On JoyAI-Echo~1.5 and MiniMax-H3, REEL improves cross-shot consistency on ST-Bench by 15.1% and 20.7%, and Core-100 on our proposed REEL-Bench by 2.41 and 8.32 points, respectively, over the no-memory baseline.
Hang Xu, Guohui Zhang, Xiaoxiao Ma +7
University of Science and Technology of China · JD Explore Academy
Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.
Long autoregressive video generation faces a fundamental memory challenge: with a finite attention window, a model must decide which information from an ever-expanding history to retain. Existing methods organize memory temporally, preserving recent frames while compressing or discarding older ones. We instead propose RECAP-Forcing, organizing memory by appearance novelty. A long video is not merely a sequence of frames, but an evolving cast of subjects, objects, and scenes whose identities must remain consistent over time. We organize memory by retaining the KV cache associated with newly appearing content--such as entering subjects, disoccluded regions, and newly introduced scenes--at the moment it first becomes visible, prioritizing novelty over recency. Memory should scale with the amount of newly introduced content, rather than with video length. This appearance-indexed memory makes long-range consistency an explicit property of the memory structure. Our framework unifies two mechanisms under this single principle. At the beginning of a video, when all visible content is novel, an attention sink preserves the initial scene. As the video evolves, an optical-flow-based novelty bank extends the same principle by selectively retaining newly revealed content. As a training-free inference method with no additional learnable parameters, RECAP-Forcing consistently improves visual quality and semantic fidelity across multiple strong baselines and outperforms existing memory methods.