Memory for long-horizon video generation must satisfy two competing requirements: it must remain bounded so generation can continue indefinitely, while still recovering what was observed at a place when the camera returns, regardless of how long ago it was seen. Reconstruction-based memories provide spatial recall by caching and reprojecting explicit geometry, but require growing storage and depend on estimated 3D structure. Recurrent memories keep a constant-size state, but retention is governed by temporal recurrence, causing old observations to fade with time. We introduce SMART, a slot memory addressed by region tokens, which combines bounded storage with spatially addressed recall. SMART stores content in a constant-size slot recurrence whose writes are addressed by the observing camera and whose reads are queried by the rendering camera, without constructing an explicit 3D representation or maintaining a growing cache. Consequently, recall is conditioned on inter-view geometry rather than on recency. Each read additionally provides a confidence estimate that conditions a chunk-causal diffusion transformer, allowing it to selectively trust retrieved content while relying on its generative prior elsewhere. On long trajectories in Memory Maze and Minecraft, SMART maintains high-fidelity recall far beyond its training horizon while using constant memory and per-step cost.
Figures & tables
Figure 1: Long-horizon video generation. Top: on Memory Maze , Smart and DFoT ( Song et al., 2025 ) observe a forward trajectory and generate the return leg, revisiting places seen long before. Bottom: on Minecraft , Smart and MilliVid ( Chandratreya et al., 2026 ) generate a continuation trajectory from 256 context frames written into the memory. Smart retains observed content more faithfully than the baselines at constant per-step cost.
Figure 2: Smart inference overview.
Figure 3: Tiled tokenization. Write token j queries only the patches of tile j .
Figure 4: Slot read and write. Address routes the write; Read is a cross-attention over slot states.
Figure 5: Smart memory module. Write and read paths of the slot memory.
Figure 6: Qualitative comparison on Minecraft . Generated frames for each method in Table 2 , on the same trajectory, at the 2048 frame horizon with 256 initial context frames.
Method
(201+21)+201=423 frames
(401+21)+401=823 frames
PFLOPs ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
FVD ↓
PFLOPs ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
FVD ↓
Causal (100F)
0.096
25.0
0.809
0.131
6.39
4.06
0.193
21.8
0.739
0.198
8.85
7.19
Causal (Full)
0.117
30.0
0.891
0.034
5.36
2.13
0.299
24.3
0.789
0.100
7.96
4.61
GeoCache (GT-D)
0.454
24.6
0.811
0.098
6.25
3.57
0.936
24.6
0.810
0.095
6.03
2.52
DFoT
0.566
32.0
0.919
0.030
5.74
1.89
1.130
28.4
0.865
0.079
5.54
1.89
Smart
0.108
31.1
0.919
0.040
5.22
2.25
0.213
29.9
0.900
0.046
5.08
1.83
Table 1: 3D consistent video generation on Memory Maze . Methods generate the reverse leg from the observed forward trajectory in parentheses. Column groups are total trajectory length. FID pools a band’s frames, FVD its 16 -frame clips.
Method
1–64 frames
65–256 frames
PFLOPs ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
FVD ↓
PFLOPs ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
FVD ↓
Causal (7F)
0.72
17.18
0.583
0.325
43.36
64.37
2.62
11.67
0.437
0.551
47.78
182.03
FramePack
1.71
19.13
0.612
0.260
44.66
56.67
6.62
14.04
0.498
0.448
48.67
112.84
MilliVid
2.40
21.16
0.615
0.196
44.68
47.83
8.95
18.43
0.562
0.284
43.34
44.28
Smart
0.91
21.96
0.647
0.187
44.60
45.86
3.46
18.54
0.571
0.294
44.01
56.00
Table 2: 3D consistent video generation on Minecraft . Methods generate frames 257 – 2048 from 256 ground-truth context frames. Column groups are distance past the context. FID pools a band’s frames, FVD its 16 -frame clips.
Figure 7: What the slot memory stores, and how a read recovers it. Smart -SM on 401 -frame maze episodes: the forward leg is written, the reverse leg read from the state. Write. (a) Euclidean distance between two forward frame positions and (b) the probability that their region tokens land in the same slot. Read. (d) Mean read mass per slot against the distance from the query to the content that slot currently holds where content position is the decayed average of all content written to respective slot. (e, f) Attribution of each reverse read to the forward frames in memory slots where the white trace is the pose-nearest forward frame.
Variant
401 frames
1601 frames
3201 frames
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Smart-Slot Memory
32.7
0.940
0.024
32.4
0.936
0.024
31.9
0.931
0.026
No SE(3) Value Warping
31.5
0.924
0.031
30.0
0.906
0.041
29.6
0.900
0.045
Smart-Dense Memory
32.9
0.939
0.020
26.5
0.852
0.074
21.5
0.756
0.191
Scaling Down Heads (/2)
32.2
0.935
0.028
24.5
0.810
0.146
22.9
0.772
0.192
Table 3: Memory Only ablation results. Reconstruction quality on Memory Maze , changing one component of Smart at a time. Smart applies the SE(3) value transform to the 64 value channels of the head. All models are trained on 401 frame episodes.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Evaluation trajectories. Top-down view of one episode per dataset: (a) a Memory Maze forward–turn–return rollout, and (b) a Minecraft continuation rollout.
Figure 9: Attention masks. Allowed attention between query and key chunks for the generators in this work: (a) the bidirectional pretraining of Section A.4 , (b) the full-history causal DiT of the Causal (Full) arm, (c) the windowed chunk-causal attention of Smart , in which a generation chunk reads the W clean frames before it and the memory’s draft. Anchor tokens are the clean frame 0 .
Memory Maze
Minecraft
Backbone
Wan2.1-style DiT ( Wan et al., 2025 )
DiT-B ( Peebles and Xie, 2023 ; Chandratreya et al., 2026 )
Parameters
250 M
130 M
Blocks × width × heads
14×1,024×8
12×768×12
Control signal
camera pose (PRoPE)
discrete action for modulation
Positional code
temporal and 2D rotary, camera PRoPE
sinusoidal over frame, row and column
Autoencoder
Wan2.1 VAE, finetuned
MilliVid hierarchical autoencoder, pretrained
Appendix
Table 4: Instantiation of Smart on the two datasets. The memory interface, chunk size and context window are shared; everything else follows the backbone each dataset inherits.
Variant
401 frames
1601 frames
3201 frames
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Smart-Slot Memory
32.7
0.940
0.024
32.4
0.936
0.024
31.9
0.931
0.026
Scaling Up Memory Slots (×2)
33.0
0.943
0.023
33.3
0.947
0.020
33.2
0.946
0.020
Appendix
Table 5: Memory capacity. Reconstruction quality of the memory alone on Memory Maze , trained on 401 -frame episodes. Doubling the memory parameters gains 0.3 dB at 401 frames and 1.3 dB at 3,201 .
Method
(25+21)+25=71 frames
(51+21)+51=123 frames
(101+21)+101=223 frames
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
StateSpaceDiffuser
28.5
0.874
0.145
24.8
0.806
0.177
21.0
0.717
0.290
Smart
30.2
0.906
0.108
32.3
0.931
0.073
33.3
0.939
0.039
Appendix
Table 6: Comparison against StateSpaceDiffuser ( Savov et al., 2026 ) reimplementation on Memory Maze . Both generators share the DiT architecture and draft conditioning differing in how the memory is conditioned and read. Column groups are total trajectory length. PSNR and SSIM are computed over each target frame that were visible during the forward leg and averaged per frame.
Figure 10: Qualitative comparison on Memory Maze. Generated frames for each method in Table 1 , on the same trajectory with 823 frame context frame and 801 target frame generation.
Figure 11: Additional qualitative comparisons on Minecraft . Generated frames for each method in Table 2 , on the same trajectory, at the 2048 frame horizon with 256 initial context frames. Note that only Smart faithfully retrieves the position of the sun.
Figure 12: Additional qualitative comparisons on Minecraft .
Figure 13: Additional qualitative comparisons on Minecraft .
Long-horizon video generation requires retaining an ever-expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual-pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench-Long overall score of 80.1 and competitive static-scene revisitation performance on R2M-Bench. It also leads on all four consistency metrics for off-screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.
Yukuo Ma, Cong Liu, Junke Wang +4
Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom
Long-video generation and world models have shown strong potential for interactive entertainment and embodied simulation by predicting future observations conditioned on user actions and historical memory. However, as memory sequences grow longer and their structures become increasingly complex, managing long-range spatial context becomes increasingly challenging, calling for a more intelligent and systematic memory-management strategy. Building on the advancing spatial reasoning capabilities of multimodal large language models (MLLMs) and the broader vision of unified models, we propose Spatial Memory Intelligence (SMI), the first framework to systematically employ an understanding model for spatial-memory management in long-video world models. SMI introduces four coordinated atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. Extensive experiments across multiple baselines, benchmarks, and world-model backbones demonstrate the effectiveness and generalizability of SMI, achieving comprehensive improvements in memory sparsity, generation stability, and spatial consistency.
Ying Yang, Guiyu Zhang, Lianghua Huang +5
The Chinese University of Hong Kong, Shenzhen · Alibaba Group · Nanjing University +3
Streaming video generation models typically rely on temporal-centric memory, which organizes historical context as raw frames, chunk segments, or unclustered tokens. This organization frequently leads to identity drift and semantic inconsistency when entities exit the frame or during interactive prompt transitions. To address these limitations, we propose SlotMemory, an object-centric Key-Value memory mechanism for streaming video diffusion. Our approach shifts the memory abstraction from "when" an event occurred to "what" is being represented by decomposing the transformer's key-value manifold into discrete, reusable semantic slots. By utilizing these slots as routing addresses to index and store high-fidelity key-value tokens, we enable entity-level persistence and prompt-aware retrieval across long horizons. Evaluated on 60-second interactive narratives using the Wan2.1-T2V-1.3B backbone, SlotMemory achieves a state-of-the-art quality score of 81.61 and a 22.8 percent relative improvement in dynamic consistency over the strongest existing streaming baseline. Our results demonstrate that structured semantic representation, rather than raw temporal capacity, is the essential primitive for persistent long-form video synthesis. Our codes and checkpoints are available at https://tj12323.github.io/SlotMemory/.
Weijia Dou, Hui Li, Jiahao Cui +3
Fudan University · Meta Superintelligence Labs · Baidu