Organizations: East China Normal University · Shanghai Artificial Intelligence Laboratory · Fudan University · Beihang University · The Chinese University of Hong Kong
Long-horizon consistency is essential for video world models, yet it becomes difficult to maintain once observed content falls outside the context window. Existing memory mechanisms either retrieve history as implicit context, which remains constrained by context capacity and entangles static and dynamic information or consolidate history into explicit global representations, which struggle to model evolving dynamics and accumulate geometric errors. To overcome these limitations, we introduce GROME, a novel factorized memory framework that augments video world models for long-horizon coherence across both static scenes and dynamic entities. GROME operates via three synergistic components: 1)Static-Dynamic Factorization decouples historical evidence into static and dynamic components, preserving static scene geometry and dynamic entity states separately; 2)Local Memory Retrieval selects spatially aligned static history to eliminate geometric drift alongside the latest dynamic entity states to ensure temporal continuity; 3)~Training-Free Memory Injection seamlessly grounds the factorized memory into video world models via their native conditioning pathways. Experiments on long-horizon revisit scenarios demonstrate that GROME substantially enhances scene persistence and dynamic entity fidelity over existing methods, all while preserving the visual quality and controllability of the frozen backbone without additional training. Beyond self-rollouts, GROME readily accommodates external observations as plug-and-play memory, unlocking flexible and controllable video continuation and scene exploration.
Figures & tables
Figure 1: (1) During autoregressive rollout, GROME progressively updates historical observations into a factorized memory and injects retrieved local evidence into a frozen video world model. (2) By decoupling static scene structures from dynamic entities, it ensures scene persistence while accommodating entity dynamics. (3) GROME also supports lifting real-world videos into factorized memory as plug-and-play references for controllable video continuation. (4) Multiple point clouds can be seamlessly composed as unified external memory for open-ended spatial exploration.
Figure 2: Comparison of memory paradigms. Implicit memory suffers from substantial scene-content overlap and ambiguous in dynamic states, while explicit memory consolidates historical frames into a global 3D scene but struggles with evolving dynamic entities and accumulated geometric drift. GROME instead factorizes historical frames into static-dynamic memory, preserving both persistent static scene structure and evolving dynamic entities over long-horizon generation.
Figure 3: Overview of GROME . GROME factorizes historical evidence into static scene and dynamic entity components. Both are retained over extended rollouts as local spatial memories, with the dynamic component further organized by explicit temporal indexing. Given a target camera trajectory, relevant historical evidence is separately retrieved from the static and dynamic components to render references. These static-dynamic references are then injected into the model’s native history-conditioning pathway through a set of training-free injection strategies, conditioning generation together with the local context and user actions. This allows GROME to better preserve consistency over long rollouts while maintaining generation quality and controllability.
Figure 4: Qualitative comparison. GROME better preserves revisit scene structure and entity consistency simultaneously across diverse revisit trajectories compared with existing world models.
Revisit Consistency
Visual Quality
Dynamic Quality
Controllability
Model
PSNR ↑
SSIM ↑
LPIPS ↓
CLIP Aesthetic ↑
Subjective Qual. ↑
Entity Cons. ↑
Motion Acc. ↑
Trans. Err. ↓
Rot. Err. ( ∘ ) ↓
Prompt Cons. ↑
Static
Gen3C
21.725 ± 0.5582
0.749 ± 0.0203
0.173 ± 0.0140
5.334 ± 0.0874
0.469 ± 0.0264
N/A
N/A
0.420 ± 0.0513
3.878 ± 0.6494
0.258 ± 0.0093
VMem
15.784 ± 0.5323
0.460 ± 0.0221
0.367 ± 0.0132
5.089 ± 0.0760
0.338 ± 0.0230
N/A
N/A
0.624 ± 0.4502
7.187 ± 1.0210
0.251 ± 0.0121
WAH
16.911 ± 0.6472
0.540 ± 0.0320
0.369 ± 0.0266
5.116 ± 0.0739
0.347 ± 0.0308
N/A
N/A
2.141 ± 0.7641
6.709 ± 1.3273
0.231 ± 0.0104
WorldPlay
16.087 ± 0.4921
0.465 ± 0.0236
0.289 ± 0.0139
5.265 ± 0.0869
0.404 ± 0.0265
N/A
N/A
0.471 ± 0.0504
6.391 ± 0.9905
0.260 ± 0.0090
Table 1: Quantitative comparison on static and dynamic long-horizon revisit trajectories. LingBotV2 RC denotes LingBotV2 with raw-context conditioning, while LingBotV2 TR denotes LingBotV2 with temporal retrieval. Values are reported as mean ± the half-width of the 95% paired bootstrap confidence intervals. Best and second-best values are shown in bold and underlined .
Revisit Consistency
Visual Quality
Dynamic Quality
Controllability
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
CLIP Aesthetic ↑
Subjective Qual. ↑
Entity Cons. ↑
Motion Acc. ↑
Trans. Err. ↓
Rot. Err. ( ∘ ) ↓
Prompt Cons. ↑
LingBotV2 RC
15.505 ± 3.1742
0.456 ± 0.0418
0.500 ± 0.0147
5.247 ± 0.2950
0.572 ± 0.0875
0.711 ± 0.0482
16.333 ± 13.4337
1.903 ± 1.2061
8.391 ± 1.2254
0.247 ± 0.0103
Joint Attention.
5.470 ± 0.8511
0.203 ± 0.0115
0.533 ± 0.0739
4.417 ± 0.0275
0.153 ± 0.0430
0.641 ± 0.0170
10.987 ± 17.7273
2.573 ± 0.1557
8.834 ± 3.4000
0.224 ± 0.0142
w/o Static
18.512 ± 2.9186
0.609 ± 0.0471
0.323 ± 0.0152
5.309 ± 0.3987
0.566 ± 0.1199
0.735 ± 0.0160
15.709 ± 12.6554
1.586 ± 0.0388
7.252 ± 0.4953
0.247 ± 0.0111
w/o Dynamic
19.542 ± 3.2823
0.640 ± 0.0354
0.270 ± 0.0104
5.231 ± 0.3214
0.550 ± 0.0464
0.609 ± 0.0216
15.430 ± 11.5595
0.450 ± 0.1339
2.867 ± 3.5811
0.244 ± 0.0098
w/o Ref. Adapt
6.344 ± 2.7415
0.233 ± 0.0446
0.475 ± 0.0138
4.524 ± 0.3682
0.208 ± 0.1082
0.682 ± 0.0120
14.135 ± 12.5521
0.468 ± 0.0339
2.695 ± 0.4918
0.242 ± 0.0124
Table 2: Quantitative ablation on Static-Dynamic Factorization and Injection across 50 scenes.
Figure 5: Qualitative ablation of GROME. Static-dynamic factorization and three injection strategies demonstrate their complementary roles in maintaining static scene and evolving entity states.
Figure 6: Local retrieval vs. Global consolidation.
Figure 7: Applications of GROME.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Static Scene
Moving Regions
VMem ( Li et al., 2025b )
Temporal
Temporal
WAH ( Wang and He, 2026 )
Temporal
Temporal
AlayaWorld ( Team et al., 2026 )
Spatial
Temporal
Gen3C ( Ren et al., 2025 )
Spatial
Spatial
WorldPlay ( Sun et al., 2025 )
Temporal
Temporal
LingBotV2 RC
Temporal
Temporal
Appendix
Table 3: Comparison of historical memory access patterns in long-horizon world models. Implicit memory methods rely on temporally indexed 2D frames, whereas explicit memory methods maintain global 3D reconstructions. In contrast, GROME introduces a unified, factorized memory: both static and dynamic components are grounded as local pointmaps, with the dynamic stream retaining source timestamps for temporal indexing.
Figure 8: Evaluation set and trajectory design. The top panels show representative static and dynamic scene categories and their proportions. The bottom panels illustrate exploration-and-revisit camera trajectories; dashed arrows mark returns to previously observed regions.
Figure 12
Figure 10: Runtime scaling with rollout length. (a) During initialization, the DA3 input window grows to a fixed 93-frame window; subsequent geometry updates have approximately constant cost. (b) Reference K/V computation and total history-token preparation remain nearly constant with rollout length. The step increase in pointmap fusion around 333 frames reflects an expanded active working set rather than continuous growth with the complete history.
Component
Mean (s)
Std. (s)
Geometry and memory construction
DA3 geometry update
54.772
1.055
Pointmap fusion
2.117
0.977
History-token preparation
Context K/V computation
3.782
0.056
Reference K/V computation
1.203
0.016
Appendix
Table 5: Runtime breakdown in the steady-state long-horizon regime. Reference K/V computation, highlighted in light green, represents the reference-token preparation overhead introduced by GROME.
Figure 11: Static complex-memory revisit-case 1. When exploring both sides of the kitchen, the visual evidence required for the return view is distributed across multiple historical frames, exceeding the model’s finite context window. GROME retrieves and consolidates local memories to reconstruct comprehensive, high-fidelity static scene references. Red boxes highlight representative inconsistencies in the baselines.
Figure 12: Static complex-memory revisit-case 2. The L-shaped explore–return trajectory distributes pose and scene evidence across several views, causing context-based methods to drift or omit content. GROME retrieves complementary static scene evidence through max-coverage retrieval, preserving the revisit scene structure. Red boxes mark representative failures.
Figure 13: Static complex-memory revisit-case 3. Complex camera trajectories create multi-view spatial configurations where different parts of the target view are scattered across disparate historical references, exposing the limits of single-frame retrieval. While maintaining a monolithic global cache often suffers from accumulated drift and high overhead, GROME retains historical evidence as local point clouds and selectively retrieves them under target poses. This strategy synthesizes complementary scene evidence to generate the target view with high-fidelity revisit consistency. Red boxes mark representative baseline failures.
Figure 14: Dynamic complex-memory revisit-case 1. Explicit-memory baselines overlook dynamic entities during memory construction, freezing them at outdated historical states, whereas context-only generation inevitably loses long-range environmental evidence. By factorizing static structure from dynamic entities, GROME simultaneously preserves background consistency and enables dynamic states to evolve. Red boxes mark representative failures.
Figure 15: Dynamic complex-memory revisit-case 2.
Figure 16: Dynamic complex-memory revisit-case 3.
Figure 17: Long-horizon qualitative comparison. When historical views leave the finite context, implicit memory and explicit memory-based baselines drift from the scene, while GROME preserves static structure beyond 1,400 frames. On dynamic scenes, factorized memory retains evolving dynamic entities without forcing them to remain in a stale state.
Figure 18: Video-to-Memory Continuation. Additional real-world video-continuation cases. Each row pairs an observed video with a GROME rollout generated along a new camera trajectory. Revisit frames assess retention of previously observed scene content, while novel-view frames show the continuation beyond the input.
Figure 19: Spatially Grounded Exploration. Additional spatially grounded exploration cases. Each row shows the source evidence, the memory transformation or composition, and the resulting rollout under a novel camera trajectory. This arrangement exposes which visual content originates from each memory region.
Figure 20: Robustness to imperfect geometric projection. Although rendered historical memories often exhibit geometric artifacts such as disocclusion holes and blur, GROME leverages them as soft spatial guidance rather than rigid pixel-level targets via timestep-aware memory routing. Guided by pretrained world priors, the frozen model corrects moderate rendered distortions and plausibly synthesizes unobserved regions during finer denoising steps while maintaining scene consistency.
Figure 21: Representative failure cases of GROME. (a) At long rollout horizons, appearance and color drift inherited from the frozen backbone can still accumulate despite Reference Value Adaptation. (b) Severe local geometry errors can produce sparse or corrupted warped evidence. Although the pretrained VWM usually tolerates moderate projection imperfections, extreme artifacts may be interpreted as plausible scene content and propagate into the generated frame.
Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines.
Zhengxuan Wei, Xu Guo, Xinghui Li +8
School of Intelligence Science and Technology, Nanjing University · 2Kling Team, Kuaishou Technology · 3Tsinghua University
Video world models aim to generate explorable, 3D-consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long-horizon generation. However, these auxiliary memory pathways operate outside the model's internal generative dynamics, preventing the model from intrinsically learning when and what historical information should be retrieved. We propose to internalize memory retrieval into the generation process, allowing retrieval to emerge as an intrinsic behavior of the video world model rather than relying on an external memory system. Based on this principle, we introduce Learning-to-Retrieve (L2R), which repurposes the model's persistent internal state as a memory for historical context. A camera-conditioned retrieval gate selectively accesses relevant historical information from this state, determining what to retrieve, while a retrieval trigger determines when to retrieve. We further supervise the trigger with a 3D re-visibility signal, activating retrieval when previously observed content re-enters the current view while otherwise preserving the existing context. Together, these components enable the model to intrinsically acquire memory retrieval behavior and incorporate relevant historical observations into generation without a separate retrieval pathway. Across multiple base models and camera-revisit benchmarks, L2R improves long-term scene consistency while eliminating the need for an external memory bank or 3D conditions. https://jkhu29.github.io/l2r
Recent advances in video generative models have promoted rapid progress in controllable world models. However, maintaining fine-grained spatio-temporal consistency under long-horizon reasoning remains a key challenge. In this work, we move beyond explicit 3D memory and coarse frame-level implicit modeling, and propose a fine-grained, learnable, and scalable memory for consistent world generation. We first identify two fundamental limitations of naïve learnable memory architectures in long-horizon extrapolation, namely computational inefficiency and attention dispersion. Through a systematic analysis of attention dispersion, we propose DecMem, a decoupled memory architecture that employs Sparse Global Memory for efficient fine-grained access to global history and Anchored Local Memory for stable and high-quality extrapolation. Extensive experiments demonstrate that DecMem significantly outperforms current state-of-the-art methods. By ensuring precise and efficient long-term memory and achieving superior extrapolation capabilities, DecMem enables minute-level controllable long video generation with high fidelity and consistency.
Zhenhao Yang, Xiaoshi Wu, Zhengyao Lv +5
1The University of Hong Kong · *Work done during an internship at Kling Team, Kuaishou Tech. · 2Kling Team, Kuaishou Technology