cs.CVOct 29, 2026
SaveGROME: GROunding Factorized MEmory into Video World Models
Organizations: East China Normal University · Shanghai Artificial Intelligence Laboratory · Fudan University · Beihang University ·
Abstract
Long-horizon consistency is essential for video world models, yet it becomes difficult to maintain once observed content falls outside the context window. Existing memory mechanisms either retrieve history as implicit context, which remains constrained by context capacity and entangles static and dynamic information or consolidate history into explicit global representations, which struggle to model evolving dynamics and accumulate geometric errors. To overcome these limitations, we introduce GROME, a novel factorized memory framework that augments video world models for long-horizon coherence across both static scenes and dynamic entities. GROME operates via three synergistic components: 1)Static-Dynamic Factorization decouples historical evidence into static and dynamic components, preserving static scene geometry and dynamic entity states separately; 2)Local Memory Retrieval selects spatially aligned static history to eliminate geometric drift alongside the latest dynamic entity states to ensure temporal continuity; 3)~Training-Free Memory Injection seamlessly grounds the factorized memory into video world models via their native conditioning pathways. Experiments on long-horizon revisit scenarios demonstrate that GROME substantially enhances scene persistence and dynamic entity fidelity over existing methods, all while preserving the visual quality and controllability of the frozen backbone without additional training. Beyond self-rollouts, GROME readily accommodates external observations as plug-and-play memory, unlocking flexible and controllable video continuation and scene exploration.
Figures & tables
Figure 1: (1) During autoregressive rollout, GROME progressively updates historical observations into a factorized memory and injects retrieved local evidence into a frozen video world model. (2) By decoupling static scene structures from dynamic entities, it ensures scene persistence while accommodating entity dynamics. (3) GROME also supports lifting real-world videos into factorized memory as plug-and-play references for controllable video continuation. (4) Multiple point clouds can be seamlessly composed as unified external memory for open-ended spatial exploration.
Figure 2: Comparison of memory paradigms. Implicit memory suffers from substantial scene-content overlap and ambiguous in dynamic states, while explicit memory consolidates historical frames into a global 3D scene but struggles with evolving dynamic entities and accumulated geometric drift. GROME instead factorizes historical frames into static-dynamic memory, preserving both persistent static scene structure and evolving dynamic entities over long-horizon generation.
Figure 3: Overview of GROME . GROME factorizes historical evidence into static scene and dynamic entity components. Both are retained over extended rollouts as local spatial memories, with the dynamic component further organized by explicit temporal indexing. Given a target camera trajectory, relevant historical evidence is separately retrieved from the static and dynamic components to render references. These static-dynamic references are then injected into the model’s native history-conditioning pathway through a set of training-free injection strategies, conditioning generation together with the local context and user actions. This allows GROME to better preserve consistency over long rollouts while maintaining generation quality and controllability.
Figure 4: Qualitative comparison. GROME better preserves revisit scene structure and entity consistency simultaneously across diverse revisit trajectories compared with existing world models.
| Revisit Consistency | Visual Quality | Dynamic Quality | Controllability | |||||||
| Model | PSNR | SSIM | LPIPS | CLIP Aesthetic | Subjective Qual. | Entity Cons. | Motion Acc. | Trans. Err. | Rot. Err. ( ∘ ) | Prompt Cons. |
| Static | ||||||||||
| Gen3C | 21.725 0.5582 | 0.749 0.0203 | 0.173 0.0140 | 5.334 0.0874 | 0.469 0.0264 | N/A | N/A | 0.420 0.0513 | 3.878 0.6494 | 0.258 0.0093 |
| VMem | 15.784 0.5323 | 0.460 0.0221 | 0.367 0.0132 | 5.089 0.0760 | 0.338 0.0230 | N/A | N/A | 0.624 0.4502 | 7.187 1.0210 | 0.251 0.0121 |
| WAH | 16.911 0.6472 | 0.540 0.0320 | 0.369 0.0266 | 5.116 0.0739 | 0.347 0.0308 | N/A | N/A | 2.141 0.7641 | 6.709 1.3273 | 0.231 0.0104 |
| WorldPlay | 16.087 0.4921 | 0.465 0.0236 | 0.289 0.0139 | 5.265 0.0869 | 0.404 0.0265 | N/A | N/A | 0.471 0.0504 | 6.391 0.9905 | 0.260 0.0090 |
Table 1: Quantitative comparison on static and dynamic long-horizon revisit trajectories. LingBotV2 RC denotes LingBotV2 with raw-context conditioning, while LingBotV2 TR denotes LingBotV2 with temporal retrieval. Values are reported as mean the half-width of the 95% paired bootstrap confidence intervals. Best and second-best values are shown in bold and underlined .
| Revisit Consistency | Visual Quality | Dynamic Quality | Controllability | |||||||
| Setting | PSNR | SSIM | LPIPS | CLIP Aesthetic | Subjective Qual. | Entity Cons. | Motion Acc. | Trans. Err. | Rot. Err. ( ∘ ) | Prompt Cons. |
| LingBotV2 RC | 15.505 3.1742 | 0.456 0.0418 | 0.500 0.0147 | 5.247 0.2950 | 0.572 0.0875 | 0.711 0.0482 | 16.333 13.4337 | 1.903 1.2061 | 8.391 1.2254 | 0.247 0.0103 |
| Joint Attention. | 5.470 0.8511 | 0.203 0.0115 | 0.533 0.0739 | 4.417 0.0275 | 0.153 0.0430 | 0.641 0.0170 | 10.987 17.7273 | 2.573 0.1557 | 8.834 3.4000 | 0.224 0.0142 |
| w/o Static | 18.512 2.9186 | 0.609 0.0471 | 0.323 0.0152 | 5.309 0.3987 | 0.566 0.1199 | 0.735 0.0160 | 15.709 12.6554 | 1.586 0.0388 | 7.252 0.4953 | 0.247 0.0111 |
| w/o Dynamic | 19.542 3.2823 | 0.640 0.0354 | 0.270 0.0104 | 5.231 0.3214 | 0.550 0.0464 | 0.609 0.0216 | 15.430 11.5595 | 0.450 0.1339 | 2.867 3.5811 | 0.244 0.0098 |
| w/o Ref. Adapt | 6.344 2.7415 | 0.233 0.0446 | 0.475 0.0138 | 4.524 0.3682 | 0.208 0.1082 | 0.682 0.0120 | 14.135 12.5521 | 0.468 0.0339 | 2.695 0.4918 | 0.242 0.0124 |
Table 2: Quantitative ablation on Static-Dynamic Factorization and Injection across 50 scenes.
Figure 5: Qualitative ablation of GROME. Static-dynamic factorization and three injection strategies demonstrate their complementary roles in maintaining static scene and evolving entity states.
Figure 6: Local retrieval vs. Global consolidation.
Figure 7: Applications of GROME.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Static Scene | Moving Regions |
| VMem ( Li et al., 2025b ) | Temporal | Temporal |
| WAH ( Wang and He, 2026 ) | Temporal | Temporal |
| AlayaWorld ( Team et al., 2026 ) | Spatial | Temporal |
| Gen3C ( Ren et al., 2025 ) | Spatial | Spatial |
| WorldPlay ( Sun et al., 2025 ) | Temporal | Temporal |
| LingBotV2 RC | Temporal | Temporal |
Table 3: Comparison of historical memory access patterns in long-horizon world models. Implicit memory methods rely on temporally indexed 2D frames, whereas explicit memory methods maintain global 3D reconstructions. In contrast, GROME introduces a unified, factorized memory: both static and dynamic components are grounded as local pointmaps, with the dynamic stream retaining source timestamps for temporal indexing.
Figure 8: Evaluation set and trajectory design. The top panels show representative static and dynamic scene categories and their proportions. The bottom panels illustrate exploration-and-revisit camera trajectories; dashed arrows mark returns to previously observed regions.
Figure 10: Runtime scaling with rollout length. (a) During initialization, the DA3 input window grows to a fixed 93-frame window; subsequent geometry updates have approximately constant cost. (b) Reference K/V computation and total history-token preparation remain nearly constant with rollout length. The step increase in pointmap fusion around 333 frames reflects an expanded active working set rather than continuous growth with the complete history.
| Component | Mean (s) | Std. (s) |
| Geometry and memory construction | ||
| DA3 geometry update | 54.772 | 1.055 |
| Pointmap fusion | 2.117 | 0.977 |
| History-token preparation | ||
| Context K/V computation | 3.782 | 0.056 |
| Reference K/V computation | 1.203 | 0.016 |
Table 5: Runtime breakdown in the steady-state long-horizon regime. Reference K/V computation, highlighted in light green, represents the reference-token preparation overhead introduced by GROME.
Figure 11: Static complex-memory revisit-case 1. When exploring both sides of the kitchen, the visual evidence required for the return view is distributed across multiple historical frames, exceeding the model’s finite context window. GROME retrieves and consolidates local memories to reconstruct comprehensive, high-fidelity static scene references. Red boxes highlight representative inconsistencies in the baselines.
Figure 12: Static complex-memory revisit-case 2. The L-shaped explore–return trajectory distributes pose and scene evidence across several views, causing context-based methods to drift or omit content. GROME retrieves complementary static scene evidence through max-coverage retrieval, preserving the revisit scene structure. Red boxes mark representative failures.
Figure 13: Static complex-memory revisit-case 3. Complex camera trajectories create multi-view spatial configurations where different parts of the target view are scattered across disparate historical references, exposing the limits of single-frame retrieval. While maintaining a monolithic global cache often suffers from accumulated drift and high overhead, GROME retains historical evidence as local point clouds and selectively retrieves them under target poses. This strategy synthesizes complementary scene evidence to generate the target view with high-fidelity revisit consistency. Red boxes mark representative baseline failures.
Figure 14: Dynamic complex-memory revisit-case 1. Explicit-memory baselines overlook dynamic entities during memory construction, freezing them at outdated historical states, whereas context-only generation inevitably loses long-range environmental evidence. By factorizing static structure from dynamic entities, GROME simultaneously preserves background consistency and enables dynamic states to evolve. Red boxes mark representative failures.
Figure 15: Dynamic complex-memory revisit-case 2.
Figure 16: Dynamic complex-memory revisit-case 3.
Figure 17: Long-horizon qualitative comparison. When historical views leave the finite context, implicit memory and explicit memory-based baselines drift from the scene, while GROME preserves static structure beyond 1,400 frames. On dynamic scenes, factorized memory retains evolving dynamic entities without forcing them to remain in a stale state.
Figure 18: Video-to-Memory Continuation. Additional real-world video-continuation cases. Each row pairs an observed video with a GROME rollout generated along a new camera trajectory. Revisit frames assess retention of previously observed scene content, while novel-view frames show the continuation beyond the input.
Figure 19: Spatially Grounded Exploration. Additional spatially grounded exploration cases. Each row shows the source evidence, the memory transformation or composition, and the resulting rollout under a novel camera trajectory. This arrangement exposes which visual content originates from each memory region.
Figure 20: Robustness to imperfect geometric projection. Although rendered historical memories often exhibit geometric artifacts such as disocclusion holes and blur, GROME leverages them as soft spatial guidance rather than rigid pixel-level targets via timestep-aware memory routing. Guided by pretrained world priors, the frozen model corrects moderate rendered distortions and plausibly synthesizes unobserved regions during finer denoising steps while maintaining scene consistency.
Figure 21: Representative failure cases of GROME. (a) At long rollout horizons, appearance and color drift inherited from the frozen backbone can still accumulate despite Reference Value Adaptation. (b) Severe local geometry errors can produce sparse or corrupted warped evidence. Although the pretrained VWM usually tolerates moderate projection imperfections, extreme artifacts may be interpreted as plausible scene content and propagate into the generated frame.
More in this topic
Memory-Augmented Video Generation
Memory-augmented video generation gives video models persistent context to preserve identities, scenes, and events across long sequences.
See all 112 papers