Building world models with spatial consistency and real-time interactivity remains a fundamental challenge in computer vision. Current video generation paradigms often struggle with a lack of spatial persistence and insufficient visual realism, making it difficult to support seamless navigation in complex environments. To address these challenges, we propose INSPATIO-WORLD, a novel real-time framework capable of recovering and generating high-fidelity, dynamic interactive scenes from a single reference video. At the core of our approach is a Spatiotemporal Autoregressive (STAR) architecture, which enables consistent and controllable scene evolution through two tightly coupled components: Implicit Spatiotemporal Cache aggregates reference and historical observations into a latent world representation, ensuring global consistency during long-horizon navigation; Explicit Spatial Constraint Module enforces geometric structure and translates user interactions into precise and physically plausible camera trajectories. Furthermore, we introduce Joint Distribution Matching Distillation (JDMD). By using real-world data distributions as a regularizing guide, JDMD effectively overcomes the fidelity degradation typically caused by over-reliance on synthetic data. Extensive experiments demonstrate that INSPATIO-WORLD significantly outperforms existing state-of-the-art (SOTA) models in spatial consistency and interaction precision, ranking first among real-time interactive methods on the WorldScore-Dynamic benchmark, and establishing a practical pipeline for navigating 4D environments reconstructed from monocular videos.
Figures & tables
Figure 1: InSpatio-World : Toward a Versatile 4D World Simulator. Top: Our framework enables the synthesis of diverse dynamic scenes from a single video, supporting real-time, high-DoF interactive 4D roaming experiences. Middle: The system is driven by those core capabilities: Free Spatial Roaming along user-defined camera trajectories, Temporal Control over dynamic scene evolution, and the maintenance of Physical Realism . Bottom: These capabilities endow InSpatio-World with the potential to serve as a real-time 4D novel-view rendering engine, promising to support downstream tasks such as Embodied Intelligence and Autonomous Driving .
Figure 2: Architecture of the Spatiotemporal Autoregressive Framework and JDMD Pipeline. The framework constructs a spatiotemporal cache using reference information and historical generations, leveraging depth-based warping to establish explicit geometric constraints for consistent autoregressive video generation. The JDMD phase features a multi-task distillation mechanism with shared weights, supervised by a dual-teacher architecture comprising perceptual and motion teachers.
Method
Real-time/Interactive
Dynamic Score
Control
Static Score
Overall ↑
3D Consist. ↑
Motion Acc. ↑
Smoothness ↑
Camera ↑
Object ↑
Overall ↑
Photometric ↑
Content ↑
FantasyWorld-1.0
No
71.39
84.62
50.30
75.81
81.45
87.90
80.45
94.07
66.94
CogVideoX-I2V
No
59.12
86.21
69.56
60.15
38.27
40.07
62.15
88.12
36.73
Gen-3
No
57.58
68.31
54.53
68.87
29.47
62.92
60.71
87.09
50.49
LTX-Video
No
56.54
78.41
76.22
71.09
25.06
53.41
55.44
88.92
39.73
Hailuo
No
56.36
67.18
63.46
70.07
22.39
69.56
57.55
62.82
73.53
Table 1: WorldScore benchmark results. We compare InSpatio-World against leading world models on the WorldScore benchmark. Our method achieves the highest camera control score and highly competitive overall dynamic performance at a fraction of the computational cost. The best results are highlighted in bold, and the second-best are underlined .
Figure 3: Quantitative comparison on WorldScore-Dynamic. Each bubble represents a method, with the vertical axis showing the score of WorldScore-Dynamic and the horizontal axis showing model parameters × inference steps. InSpatio-World achieves a dynamic score of 68.72 with a significantly lower computational overhead, demonstrating a superior compute-quality trade-off by breaking the zero-sum game between geometric control and generation fidelity.
Method
FID ↓
FVD ↓
Rot ↓
Trans ↓
HY-WorldPlay
129.46
387.50
25.050
0.6725
Infinite-World
89.44
215.96
16.518
0.4715
LingBot-World
64.84
173.02
11.981
0.2064
InSpatio-World
42.68
100.55
2.8762
0.1398
Table 2: Quantitative comparison on the RE10K-Long dataset. The best results are highlighted in bold, and the second-best are underlined .
Figure 4: Qualitative comparison on RE10K-Long dataset. Qualitative comparison on RE10K-Long. For each of the two scenes, the leftmost image represents the input Source image. For each method, the top row displays the intermediate frame of the generated sequence, while the bottom row showcases the final frame. As generation progresses, baseline methods exhibit varying degrees of failure, such as camera pose drift or structural warping. In contrast, InSpatio-World maintains precise trajectory control and persistent geometric consistency throughout the extended sequence.
Method
OpenVid
Blender
VBench ↑
Rot ↓
Trans ↓
FID ↓
FVD ↓
Rot ↓
Trans ↓
Aesth.
Imag.
Flick.
Smooth.
Subj.
Bg.
Overall
TrajectoryCrafter
0.5210
0.6527
0.9444
0.9736
0.8749
0.8961
0.8105
2.1650
0.1710
256.69
818.73
4.1780
0.2015
ReCamMaster
0.5666
0.6863
0.9736
0.9928
0.9373
0.9163
0.8455
3.8640
0.2310
116.53
311.06
3.5062
0.2001
NeoVerse
0.5583
0.7272
0.9646
0.9904
0.9234
0.9279
0.8486
1.5780
0.1340
103.23
230.87
1.2148
0.0636
InSpatio-World
0.5742
0.7296
0.9638
0.9901
0.9216
0.9249
0.8507
1.6000
0.1240
44.46
110.11
1.2386
0.0667
Table 3: Quantitative comparison on Camera Controlled Video Rerendering. We evaluate our method against state-of-the-art baselines on both the OpenVid dataset and the synthetic Blender dataset. The best results are highlighted in bold, and the second-best are underlined . For OpenVid dataset, Overall represents the average score of the six VBench metrics.
Figure 5: Qualitative comparison on Camera Controlled Video Rerendering. Each row represents a distinct scene. From left to right: the first frame of the reference video, the warped final frame, and the final frames generated by TrajectoryCrafter, ReCamMaster, NeoVerse, and our method. Compared to existing methods, our approach yields higher structural fidelity to the original scene and delivers significantly better textural details. Simultaneously, it demonstrates superior instruction-following, achieving precise camera trajectories that are nearly identical to the rendered ground truth. The reference frames showcased are sampled from online video platforms and are utilized exclusively for academic demonstration purposes.
While existing camera-controllable video generation models can produce visually compelling sequences, preserving intrinsic 4D spatiotemporal coherence remains challenging. To address this limitation, we propose ChronoWorld, an "Observation--State--Reflection" framework that leverages spatiotemporal causal cues and reconstruction priors to generate globally consistent, free-view 4D scenes. Given a context video, we introduce a Spatiotemporal Epipolar Causal Attention mechanism that enforces multi-view epipolar constraints and temporal causality throughout the generation process. In addition, we develop a reconstruction-driven geometric reflection pipeline with a 4D retrieval strategy to enable dynamic self-assessment and correction of generated outputs, improving consistency and accuracy. Extensive experiments show that ChronoWorld achieves state-of-the-art performance in spatiotemporally consistent, cinematic-quality 4D scene generation, with strong generalization and high-fidelity geometry across diverse scenarios.
Xiaoyu Zhou, Dingwei Xian, Zhenyu Wang +3
Wangxuan Institute of Computer Technology, Peking University · VGI Labs Co., Ltd. · University of California, Merced
We present InSpatio-WorldFM, an open-source real-time frame model for spatial intelligence. Unlike video-based world models that rely on sequential frame generation and incur substantial latency due to window-level processing, InSpatio-WorldFM adopts a frame-based paradigm that generates each frame independently, enabling low-latency real-time spatial inference. By enforcing multi-view spatial consistency through explicit 3D anchors and implicit spatial memory, the model preserves global scene geometry while maintaining fine-grained visual details across viewpoint changes. We further introduce a progressive three-stage training pipeline that transforms a pretrained image diffusion model into a controllable frame model and finally into a real-time generator through few-step distillation. Experimental results show that InSpatio-WorldFM achieves strong multi-view consistency while supporting interactive exploration on consumer-grade GPUs, providing an efficient alternative to traditional video-based world models for real-time world simulation.
Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling applications in games, robotics, embodied agents, and XR. Achieving stable long-horizon interactive generation, however, remains challenging, as the model must simultaneously preserve scene geometry, dynamic consistency, and camera control while supporting real-time autoregressive generation. Building upon Matrix-Game 3.0, we present Matrix-Game 3.5, as shown in Figure 1, which advances real-time interactive world generation toward geometry-aware and long-horizon consistent simulation through three key improvements. First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters, combining explicit 3D patch retrieval with projective camera conditioning to enable geometry-consistent camera control and faithful long-horizon scene recall. Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects, preserving both geometric consistency and subject identity throughout long-horizon generation. Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator through Perceptual Flow Matching and curriculum based Self-Rollout DMD, enabling minute-long real-time interactive generation. Extensive experiments demonstrate that, with a unified training corpus spanning Unreal simulation environments, open-world games, and internet videos, MatrixGame 3.5 achieves strong performance in long-horizon scene recall, precise camera control, subject consistency, prompt-driven world generation, and stable real-time open-world interaction.