InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model

InSpatio Team; Donghui Shen; Guofeng Zhang; Haomin Liu; Haoyu Ji; Jialin Liu; Jing Guo; Nan Wang

cs.CVMar 12, 2026

InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model

Authors: InSpatio Team, Donghui Shen, Guofeng Zhang, Haomin Liu, Haoyu Ji, Jialin Liu, Jing Guo, Nan Wang,

Abstract

We present InSpatio-WorldFM, an open-source real-time frame model for spatial intelligence. Unlike video-based world models that rely on sequential frame generation and incur substantial latency due to window-level processing, InSpatio-WorldFM adopts a frame-based paradigm that generates each frame independently, enabling low-latency real-time spatial inference. By enforcing multi-view spatial consistency through explicit 3D anchors and implicit spatial memory, the model preserves global scene geometry while maintaining fine-grained visual details across viewpoint changes. We further introduce a progressive three-stage training pipeline that transforms a pretrained image diffusion model into a controllable frame model and finally into a real-time generator through few-step distillation. Experimental results show that InSpatio-WorldFM achieves strong multi-view consistency while supporting interactive exploration on consumer-grade GPUs, providing an efficient alternative to traditional video-based world models for real-time world simulation.

Figures & tables

Figure 1: Examples of generated worlds across diverse styles, including photorealistic, science-fiction, game-like, and artistic environments. The joystick interface enables real-time interactive exploration with negligible latency.

Figure 2: Overview. In the offline stage, a multi-view-consistent model generates plausible observations that provide 3D anchors and reference appearances. In the online stage, frame model performs fast real-time inference while updating scene content at keyframes.

Figure 3: The pipeline of InSpatio-WorldFM. The left part illustrates the conditional novel-view synthesis pipeline of WorldFM. WorldFM takes a reference image $x_{\text{ref}}$ (implicit scene memory), noisy latents $z_{t}$ , and point cloud rendering $\hat{x}_{\text{tgt}}$ (explicit 3D anchor) as inputs, which are spatially concatenated along the width dimension. Reference pose $\pi{\text{ref}}$ and target pose $\pi_{\text{tgt}}$ are also injected as control signals. The frame-based Diffusion Transformer Blocks process these conditions and synthesizes the target view $I_{\text{tgt}}$ in real-time via Distribution Matching Distillation (DMD). The right part shows the detailed architecture of the DiT blocks in WorldFM. Camera geometry control is achieved through the Projection Relative Position Embedding (PRoPE) strategy, enhancing cross-view geometric reasoning. The hybrid spatial memory mechanism combines point cloud rendering (explicit 3D anchor) and reference image (implicit memory), interacting solely through self-attention to achieve robust 3D consistency.

Figure 4: Qualitative results of teacher model.

Figure 5: Qualitative results of InSpatio-WorldFM.

Figure 6: Qualitative results of InSpatio-WorldFM.

Figure 7: Qualitative results of InSpatio-WorldFM.

Figure 8: Qualitative results of InSpatio-WorldFM. Across observations at varying viewing distances, the content and fine details generated by the FrameModel remain consistent.

World Models

World models represent and predict how environments change, helping agents plan, act, and reason about possible futures.

See all 363 papers

MoWorld: A Flash World Model
Jul 7, 2026Team Moxin, Deyi Ji, Tianrun Chen +26Efficient Inference Interactive World Models
MoWorld Team, Moxin Technology
INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
Apr 8, 2026InSpatio Team, Donghui Shen, Guofeng Zhang +20Video World Models Real-Time Interactive Video Generation
Latent Spatial Memory for Video World Models
Jun 8, 2026Weijie Wang, Haoyu Zhao, Yifan Yang +7Video Diffusion Models Memory-Augmented Video Generation
Zhejiang University · Microsoft Research · Adelaide University +1

Jul 7, 2026J/K move · Enter open · S save

MoWorld: A Flash World Model

Team Moxin, Deyi Ji, Tianrun Chen, Xin Zhang+25

MoWorld Team, Moxin Technology

Efficient Inference Interactive World Models Video World Models

Open paper AskPDF arXiv

The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High-frame-rate inference enables responsive perception, planning, and control in real-world autonomous systems. To this end, we present MoWorld, a cost-effective yet high-performance Flash World Model with an end-to-end framework spanning data generation, pre-training, distillation, and efficient inference, enabling up to 50 FPS real-time interaction with cinematic visual quality without the need of high-end GPUs. To enable large-scale real-world deployment, MoWorld jointly optimizes model capability and cost throughout the entire development pipeline. Specifically, unlike existing approaches that primarily rely on large-scale video corpora, MoWorld is built upon a scalable 3D-native data engine accumulated from our large-scale 3D vision and generative modeling pipeline, enabling the efficient construction of geometrically consistent training data across diverse real-world and synthetic environments. Based on this foundation, a curriculum cross-frame pre-training strategy for stable and scalable World Model learning, an efficient denoising-step distillation algorithm to reduce diffusion training cost, and a mixed-precision parallel inference framework for low-cost real-time deployment. MoWorld is the first real-time interactive World Model built on the Neural Processing Unit (NPU) and can achieves up to 50 FPS in such the devices, enabling practical and efficient deployment at scale. Comprehensive evaluations demonstrate that MoWorld achieves leading performance; notably, its average inference cost is only 30%-50% of that of existing World Models, providing a practical foundation for large-scale real-world applications of World Models. We also demonstrate diverse applications of MoWorld.

Abstract

World Models

Explore similar work

MoWorld: A Flash World Model

INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling

Latent Spatial Memory for Video World Models