Autoregressive video world models enable temporally coherent generation for a single observer. Extending them to multiple agents requires consistency across independently controlled views and temporal gaps under causal streaming. We present ConsistWorld, a multi-agent world model that generates camera-controlled video streams of a static scene from one shared image. We formulate consistency as routing evidence from committed multi-agent history and concurrently generated peer views to the tokens being generated. Pose Conditioned Memory Retrieval selects relevant historical observations from all agents, recovering evidence beyond the recent context window. Visibility-Gated Peer Sharing regulates current peer information according to estimated historical coverage and current-view overlap. Together, they determine which historical observations enter the context and where concurrent peer information contributes, supporting long-term recall and coordinated exploration. Both mechanisms use camera geometry and maintain a bounded active context for a fixed agent count and retrieval budget. Experiments on evidence sharing cases and video length and agent number generalizations show that ConsistWorld achieves a strong cross-time and cross-agent consistency while preserving competitive generation quality.
Figures & tables
Figure 2: Overview of ConsistWorld. Given a shared source image I0 and K camera trajectories, ConsistWorld generates K coherent agent streams with shared masked self-attention. Pose-Conditioned Memory Retrieval retrieves the most relevant committed chunk from long-term multi-agent history using camera geometry, while Visibility-Gated Peer Sharing admits synchronous peer evidence only at spatial locations not already covered by committed observations.
Figure 3: Representative rollouts for the three consistency settings. ”f44” indicates frame 44 of the video. (a) Self-Revisit compares an agent’s first and later observations of the same scene regions. (b) Synchronous Cross-Agent Sharing shows two agents after re-overlap and along subsequent shared camera motions. (c) Asynchronous Cross-Agent Handoff compares locations first observed by a leader agent and revisited later by a follower agent.
Region LPIPS ↓
MultiCamVideo [ 1 ]
Ditto [ 3 ]
Setting
IQ ↑
AQ ↑
IQ ↑
AQ ↑
Self-Revisit
0.1181
0.6781
0.5240
0.7203
0.4897
Synchronous Cross-Agent Sharing
0.1179
0.7511
0.5735
0.7271
0.5083
Asynchronous Cross-Agent Handoff
0.0700
0.6997
0.5714
0.7350
0.5223
Table 1: Shared-observation consistency and visual quality of ConsistWorld.
Figure 4: Visual component ablation on asynchronous handoff and subsequent re-overlap. The left half compares earlier observations with later cross-agent revisits, while the right half shows both streams during re-overlap. Red boxes mark representative inconsistent regions. “f58” indicates frame 58 of the video.
Figure 5: Comparison between full-history baseline and ConsistWorld. Two agents repeatedly revisit the same four directions from a shared initial observation. Full-history baseline gradually drifts across revisits, whereas ConsistWorld preserves the previously established scene appearance and cross-agent consistency.
Method
Retrieval
Peer Gate
Region LPIPS ↓
LPIPS-MG ↑
IQ ↑
AQ ↑
ConsistWorld
✓
✓
0.1176
0.4767
0.6645
0.5052
w/o Retrieval
–
✓
0.3374
0.2273
0.6591
0.4921
w/o Gating
✓
–
0.1983
0.2401
0.6041
0.4591
w/o Retrieval & Gating
–
–
0.3818
0.1332
0.6531
0.4717
Table 2: Ablation of Pose Conditioned Memory Retrieval and Visibility-Gated Peer Sharing. Region LPIPS measures historical consistency, while LPIPS-MG measures revisit-specific memory after same-rollout temporal calibration.
Method
PSNR ↑
SSIM ↑
GT-LPIPS ↓
Full History
17.6799
0.4386
0.5203
ConsistWorld
18.5583
0.4526
0.4206
Table 3: Reference fidelity of ConsistWorld and the full-history baseline on matched rendered trajectories.
Method
16 chunks
24 chunks
32 chunks
w/o Retrieval & Gating
0.3820
0.4170
0.4503
ConsistWorld
0.1046
0.1088
0.1267
Table 4: Asynchronous handoff consistency measured with Region LPIPS ↓ at increasing rollout lengths.
Number of agents K
Region LPIPS ↓
2
0.1079
4
0.1221
5
0.1107
Table 5: Agent-count generalization under a fixed focal handoff protocol and common spatial support. K=5 lies outside the Stage-II training range.
This paper presents ShareVerse, a video generation framework enabling multi-agent shared world modeling, addressing the gap in existing works that lack support for unified shared world construction with multi-agent interaction. ShareVerse leverages the generation capability of large video models and integrates three key innovations: 1) A dataset for large-scale multi-agent interactive world modeling is built on the CARLA simulation platform, featuring diverse scenes, weather conditions, and interactive trajectories with paired multi-view videos (front/ rear/ left/ right views per agent) and camera data. 2) We propose a spatial concatenation strategy for four-view videos of independent agents to model a broader environment and to ensure internal multi-view geometric consistency. 3) We integrate cross-agent attention blocks into the pretrained video model, which enable interactive transmission of spatial-temporal information across agents, guaranteeing shared world consistency in overlapping regions and reasonable generation in non-overlapping regions. ShareVerse, which supports 49-frame large-scale video generation, accurately perceives the position of dynamic agents and achieves consistent shared world modeling.
Jiayi Zhu, Jianing Zhang, Yiying Yang +2
Shanghai Jiao Tong University, China · Fudan University, China · StepFun, China
Video world models have made rapid progress in generating controllable visual experiences, but most of them still simulate the world from a single observer. Extending such models to multiple agents raises a central challenge: if each agent's future state is generated independently, overlapping views may instantiate different versions of the same scene, leading to inconsistent objects, layouts, and appearances across agents. Conventional camera conditioning controls individual trajectories, but it does not explicitly couple the generation of views that should agree under shared scene geometry. We introduce Prisma-World, a camera-controllable multi-agent world model that formulates multi-agent generation as a joint geometry-aware denoising process for cross-view consistency. Prisma-World processes all agent videos within one full-attention sequence, uses a multi-agent RoPE design to distinguish agent identities while preserving synchronized temporal coordinates, and injects relative camera geometry into attention to bias overlapping viewpoints toward shared scene evidence. To further strengthen multi-view consistency and enhance global spatial perception, we augment our framework with an overlap-decaying curriculum training paradigm alongside minimap-conditioned structural guidance. To facilitate the training and evaluation of multi-agent models, we introduce PrismaDataset, a large-scale UE5 dataset with panoramic acquisition across diverse scenes, composable multi-agent view groups with flexible agent counts and complex camera trajectories, and precise camera/action annotations for consistency training and evaluation. Experiments show that a single Prisma-World model can generate high-fidelity multi-agent videos with flexible agent numbers, camera controllability, improved cross-view consistency, and spatial grounding under minimap guidance.
Huiqiang Sun, Zhan Peng, Size Wu +9
School of AIA, HUST · S-Lab, NTU · SenseTime Research +4
Video world models have achieved remarkable success in simulating environmental dynamics in response to actions by users or agents. They are modeled as action-conditioned video generation models that take historical frames and current actions as input to predict future frames. Yet, most existing approaches are limited to single-agent scenarios and fail to capture the complex interactions inherent in real-world multi-agent systems. We present MultiWorld, a unified framework for multi-agent multi-view world modeling that enables accurate control of multiple agents while maintaining multi-view consistency. We introduce the Multi-Agent Condition Module to achieve precise multi-agent controllability, and the Global State Encoder to ensure coherent observations across different views. MultiWorld supports flexible scaling of agent and view counts, and synthesizes different views in parallel for high efficiency. Experiments on multi-player game environments and multi-robot manipulation tasks demonstrate that MultiWorld outperforms baselines in video fidelity, action-following ability, and multi-view consistency. Project page: https://multi-world.github.io/