Multiplayer generative world models must produce observations that agree on shared entities despite different camera motions and occlusions. Explicit world representations provide spatial control, but precise modeling of individual game maps limits scene diversity, while shared geometry alone cannot ensure consistent appearance across views. We present CoCode-World, a shared 3D proxy framework that separates structural control from visual synthesis for two-player video generation. Programmable proxies specify scene layout, motion, and occlusion, while RGB references guide character and scene appearance. To coordinate visual details across players, we concatenate paired proxy videos, appearance references, and target views in a consistent spatial layout and generate both views jointly with a pretrained video model. This formulation uses the model's native attention for cross-view communication within a single generation branch, with LoRA adaptation learning proxy adherence and appearance correspondence from paired observations. We construct a two-player GTA~V dataset that pairs synchronized RGB observations with proxy videos derived from shared map geometry and game-engine state. Experiments demonstrate appearance transfer to characters and scene styles unseen during training, with improvements in video quality and cross-view player identity over the evaluated baseline. Ablations further examine how proxy construction and cross-view communication affect generation. Project page: https://iapple233.github.io/CCW/.
Figures & tables
Figure 1: Two-player views of a shared game world. Paired views illustrate environment interaction, combat, exploration, and vehicle riding. Each pair shows the scene from the two players’ perspectives.
Figure 2: Joint two-player rendering. Map geometry and entity observations produce a proxy for each camera. The two proxy views share a fixed left–right layout with the RGB appearance reference and target video. The H3 transformer jointly denoises both target views through its native reference interface, with full attention across the context and LoRA adaptation of the frozen backbone.
Figure 3: Gameplay categories and examples. The donut shows clip proportions across four categories, with colored connectors linking each category to paired gameplay views. Each of the 7,846 approximately five-second clips receives one primary category from caption-described temporal coverage across both views; ties follow fight, riding, interaction, then exploration. Proportions are automatic caption-derived estimates.
Method
DOVER ↑
LAION ↑
DINO Adj. ↑
Edge F1 ↑
First-frame ↑
Cross-view ↑
Gamma-World
0.170
4.364
0.923
0.038
0.160
0.697
SolarRenderer
0.454
5.477
0.968
0.073
0.655
3.696
MiniMax-H3
0.441
5.397
0.958
0.060
0.659
2.868
Seedance 2.5
0.469
5.568
0.973
0.120
0.700
3.552
Ours
0.524
5.655
0.970
0.124
0.684
3.847
Table 1: Quantitative comparison. The first five metrics follow SolarRenderer: visual quality (DOVER and LAION), temporal appearance stability (DINO Adj.), proxy contour agreement (Edge F1), and reference preservation (First-frame). Cross-view reports the player-identity score on a 0–5 scale. Higher is better for every metric.
Figure 4: Comparison with Seedance 2.5 across two player views. Boxes highlight the compared player regions.
Figure 5: Effect of proxy construction. Rows compare Raw, Detail & Lighting, and Skeleton, with the corresponding proxy shown in each inset.
Figure 6: Dual Branch and Joint (Ours) on a mosaic-courtyard case at two matched times. Both methods use step-2,000 checkpoints. Outlines and enlarged crops highlight player B in player A’s view for comparison with player B’s appearance in their own view.
Multiplayer world models must generate independently controlled views with consistent representations of both players and their shared environment. Most existing approaches coordinate multiple players through joint multi-view generation, whose cost grows with each additional player. We present WorldCast, a distributed multiplayer world model in which each player runs a local client comprising a video generator and a state model. Using recorded player positions and map geometry during training, the state model estimates the player's position from generated video and control inputs. Clients exchange player states and project them into camera-aligned player state fields that guide where and how other players are rendered. Shared scene state enables clients to reuse one another's generated observations to maintain consistent scene appearance across views. Experiments on Counter-Strike 2 demonstrate WorldCast's consistency, real-time performance, and distributed scalability. The camera-aligned player state field improves player rendering rates by over an order of magnitude over joint-generation methods, while shared scene state improves visual consistency over whole rounds. Each client runs in real time and exchanges only player and scene states, enabling scalable multiplayer generation without a centralized computational bottleneck. Image quality remains stable over hour-long rollouts.
This paper presents ShareVerse, a video generation framework enabling multi-agent shared world modeling, addressing the gap in existing works that lack support for unified shared world construction with multi-agent interaction. ShareVerse leverages the generation capability of large video models and integrates three key innovations: 1) A dataset for large-scale multi-agent interactive world modeling is built on the CARLA simulation platform, featuring diverse scenes, weather conditions, and interactive trajectories with paired multi-view videos (front/ rear/ left/ right views per agent) and camera data. 2) We propose a spatial concatenation strategy for four-view videos of independent agents to model a broader environment and to ensure internal multi-view geometric consistency. 3) We integrate cross-agent attention blocks into the pretrained video model, which enable interactive transmission of spatial-temporal information across agents, guaranteeing shared world consistency in overlapping regions and reasonable generation in non-overlapping regions. ShareVerse, which supports 49-frame large-scale video generation, accurately perceives the position of dynamic agents and achieves consistent shared world modeling.
Jiayi Zhu, Jianing Zhang, Yiying Yang +2
Shanghai Jiao Tong University, China · Fudan University, China · StepFun, China
Lightweight scene proxies let creators control scene layout and motion while leaving room for imagination in appearance, lighting, and visual effects. However, a suitable proxy is not uniquely defined, making paired proxy-video data difficult to construct automatically at scale. We present Proxy2World, a controllable world model that learns these complementary capabilities from ordinary posed RGBD videos, without training on authored proxy-video pairs. The model jointly learns depth-conditioned RGB generation and joint RGBD generation through cross-modal flow matching. Learning both tasks enables proxy-camera hybrid denoising at inference to follow the proxy structure while producing natural, detailed visuals. We further introduce ProxyBench to evaluate this capability across a diverse set of scenes, camera trajectories, and subject motions. Experiments on ProxyBench show that Proxy2World achieves a better balance between structural adherence and visual quality than camera-controlled and geometry-conditioned methods, supported by quantitative metrics, VLM assessments, human evaluations and diverse qualitative results.