Despite the success of video world models, extending them to multiple agents is difficult because it requires generating several agent-specific video streams that are consistent in time and between agents. To address this gap, we introduce OverSeer, a multi-agent world model that augments egocentric agent views with auxiliary camera views. We suggest that, by fixing the auxiliary views in space, they act as a proxy for a world-centric representation: Unlike the agents' views, which can change entirely due to a simple change in agent position, changes in the auxiliary views directly map to localized changes in the world state. We show that these fixed views substantially help the model to maintain consistency. We also show that, due to their redundant and semi-static nature, the auxiliary views can be efficiently compressed with a delta-VAE that encodes only the changes relative to a static reference image, reducing the number of tokens required to process them. Experiments in Minecraft and Doom demonstrate improved world and agent consistency, with qualitative results on real-world multi-camera captures indicating that the approach is not tied to game engines. Our compression analysis further shows that delta-VAE substantially reduces the computational overhead of auxiliary-view generation. We conclude that these fixed views provide a simple and efficient way to improve multi-agent world models.
Figures & tables
Figure 1: OverSeer sees beyond what agents see. Two agents interact in a shared Minecraft environment (Left). In addition to generating each agent’s egocentric view (Bottom), OverSeer models six fixed-pose auxiliary cameras (Right). These auxiliary streams establish a persistent shared reference, enforcing world consistency over time and agent consistency across views.
Figure 2: OverSeer design overview. A δ -VAE (Left) compresses each fixed auxiliary stream into compact residual codes relative to its first-frame reference, and OverSeer (Right) jointly generates agent latents and these residual codes, conditioned on T5-encoded actions via cross-attention and Plücker-ray camera poses via AdaLN.
Figure 3: OverSeer keeps the agents consistent across views. Generated videos in Minecraft , GZDoom and a multi-camera Real Capture , each at two moments (before, after). Each agent’s trail is one 3D track projected into every auxiliary camera (Agent 1 blue, Agent 2 orange); it lands on the generated agent in each of them, so the agents are consistent across views.
Figure 4: δ -VAE residual codes carry the movers, not the scene. Four cross-scene transfers: a host anchor is decoded with a donor clip’s residual code (transfer, purple band) instead of its own (control), at two moments (before, after). The donor’s mover (white box) appears in the host scene (purple box) while the host scene itself is preserved.
Solaris Benchmark
MAC-Bench
Movement
Grounding
Memory
Building
Consistency
Reunite at Start
Reunite Far
Dynamic Event
Method
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
Solaris
68.8
22.38
66.7
14.92
20.8
47.11
22.9
76.01
56.3
68.42
0.0
17.06
6.2
20.23
12.5
26.52
γ -World
64.6
17.13
62.5
29.39
16.7
29.57
43.8
24.62
47.9
56.39
0.0
23.83
0.0
25.71
8.3
31.72
OverSeer
59.1
18.27
91.7
11.33
87.5
12.95
41.7
32.50
60.4
26.47
60.2
13.99
37.9
12.79
40.5
17.61
Table 1: Results on Minecraft. Comparison on the Solaris benchmark and our MAC-Bench. VLM denotes VLM-judged accuracy (%) and FID denotes frame-level FID; MAC-Bench results are averaged over 5 runs. Bold and underline denote the best and second-best results in each column, respectively.
Figure 5: OverSeer keeps the world consistent over time. On the three revisiting tasks (reunite at start, reunite far, dynamic event), OverSeer shows the partner and the crater where the ground truth does, while Solaris and γ -World mostly do not.
Minecraft
Doom
Reunite at Start
Reunite Far
Dynamic Event
Reunite at Start
Reunite Far
Variant
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
OverSeer
60.2
14.0
37.9
12.8
40.5
17.6
62.0
23.2
67.0
20.9
w/o Aux. Views
25.0
24.6
16.7
27.2
20.8
33.1
12.0
50.6
0.0
87.4
w/o δ -VAE
59.4
8.2
40.6
13.4
41.7
10.3
78.0
15.8
84.0
15.4
Table 2: Ablation of auxiliary views and δ -VAE. Comparison on MAC-Bench in Minecraft and Doom. w/o δ -VAE models auxiliary views with uncompressed video-VAE latents, using 8× more auxiliary tokens. Bold denotes the best result in each column.
Figure 6: Effect of auxiliary camera count. Adding auxiliary views systematically improves FID and VLM accuracies on consistency and memory metrics across both benchmarks, with diminishing returns once spatial coverage saturates.
Figure 7: Effect of δ -VAE compression. δ -VAE keeps compute nearly flat as auxiliary views are added. Stronger compression trades reconstruction fidelity for disentanglement: donor leakage drops steadily, while mover transfer peaks at the default 8× before degrading.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Start
Mid-episode
End
Query
Reunite at Start
13 blocks apart, facing
each wanders ≤9 blocks
exact start poses
Is the partner visible?
Reunite Far
17 blocks apart
∼59 blocks each, converging; partner 86∘ off-axis
2.9 blocks apart, facing
Is the partner visible?
Dynamic Event
13 blocks apart, event at midpoint
separate to 40 blocks by frame 76
4.3 blocks apart
What happened to the event?
Appendix
Table 3: MAC-Bench task specifications in Minecraft. Mid-episode angles are medians over episodes.
δ -VAE
Minecraft Multi-agent Consistency Benchmark
PSNR ↑
Reunite at Start
Reunite Far
Dynamic Event
Variant
dB
VLM ↑
FID ↓
VLM ↑
FID ↓
VLM ↑
FID ↓
w/o Lalign
25.1
56.9
13.47
33.4
12.26
38.1
17.08
Full δ -VAE
24.7
60.2
13.99
37.9
12.79
40.5
17.61
Appendix
Table 4: Ablation of the feature alignment loss. We evaluate its effect on δ -VAE reconstruction quality and downstream world consistency in Minecraft.
Entry
Sentence
forward
The player is walking forward
left
The player is strafing to the left
back
The player is walking backward
right
The player is strafing to the right
jump
The player is jumping into the air
sneak
The player is sneaking and crouching down
Appendix
Table 5: The action vocabulary, one sentence per entry. The nine hotbar entries differ only in the number they name. drop , inv , esc , tab and swap_hands are never pressed in our episodes and are kept only so that the vocabulary matches the game’s input.
Figure 8: Additional MAC-Bench results in Minecraft. Top: the three benchmark tasks. Bottom: three further dynamic events. OverSeer recovers the partner or the evolved event state where the ground truth does, while the egocentric-only variant does not.
Figure 9: MAC-Bench results in Doom. Reunite tasks on three Freedoom maps. OverSeer restores the partner after the agents separate; the egocentric-only variant does not.
Figure 10: Long rollouts in Minecraft. Each block shows both agent views and all six auxiliary views over 30 s.
Figure 11: Long rollouts in Doom. Each block shows both agent views and all eight auxiliary views over 21 s.
Figure 12: Long rollouts on real-world multi-camera captures. Left: both agent views and all six auxiliary views over 30 s. Right: top-down layout of the auxiliary cameras and agent trajectories.