FlockWorld: Toward Population-Scale Multi-Agent Video World Models through Artificial Life Simulation

Meryem Koksal; Yunzhe Wang; Volkan Ustun

cs.CVOct 25, 2026

FlockWorld: Toward Population-Scale Multi-Agent Video World Models through Artificial Life Simulation

Authors: Meryem Koksal, Yunzhe Wang, Volkan Ustun

Organizations: Grinnell College Grinnell, United States · University of Southern California Los Angeles, United States · USC Institute for Creative Technologies Los Angeles, United States

Abstract

Most action-conditioned video world models remain limited to a single player, while existing multiplayer models support only small, fixed groups of agents. We introduce FlockWorld, an artificial-life environment built on Reynolds's Boids algorithm for studying population-scale multi-agent video world models. Specifically, FlockWorld renders synchronized egocentric crops, assigns distinct colors to preserve camera-agent identity across views, exposes rule-derived accelerations as actions, and lets the number of camera agents be configured independently of the size of the surrounding population. We further introduce FlockDiT, an action-conditioned latent diffusion transformer trained with flow matching on FlockWorld data to jointly generate synchronized egocentric videos for multiple agents. Our results suggest that joint generation benefits agent detection at density-matched settings and that diffusion forcing improves rollout fidelity. Together, FlockWorld and FlockDiT provide a controlled testbed for multi-agent video generation.

Figures & tables

Figure 1: FlockWorld rendering conditions. A black background (left) provides no global-position cue, while a spatial gradient (right) encodes world position. Each condition shows the global view and two of its $128\times 128$ agent-centered crops (boxed). Colored boids are identifiable camera agents, while white boids are ambient entities.

Figure 2: Overview of the FlockDiT model architecture.

Panel of Figure 3 — Figure 3: Cross-view consistency metrics (Appendix J ) for one agent pair seen from both views, showing (a) mutual detection, (b) disagreement about the pair’s relative offset and its frame-to-frame change, and (c) agreement about ambient white boids in the shared region.

	Det./ frame	Det. rate $\uparrow$	Pos. err. px $\downarrow$	Recip.	White corr. $\uparrow$	Disp. err. px $\downarrow$	Motion err. px $\downarrow$	SSIM $\uparrow$
Ceiling	4.88	96.5 $\pm$ 0.4	1.496 $\pm$ 0.013	51.9 $\pm$ 7.0	85.6 $\pm$ 5.4	4.62 $\pm$ 1.77	1.05 $\pm$ 0.60	94.0 $\pm$ 0.7
1 Baseline	13.08	28.5 $\pm$ 0.9	2.511 $\pm$ 0.048	17.4 $\pm$ 1.8	13.6 $\pm$ 1.2	62.02 $\pm$ 1.96	19.04 $\pm$ 2.05	43.1 $\pm$ 2.8
2 Tiled	23.14	22.6 $\pm$ 0.5	3.444 $\pm$ 0.033	43.5 $\pm$ 0.9	15.2 $\pm$ 0.5	64.08 $\pm$ 0.58	32.49 $\pm$ 0.66	37.1 $\pm$ 1.0
3 DF	4.51	26.7 $\pm$ 1.1	1.932 $\pm$ 0.039	22.6 $\pm$ 6.4	40.7 $\pm$ 11.1	14.22 $\pm$ 6.72	3.14 $\pm$ 1.63	84.3 $\pm$ 4.1
4 2-stage	8.94	29.5 $\pm$ 1.1	2.076 $\pm$ 0.070	46.4 $\pm$ 3.1	4.2 $\pm$ 1.1	61.06 $\pm$ 3.01	22.78 $\pm$ 2.23	63.2 $\pm$ 2.4
5 Tiled+DF	4.17	13.5 $\pm$ 1.1	2.594 $\pm$ 0.047	12.2 $\pm$ 3.9	33.5 $\pm$ 12.0	15.48 $\pm$ 10.60	1.14 $\pm$ 0.44	85.8 $\pm$ 3.8

Table 1: Mechanism comparison at

P=10

(black background). Metrics in

[0,1]

are reported

\times 100

, and

\pm

values are 95% confidence intervals across episodes (Appendix J ). Bold marks the best of Exps. 1-7 and any overlapping interval, and density and reciprocity are unranked. Exps. 3 and 5 average 27 and 26 episodes for Disp. err. and SSIM, and 27 and 25 for Motion err.

	Detection rate $\uparrow$			Detections per frame			Recip.
$P$	ceiling	model	floor	GT visible	model	floor	ceiling
2	97.0 $\pm$ 0.7	22.7 $\pm$ 2.5	20.1 $\pm$ 2.3	5.02	8.15	7.99	14.8
5	96.6 $\pm$ 0.3	27.0 $\pm$ 0.9	19.7 $\pm$ 1.2	4.98	22.01 †	8.06	47.2
8	96.4 $\pm$ 0.4	31.8 $\pm$ 0.9	18.7 $\pm$ 1.2	5.04	18.45 †	6.99	34.5
10	96.5 $\pm$ 0.4	27.4 $\pm$ 1.1	19.0 $\pm$ 0.9	5.04	7.11	7.48	51.9
20	96.5 $\pm$ 0.3	28.1 $\pm$ 0.6	19.5 $\pm$ 0.7	5.11	18.33 †	7.56	14.3

Table 2: Agent-count sweep, using the black background, the base configuration, and an equal 48-hour budget per run, with metrics in

[0,1]

reported

\times 100

. Bold marks density-matched settings where the model-floor gap exceeds the episode spread, and † marks rows where the model renders

{\sim}2.5\times

the floor’s detections. The

P=10

row is a different run from Exp. 1 of Table 1 (Appendix J ).

	Black background		Gradient background
	ceiling	model	ceiling	model
Detections / frame	4.88	13.08	4.81	7.11
Detection rate $\uparrow$	96.5 $\pm$ 0.4	28.5 $\pm$ 0.9	95.1 $\pm$ 0.6	21.3 $\pm$ 1.1
Position error px $\downarrow$	1.496 $\pm$ 0.013	2.511 $\pm$ 0.048	1.504 $\pm$ 0.012	2.452 $\pm$ 0.065
Reciprocity $\uparrow$	51.9 $\pm$ 7.0	17.4 $\pm$ 1.8	76.8 $\pm$ 5.2	17.4 $\pm$ 2.8
Displacement err. px $\downarrow$	4.62 $\pm$ 1.77	62.02 $\pm$ 1.96	2.77 $\pm$ 0.25	56.08 $\pm$ 6.15

Table 3: We evaluate the base configuration under both visual conditions, each against its own ceiling. Metrics in

[0,1]

are reported

\times 100

. The two conditions use separately trained frozen autoencoders, so their ceilings are not comparable.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

#	Experiment	Views	Objective	Curriculum	Configuration
1	Baseline	embedded	teacher forcing	one-stage	multi_stream
2	Tiled views	tiled	teacher forcing	one-stage	…_tiled
3	Diffusion forcing	embedded	diffusion forcing	one-stage	…_df
4	Two-stage	embedded	teacher forcing	two-stage	…_twostage
5	Tiled + DF	tiled	diffusion forcing	one-stage	…_tiled_df
6	Tiled + DF + 2-stage	tiled	diffusion forcing	two-stage	…_triple

Appendix

Table 4: Experiment matrix. Every run trains the same world model (27.5M parameters embedded, 27.7M tiled) under the same 48-hour single-GPU allocation in the black-background condition. Only the listed ingredients differ. Two-stage runs divide the budget between single-agent pretraining and subsequent multi-agent training.

Transformer architecture
Hyperparameter	Value
Parameters	27.5M embedded / 27.7M tiled
Hidden dimension	512
Blocks	8
Attention heads	8 (head dim. 64, QK RMS-norm)
Feed-forward	2048, GELU

Appendix

Table 5: World model hyperparameters shared across all experiments. Table 4 lists the per-experiment ingredient flags.

Hyperparameter	Value
Architecture	causal 3D convolutional VAE (Wan-style)
Base width	32 (channel multipliers 1,2,4,4 and 2 residual blocks/stage)
Input patchify	$2\times 2$ spatial
Compression	$16\times$ spatial, $4\times$ temporal (causal, $1+4k$ )
Latent	8 channels, $128^{2}\times T\rightarrow 8^{2}\times(1+(T-1)/4)$
Loss	$L_{1}$ reconstruction $+$ KL, weight $10^{-6}$

Appendix

Table 6: Video autoencoder hyperparameters. The architecture, loss, and optimizer rows are shared by both condition-specific autoencoders, while the training-data and training-length rows describe the black-background checkpoint (epoch 81, validation loss 0.0042), which every run except the gradient-background one uses. The gradient-background autoencoder was trained for longer and on streamed rather than recorded clips (Appendix E ).

Parameter	Value	Role
Detection
Foreground threshold	$\max(R,G,B)>0.25$	separates boids from background
Minimum component size	2 px	rejects single-pixel noise
Maximum elongation	5	rejects arena border lines
Maximum extent	40 px	rejects arena corners
Border band / minimum size	5 px / 25 px	rejects border clippings

Appendix

Table 7: Detector, matching, and identity parameters. All metrics in Appendix J use these settings, and the ceiling and floor references use the same pipeline.

Panel of Figure 4 — Figure 4: One camera agent’s egocentric view across a 10-second rollout, sampled approximately every $2$ s. The top row shows ground-truth latents decoded through the frozen autoencoder (ceiling), the middle row shows diffusion forcing, the model with the lowest cross-view displacement error, and the bottom row shows the floor, in which each view is rolled out independently by a single-agent model. All rows share the same ground-truth context, so the leftmost column is identical and all subsequent frames are generated. The floor hallucinates camera-agent colors and does not reliably preserve the focal agent’s hue, whereas the diffusion-forcing model maintains the focal identity and renders most neighbors as uncolored boids.

Video World Models

Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.

See all 257 papers

MultiWorld: Scalable Multi-Agent Multi-View Video World Models
Apr 20, 2026Haoyu Wu, Jiwen Yu, Yingtian Zou +1Multi-View Consistency Action-Conditioned World Models
The University of Hong Kong · Sreal AI
MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data
Jun 1, 2026Teng Hu, Mingchun Lu, Yating Wang +6Multi-View Consistency Video Diffusion Models
Shanghai Jiao Tong University · Zhejiang University · Nanyang Technological University
Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players
May 27, 2026Fangfu Liu, Kai He, Tianchang Shen +7Interactive World Models Video World Models
NVIDIA · Tsinghua University · University of Toronto +1

Apr 20, 2026J/K move · Enter open · S save

MultiWorld: Scalable Multi-Agent Multi-View Video World Models

Haoyu Wu, Jiwen Yu, Yingtian Zou, Xihui Liu

The University of Hong Kong · Sreal AI

Multi-View Consistency Action-Conditioned World Models Multi-Agent Control Controllable Video Generation Video World Models

Open paper AskPDF arXiv

Video world models have achieved remarkable success in simulating environmental dynamics in response to actions by users or agents. They are modeled as action-conditioned video generation models that take historical frames and current actions as input to predict future frames. Yet, most existing approaches are limited to single-agent scenarios and fail to capture the complex interactions inherent in real-world multi-agent systems. We present MultiWorld, a unified framework for multi-agent multi-view world modeling that enables accurate control of multiple agents while maintaining multi-view consistency. We introduce the Multi-Agent Condition Module to achieve precise multi-agent controllability, and the Global State Encoder to ensure coherent observations across different views. MultiWorld supports flexible scaling of agent and view counts, and synthesizes different views in parallel for high efficiency. Experiments on multi-player game environments and multi-robot manipulation tasks demonstrate that MultiWorld outperforms baselines in video fidelity, action-following ability, and multi-view consistency. Project page: https://multi-world.github.io/

Abstract

Video World Models

Explore similar work

MultiWorld: Scalable Multi-Agent Multi-View Video World Models

MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data

Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players