Roz: A Taste Test for World Model Latents

Kenneth Ge; Dhruv Sheth; Kunvar Thaman

cs.ROOct 31, 2026

Roz: A Taste Test for World Model Latents

Authors: Kenneth Ge, Dhruv Sheth, Kunvar Thaman

Organizations: Pantheon

Abstract

State of the art latent dynamics world models achieve impressive performance for robot planning. But do they actually encode the physics of the world? In this paper, we show that popular world models today achieve strong results on tasks that reduce to image similarity but forget key details about objects in the scene like mass, deformation, friction, and bounciness. We achieve this by defining precisely what ought to be encoded in such a world model and introducing two new evaluation methods that can directly tease physical properties and easily scale to new ones. Our evaluation, which we call Roz, includes 10 contrastive tasks and 3 continuous tasks, totaling 1734 individual scenarios. We validate our eval on control tasks. On our own model, AnonWM, we find that performance roughly improves with training steps, while not necessarily being correlated with reconstruction MSE. We argue current model failures are a symptom of evaluation strategies (probes and planners) that are opaque, model-specific, and high-compute. Roz requires only a model that can ingest still frames or video, a way to extract latents, and a distance function between them, and it can be run in one pass. We hope future evaluations will adopt Roz's evaluation techniques. More broadly, we hope Roz can be a case study for a future where benchmarks allow rapid iteration and comparison across models.

Figures & tables

Figure 1: Seeing that the shelf contains a ball (left, Scenario 1) should give a different representation than not gaining this information (right, Scenario 2), even if the end state is the same.

Figure 2: The Paper Airplanes task, one of our real-world tasks. A and B are separate flights toward the same endpoint; C shows the airplane only at its final destination.

	V-JEPA 2		Qwen		Cosmos		FastWAM		DreamDojo †
scenario (N)	cos	L1	cos	L1	cos	L1	cos	L1	cos	L1
bowl (4)	1.00	1.00	1.00	1.00	0.75	0.75	1.00	1.00	1.00	1.00
path ind (30)	0.97	0.97	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
basic color (30)	0.90	0.90	0.97	1.00	1.00	1.00	1.00	1.00	1.00	1.00
basic position (30)	1.00	1.00	0.97	0.97	1.00	0.97	1.00	1.00	1.00	1.00
roll (30)	0.00	0.00	0.77	0.83	0.93	0.97	0.40	0.43	1.00	1.00

Table 1: Non-sweep contrastive task pass rates. Cells shaded on a colorblind-safe diverging scale around chance (

\approx

0.33): red = at/below chance, neutral gray

\approx

chance, darker blue = higher pass rate. † DreamDojo was evaluated on a 4-scenario subset of each family (Appendix C ); its pass rates are identical across all nine checkpoints we tested, so we report the 14B pretrain checkpoint at 10k steps.

family	distance	V-JEPA 2	Qwen	Cosmos	FastWAM	DreamDojo †
translation	nn cos	0.89	0.71	1.00	1.00	1.00
translation	nn L1	0.92	0.71	1.00	1.00	1.00
rotation	nn cos	0.47	0.76	1.00	0.91	1.00
rotation	nn L1	0.49	0.91	1.00	0.92	1.00
velocity	nn cos	0.50	0.53	0.16	0.04	0.32
velocity	nn L1	0.54	0.59	0.22	0.06	0.14

Table 3: Results for our continuous sweep evals. † DreamDojo 14B pretrain, 10k steps, on the two-ladder subset described in Appendix C .

size	stage	steps	MSE $\times 10^{-3}$	contrastive (of 42)	velocity nn cos
2B	pretrain	5k	6.6	30	0.29
2B	pretrain	10k	5.9	30	0.21
2B	YAM	1k	13.3	30	0.21
2B	YAM	10k	7.1	30	0.25
14B	pretrain	1k	8.0	30	0.21
14B	pretrain	5k	7.2	30	0.21

Table 4: DreamDojo checkpoint ladder. MSE is conditional reconstruction error on 20 held-out robot episodes (lower is better). Contrastive is the primary-setting pass count over 42 non-null cases; it is identical for every checkpoint and does not exceed the frozen VAE baseline.

pairwise wins (ties), row beats column
	worst	low	middle	best
mean rank (1 = best)	3.00	2.81	2.04	2.15
worst	—	6 (0)	3 (1)	3 (1)
low	7 (0)	—	4 (1)	3 (2)
middle	9 (1)	8 (1)	—	7 (1)
best	9 (1)	8 (2)	5 (1)	—

Table 5: Rank-based view of the PantheonWM checkpoint ladder

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

family	$8\times 8$ grid	global mean
basic color	1.00	1.00
basic position	1.00	1.00
bowl	1.00	1.00
path ind	1.00	0.50
roll	1.00	1.00
occlusion	0.00	0.00

Appendix

Table 6: DreamDojo 14B pretrain, 10k steps, final block, cosine, noise level 500, under two pooling choices. Bounce, Deform, and Collision fractions are over all six cases including the two identical-property controls, so 0.67 is the maximum attainable.

World Model Evaluation

World Model Evaluation measures whether learned simulators predict consistent, physically plausible, action-responsive futures that help agents plan and act.

See all 227 papers

RoboJEPA: Scaling Robotic Latent World Models
Oct 7, 2026Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan +9World Models Latent World Models
FAIR at Meta · Chandar Research Lab · Mila - Quebec AI Institute +2
stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation
May 20, 2026Lucas Maes, Quentin Le Lidec, Luiz Facury +9World Models ML Reproducibility
Mila & Université de Montréal · New York University · Universidade Federal de Minas Gerais +4
Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models
May 7, 2026Nilaksh, Saurav Jha, Artem Zholus +1World Model Learning World Models for Robotics
Chandar Research Lab · Mila – Quebec AI Institute · Polytechnique Montréal +1

Oct 7, 2026J/K move · Enter open · S save

RoboJEPA: Scaling Robotic Latent World Models

Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan, Sarath Chandar+8

FAIR at Meta · Chandar Research Lab · Mila - Quebec AI Institute+2

World Models Latent World Models World Model-Based Planning World Model Evaluation Joint-Embedding Predictive Architecture Cross-Embodiment Robot Learning

Open paper AskPDF arXiv

Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.

Abstract

World Model Evaluation

Explore similar work

RoboJEPA: Scaling Robotic Latent World Models

stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation

Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models