World models have demonstrated significant potential for perceiving and simulating complex environments. Despite their strong performance, the fundamental nature of their learned representations remains poorly understood. In this paper, we investigate the Platonic Representation Hypothesis within this domain by proposing the Predictive Consistency Assumption: we posit that the optimization of a shared state transition objective acts as a selective pressure that encourages heterogeneous models to converge toward a shared latent structure. Through systematic experiments with the DINO World Model (DINO-WM), in which we vary visual encoders to create heterogeneous models, we find that capable world models evolve toward geometrically similar internal structures. Moreover, via model stitching, we show that the internal features of one world model can be mapped to another with limited performance degradation, providing evidence of functional compatibility. Our findings suggest that the pursuit of predictive consistency can promote shared, transition-compatible latent structure across world models.
Figures & tables
Figure 1: The Platonic Representation Hypothesis on World Models. World models initialized from heterogeneous visual priors are optimized under the same transition objective ( st→st+1 ). Predictive consistency acts as a selective pressure that can drive their internal predictor representations toward a shared latent structure.
Figure 2: Evaluation environments. We evaluate on PointMaze , PushT , Wall , Granular , and Rope , covering navigation, contact-rich manipulation, and deformable-object dynamics under the DINO-WM benchmark protocol.
Model Prior
PointMaze (SR ↑ )
PushT (SR ↑ )
Wall (SR ↑ )
Rope (CD ↓ )
Granular (CD ↓ )
Heterogeneous Priors
ResNet
0.76
0.26
0.30
1.73
0.85
MAE (ViT-B)
0.88
0.72
0.56
1.27
0.30
SigLIP (ViT-B)
1.00
0.68
0.86
1.33
0.42
DINOv2 Scaling Variants
DINOv2-S (Anchor)
1.00
0.90
0.90
1.24
0.50
Table 1: Zero-shot Planning Success Rates at Epoch 10. Comparison of World Models initialized with heterogeneous vision priors across five environments. DINOv2-S is designated as the Platonic Anchor due to its superior average performance and robustness. Success Rates (SR) are reported for navigation/pushing tasks, while Chamfer Distance (CD) is used for Rope/Granular (where lower is better).
Figure 3: Evolution of Geometric Similarity toward the Platonic Anchor. Plots showing the Anchored m-kNN Similarity Score (Y-axis) vs. Training Epoch (X-axis) across five environments. The DINOv2-S World Model (10-epoch checkpoint) serves as the fixed reference Anchor. Results: Models initialized with DINOv2-B/G/L (blue/orange/green) exhibit the highest degree of alignment. Meanwhile, MAE (red) and SigLIP (brown) models demonstrate a distinct upward trend in similarity scores across most tasks, indicating progressive convergence. In contrast, the ResNet-based model (pink) consistently yields the lowest scores, suggesting a persistently different representational topology from the ViT-based Anchor in our setting.
Figure 4: Visualizing Representational Alignment via Spectral Embeddings. We illustrate the geometric structure of predictor hidden states using Laplacian Eigenmaps for five control environments: Granular , Rope , PointMaze , Wall , and Push-T . Internal representations from heterogeneous vision priors are projected into a low-dimensional space and aligned to the DINOv2 anchor via Procrustes transformation. In the bottom row ( PointMaze , Wall , Push-T ), a clear pattern of geometric similarity emerges among the transformer-based priors (DINOv2, SigLIP, and MAE), while the ResNet prior (orange) exhibits clear topological misalignment. In the top row, Granular displays a noisier convergence pattern, whereas the Rope environment manifests as a highly concentrated cluster across all models, reflecting the unique dynamical constraints of deformable object manipulation manifolds. Detailed visualization protocols and comprehensive results are provided in Appendix E .
Figure 5: Encoder-space baseline versus predictor-space alignment. For each environment, we compare m-kNN alignment between DINOv2-S and each candidate model in raw encoder space, predictor space at epoch 1, and predictor space at epoch 10. The drop from Encoder to Predictor@1 shows that the predictor does not simply inherit encoder geometry. The recovery from Predictor@1 to Predictor@10 then indicates that transition-prediction training induces a new cross-model alignment in predictor space.
Figure 6: Within-model encoder–predictor geometry. For each individual model, we compute m-kNN between its own raw encoder features and its predictor hidden states at epochs 1 and 10. The predictor retains some encoder information, but the epoch-10 states do not generally become more encoder-like. This rules out the interpretation that the cross-model recovery in Figure 5 is simply a return to raw encoder geometry.
Layer Combination
SSRPointMaze
SSRWall
DINO(1) + SigLIP(2–6)
0.86
0.30
DINO(1–2) + SigLIP(3–6)
0.86
0.38
DINO(1–3) + SigLIP(4–6)
0.90
0.62
DINO(1–3) + MAE(4–6)
0.84
0.42
SigLIP(1–3) + DINO(4–6)
0.76
0.64
SigLIP(1–3) + MAE(4–6)
0.88
0.50
Table 2: Stitched Success Rate (SSR) for different predictor layer combinations.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Representation stitching. We splice predictor stacks at depth k and train only lightweight MLP adapters to map between feature spaces while freezing all other parameters.
Figure 8: Anchor sensitivity analysis. We recompute anchored m-kNN trajectories using DINOv2-S@10, SigLIP@10, and MAE@10 as reference anchors, excluding trivial self-anchor trajectories. While absolute scores and some local rankings vary with the anchor, the qualitative pattern remains stable: strong ViT-based predictors stay closer to one another, whereas ResNet is consistently less aligned in most environments. This suggests that the alignment trend is not unique to the DINOv2-S reference, although the anchor should still be interpreted as a performance-motivated reference rather than a true Platonic ideal.
Figure 9: Longer-training stability of anchored m-kNN. We evaluate DINOv2-S, SigLIP, MAE, and ResNet at later checkpoints using DINOv2-S@10 as the reference. Later training does not uniformly increase alignment, but the broad relative structure remains stable: DINOv2-S stays highest, SigLIP and MAE form a middle tier, and ResNet remains least aligned in Wall, Rope, and Granular. The x-axis label “late” denotes epoch 50 for most runs and epoch 49 for two PushT runs.
Figure 10: kNN flow-matching visualization of predictor layer3 features across epochs. We use the DINO predictor at epoch 10 as the anchor and visualize kNN correspondences in a 2D embedding. Across epochs, several feature distributions become more overlapping with the DINO anchor, while the trajectories remain task- and model-dependent; predictors with stronger planning performance often exhibit more similar distributions.
A key design choice in visual world models is what information the latent state is encouraged to preserve. Models that learn a latent space with only a latent-prediction objective adapt to environment dynamics, but the predictive objective alone admits collapsed solutions and provides no explicit constraint on the visual information retained by the latent. Models built directly on pretrained visual features inherit a strong visual prior, but tie prediction and planning to a frozen state space. We propose Anchor-WM, which instead uses pretrained visual features as a teacher for learning a compact latent state rather than as the world-model state itself. Anchor-WM learns this compact latent with action-conditioned dynamics, while distilling the dense DINO representations through a training-only distillation head. This encourages the compact latent to retain visual information captured by the pretrained representation while discouraging representational collapse. At inference, the DINO encoder and distillation head are discarded, allowing prediction and planning to operate entirely in the learned latent space. Anchor-WM achieves the best average success rate across the four evaluated environments while enabling substantially faster planning than world models operating directly on DINO patch features. These results show that pretrained visual features can effectively guide world-model learning without defining the state space used for prediction and planning.
A world model matters to an agent only through the state it constructs. That state must preserve some information, discard other information, and support some future function: prediction, control, planning, memory, grounding, or counterfactual reasoning. This paper treats world-model research as latent state design under sufficiency constraints. We propose a functional taxonomy that groups methods by what their latent state is for, rather than by architecture or application domain: predictive embedding, recurrent belief state, object/causal structure, latent action interface, grounded planning interface, and memory substrate. These roles expose distinctions that architecture-based groupings hide, including the gap between predictive sufficiency and control sufficiency, and the gap between passive video prediction and counterfactual action modeling. The taxonomy supports an evaluation framework that judges a model by the sufficiency constraint its latent state was built to satisfy. We compare methods along seven axes: representation, prediction, planning, controllability, causal/counterfactual support, memory, and uncertainty. We use the resulting matrix as a diagnostic for what a latent state preserves, discards, and enables. The conclusion that follows is that an actionable world model is the one whose state construction matches the task, not the one that preserves the most information.
World models are widely proposed as core components for enabling agents to reason over long temporal and spatial horizons. Yet current evaluations often emphasize task performance while providing limited insight into whether learned world models acquire predictive latent dynamics that reflect the underlying structure of the environment. Here we introduce the concept of faithful latent world models as an equivariance condition between the environment's transition map and the learned latent dynamics. We propose latent rollout consistency (LRC) as a metric to evaluate action-conditioned latent prediction. Strikingly, we find that across the studied state of the art latent world models, none solves the task by actually faithfully modeling the dynamics of the environment.
Rodrigo Gonzalez Laiz, Steffen Schneider
Helmholtz Munich Munich Center for Machine Learning (MCML)