Recovered or Memorized? Investigating Dynamical Consistency and Faithfulness in World Model Latents
Organizations: Carleton University · Algoverse AI Research
Abstract
World models develop internal simulators that can learn the structure and dynamics of an environment. It has been demonstrated that their internal representations are both meaningful and causal, but are these representations faithful to the governing dynamics of the environment they are learning? We introduce Anchored Recovery Certification (ARC), a procedure that utilizes a discrete Fréchet distance calculation with ground truth to evaluate dynamical consistency on candidate world models up to an affine gauge. We use the procedure on TD-MPC2 (decoder-free world model) that has learned the pendulum swing on an observation lift under varying hyperparameters at k = 5, 10, 25 steps. The model is then steered within a PCA-derived intervention subspace and compared against a ground-truth simulator using relative error to investigate dynamical faithfulness. ARC sufficiency held for TD-MPC2 most consistently at k = 10, with six of seven constraints statistically significant after Holm-Bonferroni correction, while the relative error between the simulator and the causal latent remained substantial. Further research could be conducted with decoder world models such as DreamerV3 (reconstruction loss) to investigate whether the results differ.
Figures & tables
| Constraint | (low / mid / high ) |
|---|---|
| Horizon (rollout length) | 1 / 3 / 5 |
| Latent Dim (bottleneck) | 128 / 512 / 1024 |
| Data Augmentation ( ) | 0 / 0.05 / 0.15 |
| Constraint | k=5 | k=10 | k=25 |
|---|---|---|---|
| Baseline | 0.6 | 0.577 | -0.0132 |
| Short Horizon | 0.615 | 0.54 | -0.421 |
| Long Horizon | 0.539 | 0.523 | 0.0682 |
| Narrow Latent Dim | 0.744 | 0.642 | -0.105 |
| Wide Latent Dim | 0.571 | 0.581 | 0.106 |
| No Data Augmentation | 0.431 | 0.403 | -0.171 |
| Constraint | |||
|---|---|---|---|
| Baseline Null | |||
| Short Horizon Null | |||
| Long Horizon Null | |||
| Narrow Latent Dim Null | |||
| Wide Latent Dim Null | |||
| No Data Augmentation Null |
| Constraint | |||
|---|---|---|---|
| Baseline | |||
| Short Horizon | |||
| Long Horizon | |||
| Narrow Latent Dim | |||
| Wide Latent Dim | |||
| No Data Augmentation |
| Constraint | ||||
|---|---|---|---|---|
| Baseline | ||||
| Short Horizon | ||||
| Long Horizon | ||||
| Narrow Latent Dim | ||||
| Wide Latent Dim | ||||
| No Data |
| Constraint | |||
|---|---|---|---|
| Baseline | 1.00 | 1.61 | 2.09 |
| Short Horizon | 0.94 | 1.50 | 2.27 |
| Long Horizon | 1.05 | 1.57 | 1.95 |
| Narrow Latent Dim | 0.92 | 1.38 | 1.93 |
| Wide Latent Dim | 0.89 | 1.42 | 1.89 |
| No Data Augmentation | 0.84 | 1.29 | 1.73 |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.499 | 0.727 | 0.561 | 0.546 | 0.665 | |
| Short Horizon | 0.632 | 0.532 | 0.715 | 0.519 | 0.675 | |
| Long Horizon | 0.41 | 0.447 | 0.514 | 0.681 | 0.645 | |
| Narrow Latent Dim | 0.671 | 0.738 | 0.962 | 0.615 | 0.733 | |
| Wide Latent Dim | 0.581 | 0.626 | 0.615 | 0.528 | 0.505 | |
| No Data Augmentation | 0.027 | 0.591 | 0.46 | 0.421 | 0.657 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.492 | 0.684 | 0.522 | 0.517 | 0.671 | |
| Short Horizon | 0.541 | 0.521 | 0.64 | 0.389 | 0.609 | |
| Long Horizon | 0.386 | 0.427 | 0.495 | 0.681 | 0.627 | |
| Narrow Latent Dim | 0.608 | 0.67 | 0.658 | 0.568 | 0.702 | |
| Wide Latent Dim | 0.573 | 0.592 | 0.658 | 0.521 | 0.562 | |
| No Data Augmentation | -0.105 | 0.577 | 0.475 | 0.349 | 0.717 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.145 | -0.362 | -0.003 | -0.124 | 0.278 | |
| Short Horizon | -0.581 | -0.32 | -0.388 | -0.638 | -0.178 | |
| Long Horizon | -0.0746 | -0.162 | 0.179 | 0.308 | 0.0907 | |
| Narrow Latent Dim | -0.0566 | -0.0701 | -0.00652 | -0.291 | -0.102 | |
| Wide Latent Dim | 0.145 | 0.0999 | 0.195 | -0.274 | 0.357 | |
| No Data Augmentation | -0.671 | 0.287 | -0.0225 | -0.271 | 0.592 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline Null | 0.412 | 0.531 | 0.393 | 0.459 | 0.53 | |
| Short Horizon Null | 0.491 | 0.464 | 0.563 | 0.495 | 0.537 | |
| Long Horizon Null | 0.389 | 0.394 | 0.418 | 0.524 | 0.505 | |
| Narrow Latent Dim Null | 0.382 | 0.469 | 0.527 | 0.471 | 0.537 | |
| Wide Latent Dim Null | 0.442 | 0.523 | 0.593 | 0.515 | 0.525 | |
| No Data Augmentation Null | 0.517 | 0.492 | 0.395 | 0.505 | 0.567 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
| Baseline | 0.204 | 0.289 | 0.206 | 0.303 | 0.383 | |
| Short Horizon | 0.278 | 0.181 | 0.384 | 0.277 | 0.367 | |
| Long Horizon | 0.164 | 0.185 | 0.229 | 0.362 | 0.324 | |
| Narrow Latent Dim | 0.106 | 0.277 | 0.323 | 0.293 | 0.345 | |
| Wide Latent Dim | 0.226 | 0.322 | 0.431 | 0.283 | 0.363 | |
| No Data Augmentation | 0.288 | 0.292 | 0.0507 | 0.31 | 0.403 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline Null | 0 | 0 | 0 | 0 | 0 | |
| Short Horizon Null | 0 | 0 | 0 | 0 | 0 | |
| Long Horizon Null | 0 | 0 | 0 | 0 | 0 | |
| Narrow Latent Dim Null | 0 | 0 | 0 | 0 | 0 | |
| Wide Latent Dim Null | 0 | 0 | 0 | 0 | 0 | |
| No Data Augmentation Null | 0 | 0 | 0 | 0 | 0 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.299 | 0.307 | 0.248 | 0.277 | 0.391 | |
| Short Horizon | 0.317 | 0.263 | 0.373 | 0.273 | 0.326 | |
| Long Horizon | 0.264 | 0.283 | 0.267 | 0.354 | 0.389 | |
| Narrow Latent Dim | 0.261 | 0.308 | 0.315 | 0.29 | 0.364 | |
| Wide Latent Dim | 0.323 | 0.339 | 0.34 | 0.218 | 0.391 | |
| No Data Augmentation | 0.384 | 0.361 | 0.23 | 0.342 | 0.422 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.0727 | -0.0344 | -0.0381 | 0.104 | 0.173 | |
| Short Horizon | 0.0708 | -0.0685 | 0.147 | 0.0188 | 0.124 | |
| Long Horizon | -0.0159 | 0.0143 | -0.0177 | 0.185 | 0.226 | |
| Narrow Latent Dim | -0.0726 | 0.039 | 0.0624 | 0.0849 | 0.152 | |
| Wide Latent Dim | 0.0783 | 0.0993 | 0.0998 | -0.0384 | 0.232 | |
| No Data Augmentation | 0.113 | 0.129 | -0.196 | 0.134 | 0.207 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -0.765 | -0.987 | -0.83 | -0.669 | -0.628 | |
| Short Horizon | -0.817 | -0.945 | -0.817 | -0.886 | -0.578 | |
| Long Horizon | -0.992 | -0.875 | -0.838 | -0.622 | -0.579 | |
| Narrow Latent Dim | -1.191 | -0.848 | -0.829 | -0.719 | -0.554 | |
| Wide Latent Dim | -0.888 | -0.768 | -0.783 | -0.845 | -0.513 | |
| No Data Augmentation | -1.085 | -0.862 | -1.210 | -0.846 | -0.587 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.412 | 0.531 | 0.394 | 0.46 | 0.531 | |
| Short Horizon | 0.491 | 0.464 | 0.563 | 0.495 | 0.537 | |
| Long Horizon | 0.391 | 0.394 | 0.42 | 0.524 | 0.505 | |
| Narrow Latent Dim | 0.382 | 0.469 | 0.527 | 0.471 | 0.537 | |
| Wide Latent Dim | 0.442 | 0.523 | 0.593 | 0.516 | 0.525 | |
| No Data Augmentation | 0.517 | 0.492 | 0.396 | 0.506 | 0.567 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0.204 | 0.29 | 0.207 | 0.306 | 0.383 | |
| Short Horizon | 0.279 | 0.181 | 0.384 | 0.277 | 0.367 | |
| Long Horizon | 0.168 | 0.186 | 0.232 | 0.362 | 0.326 | |
| Narrow Latent Dim | 0.108 | 0.278 | 0.323 | 0.295 | 0.347 | |
| Wide Latent Dim | 0.227 | 0.323 | 0.432 | 0.284 | 0.363 | |
| No Data Augmentation | 0.289 | 0.293 | 0.0524 | 0.312 | 0.403 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -0.625 | -0.843 | -0.617 | -0.544 | -0.408 | |
| Short Horizon | -0.763 | -0.868 | -0.629 | -0.821 | -0.468 | |
| Long Horizon | -0.859 | -0.794 | -0.593 | -0.558 | -0.46 | |
| Narrow Latent Dim | -1.063 | -0.59 | -0.714 | -0.651 | -0.399 | |
| Wide Latent Dim | -0.759 | -0.605 | -0.574 | -0.872 | -0.381 | |
| No Data Augmentation | -0.882 | -0.67 | -1.029 | -0.594 | -0.351 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -2.22E-16 | -8.88E-16 | -4.44E-16 | -2.22E-16 | -4.44E-16 | |
| Short Horizon | 2.22E-16 | -4.44E-16 | -4.44E-16 | -6.66E-16 | 0 | |
| Long Horizon | -6.66E-16 | -2.22E-16 | -6.66E-16 | -4.44E-16 | -2.22E-16 | |
| Narrow Latent Dim | -2.22E-16 | -4.44E-16 | -2.22E-16 | -2.22E-16 | -4.44E-16 | |
| Wide Latent Dim | 0 | -2.22E-16 | 0 | 2.22E-16 | -2.22E-16 | |
| No Data Augmentation | -2.22E-16 | -2.22E-16 | -4.44E-16 | -4.44E-16 | -2.22E-16 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 0 | -8.88E-16 | -2.22E-16 | -4.44E-16 | 0 | |
| Short Horizon | 2.22E-16 | -6.66E-16 | -4.44E-16 | -4.44E-16 | -2.22E-16 | |
| Long Horizon | -6.66E-16 | -4.44E-16 | -4.44E-16 | -4.44E-16 | -2.22E-16 | |
| Narrow Latent Dim | -4.44E-16 | -4.44E-16 | 2.22E-16 | -4.44E-16 | -2.22E-16 | |
| Wide Latent Dim | 0 | 0 | 0 | 3.33E-16 | -2.22E-16 | |
| No Data Augmentation | -4.44E-16 | -4.44E-16 | -4.44E-16 | -4.44E-16 | 3.33E-16 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | 2.22E-16 | -2.22E-16 | 0 | -2.22E-16 | 2.22E-16 | |
| Short Horizon | 2.22E-16 | -4.44E-16 | -2.22E-16 | -4.44E-16 | -2.22E-16 | |
| Long Horizon | -6.66E-16 | -2.22E-16 | 0 | -4.44E-16 | 0 | |
| Narrow Latent Dim | 1.11E-16 | -2.22E-16 | 2.22E-16 | -2.22E-16 | 2.22E-16 | |
| Wide Latent Dim | 0 | -2.22E-16 | 1.11E-16 | 5.55E-16 | -4.44E-16 | |
| No Data Augmentation | -2.22E-16 | 0 | 0 | -2.22E-16 | 4.44E-16 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -5.399 | -6.135 | -5.823 | -5.719 | -5.458 | |
| Short Horizon | -6.159 | -5.206 | -5.822 | -6.239 | -4.675 | |
| Long Horizon | -5.169 | -5.265 | -5.734 | -5.463 | -4.878 | |
| Narrow Latent Dim | -4.779 | -4.580 | -4.502 | -4.487 | -3.563 | |
| Wide Latent Dim | -2.712 | -2.518 | -3.002 | -2.490 | -3.000 | |
| No Data Augmentation | -5.523 | -5.334 | -6.451 | -5.902 | -4.870 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -3.449 | -3.985 | -3.738 | -3.829 | -3.343 | |
| Short Horizon | -4.039 | -3.365 | -3.791 | -4.087 | -2.805 | |
| Long Horizon | -3.438 | -3.341 | -3.632 | -3.503 | -2.932 | |
| Narrow Latent Dim | -3.076 | -2.924 | -2.899 | -2.825 | -2.090 | |
| Wide Latent Dim | -1.587 | -1.409 | -1.742 | -1.398 | -1.650 | |
| No Data Augmentation | -3.541 | -3.375 | -4.187 | -3.814 | -2.868 |
| Constraint | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Uncertainty |
|---|---|---|---|---|---|---|
| Baseline | -2.886 | -3.285 | -3.069 | -3.268 | -2.552 | |
| Short Horizon | -3.473 | -2.853 | -3.176 | -3.413 | -2.202 | |
| Long Horizon | -3.064 | -2.857 | -2.907 | -2.930 | -2.210 | |
| Narrow Latent Dim | -2.636 | -2.382 | -2.449 | -2.369 | -1.483 | |
| Wide Latent Dim | -1.260 | -1.020 | -1.308 | -1.072 | -1.098 | |
| No Data Augmentation | -2.982 | -2.830 | -3.339 | -3.220 | -2.077 |
World Models
World models represent and predict how environments change, helping agents plan, act, and reason about possible futures.
See all 398 papers