Structural-Test Geometry as an Inductive Bias for Partially Observable World Models
Organizations: Harvard University (Undergraduate Student), Cambridge, MA, USA
Abstract
Predictive world-model objectives need not organize latent histories according to distinctions required for later control. With privileged simulator access during training, should response-derived structural information be predicted, decoded, or imposed directly on latent geometry--and how do these choices compare with structured belief distillation? We define posterior-marginalized structural tests whose response distributions induce a pseudometric over histories, then train quotient-preserving (QP) world models by aligning shared latent Euclidean distances to that metric. Across a fixed six-task diagnostic suite, the largest differences appear under held-out structural variation: QP improves planning success by 26.1 points over metric decoding and 6.4 points over structured belief distillation. On GridWorld, metric decoding accurately recovers held-out structural distances () while its native latent geometry remains more distorted than QP's (paired IQD difference ), showing that relational recoverability and direct geometric alignment can diverge.
Figures & tables
| Domain | TR | MD | BD-S | QP (ours) |
|---|---|---|---|---|
| GridWorld | ||||
| Key–Box | ||||
| Hidden-Mass CartPole | ||||
| Visual GridWorld | ||||
| MiniWorld–Room | ||||
| MiniGrid–HiddenKey |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Domain | Method | ID [95% CI] | Shift [95% CI] |
|---|---|---|---|
| GridWorld | RNC | ||
| TR | |||
| MD | |||
| BD-S | |||
| QP | |||
| Key–Box | RNC |
| Domain | Method | ID [95% CI] | Shift [95% CI] |
|---|---|---|---|
| Visual GridWorld | RNC | ||
| TR | |||
| MD | |||
| BD-S | |||
| QP | |||
| MiniWorld–Room | RNC |
| Method | Intended target metric | Value | Comparison reference |
|---|---|---|---|
| TR | response-distribution TV | split-half TV | |
| MD | decoding | RMSE ; | constant RMSE |
| MD | native IQD | QP IQD | |
| Paired MD–QP | IQD difference | — | |
| BD-S | posterior NLL (nats) | prior NLL | |
| QP | native IQD | best-found SMACOF |
| Condition | ID [95% CI] | Shift [95% CI] |
|---|---|---|
| BD-S (bivariate Gaussian) | ||
| BD-S (factorized categorical ) | ||
| QP | ||
| QP, family-balanced weights | ||
| QP, uniform pair sampling | ||
| QP, 500 spurious nuisance coordinates |
| Diagnostic/subset | 95% CI | |
|---|---|---|
| IQD, all models (unstratified) | ||
| IQD, pooled within-cell ranks (non-QP) | ||
| High action-value range |
| Architecture | Method | IQD [95% CI] | ID [95% CI] | Shift [95% CI] |
|---|---|---|---|---|
| TD-MPC style | Value only | |||
| TD-MPC style | RNC | |||
| TD-MPC style | QP | |||
| RSSM reference | QP |
| Domain | Horizon | Hidden factor | Train/val episodes | Observation | |
|---|---|---|---|---|---|
| Hidden-Goal GridWorld | 50 | 5 | 4 goal corners | 20k/2k | coordinates |
| Key–Box | 50 | 4 | key room/possession + door | 20k/2k | symbolic |
| Hidden-Mass CartPole | 200 | 2 | mass in | 10k/1k | continuous |
| Visual GridWorld | 50 | 5 | 4 goal corners | 20k/2k | RGB |
| MiniWorld–Room | 100 | 4 | 4 object locations | 10k/1k | first-person RGB |
| MiniGrid–HiddenKey | 100 | 3 | 3 key regions | 10k/1k | egocentric |
| Domain | Probe settings | Continuations | Outcome | |
|---|---|---|---|---|
| GridWorld | corner teleport (4), centre teleport (1), no-op (1) | 5 each (3 random walks, 2 heuristics), plus 70 random-walk variants | revealed corner (4-way) | 100 |
| Key–Box | grab key (1), open door (2), door-state probe (2), no-op (1) | 5 each, plus 70 random-walk variants | binary | 100 |
| CartPole | set mass , no-op (1) | 4 each, plus 76 random continuations | scalar return | 100 |
| Visual GridWorld | rendered corner teleport (4), centre teleport (1), no-op (1) | GridWorld construction with rendered observations | revealed corner (4-way) | 100 |
| MiniWorld | location teleport (4), look-at location (4), no-op (1) | 5 each, plus 55 random-walk variants | object visible | 100 |
| MiniGrid | reveal key (3), quadrant teleport (3), no-op (1) | 5 each, plus 65 random continuations | key quadrant | 100 |
| Domain | Training values/compositions | Held-out shift |
|---|---|---|
| GridWorld | goals | goal |
| Key–Box | key room A; door A–B | key room B; door B–C |
| CartPole | mass | mass |
| Visual GridWorld | goals | goal |
| MiniWorld–Room | desk, shelf, table | cabinet; new room configuration |
| MiniGrid–HiddenKey | top-left, top-right | bottom-centre |
| Domain | BD-S target | Prediction family | Auxiliary loss |
|---|---|---|---|
| GridWorld / Visual GridWorld | goal coordinate | bivariate Gaussian | posterior-coordinate NLL |
| Key–Box | key room, possession, door state | factorized categoricals | summed cross-entropy |
| CartPole | pole mass on | five-component Gaussian mixture | posterior-mass NLL |
| MiniWorld–Room | object coordinate | bivariate Gaussian | posterior-coordinate NLL |
| MiniGrid–HiddenKey | key-region coordinates | factorized categorical | summed cross-entropy |
| Domain | Miss | Grid maximum | |
|---|---|---|---|
| GridWorld | |||
| Key–Box | |||
| CartPole |
| Method | Candidate coefficients | Selected coefficient |
|---|---|---|
| TR | ||
| MD | ||
| BD-S | ||
| QP |
| Method | RNC | TR | MD | BD-S | QP |
|---|---|---|---|---|---|
| GPU-hours/seed | |||||
| Total GPU-hours |
World Model Learning
World model learning builds predictive models of how environments change under actions to support planning, simulation, and decision-making.
See all 419 papers