Readable Is Not Steerable: Pre-Registered Tests of Physical Structure in Learned Fluid Representations
Organizations: Modint Intelligence
Abstract
Self-supervised world models are often said to organize their latents around physical quantities. We test this on 2D decaying Navier-Stokes flow in three pre-registered studies, each tested once on held-out data. In Study 1, a joint-embedding predictive architecture (JEPA) trained to predict the flow fails the pre-registered criteria for organizing its latent around fluid quantities better than an untrained network of the same architecture (not supported); a JEPA trained on non-physical linear dynamics and a static invariance model that never sees time do as well or better. In Study 2, plain autoencoders that render the field make enstrophy prominent in their slowest unsupervised latent directions (best r 0.80-0.86 vs 0.38 untrained; fresh seeds, sealed data). Adding VICReg lowers this prominence in every comparison, though not by the pre-registered margin for vorticity, while raising the autoencoders' linear decodability (partially supported). In Study 3, we build steerable directions ("dials") by routing: the decoder gets overall scale only through a small conditioning code, while its other pathway carries a degraded copy of spectral shape. Label-free rules find a loudness dial in all 8 trained models and a shape dial in 3 of 5 fresh seeds on sealed data, against a pre-registered bar of 4; neither control shows a shape dial (partially supported). Across the studies, what a latent contains and what can be controlled through it come apart. In exploration, a supervised direction reading spectral shape almost perfectly moved nothing when steered. During the build, the loudness dial had to be found by its effect on the output (closest to a pure rescaling), not by data variance; for shape, a variance rule and an effect-constrained rule each worked in 3 of 5 seeds. The program used about 70 logged hours on one consumer GPU; all pre-registrations, deviations, code and outputs are public.
Figures & tables
| Property | Measure | Used in |
|---|---|---|
| Decodability | Ridge-regression probe from the -scored latent to a target; on held-out trajectories | Studies 1–3 |
| Prominence | Correlation between a target and the latent’s slowest unsupervised directions | Studies 1–2 |
| Steerability | Change in the decoded field when the latent is moved along a direction | Study 3 |
| Model | Training signal | Role |
|---|---|---|
| JEPA | Predicts the target encoder’s feature map of frame from frames , ; EMA target; VICReg variance and covariance terms; no pixel loss | Main model |
| Pixel predictor | Decodes the velocity field of frame ; mean squared error | Comparison |
| Untrained | None (the Kaiming initialization trained models start from) | Control |
| Pooled JEPA | As the JEPA, but predicts the pooled 128-d vector | Ablation |
| Static VICReg | Invariance between two copies of one frame under random shifts and square symmetries; never sees time | Control |
| Non-physical JEPA | As the JEPA, trained on the same initial fields evolved by random damped, phase-rotating linear dynamics on each Fourier mode, with a hidden rate per trajectory | Control |
| Criterion | JEPA | Untrained | Needed | Outcome |
|---|---|---|---|---|
| S1 log enstrophy | 0.930 | 0.727 | 0.864 | counts: [ , ] |
| S1 log palinstrophy | 0.746 | 0.437 | 0.637 | counts: [ , ] |
| S1 log(1 + vortex count) | 0.752 | 0.531 | 0.731 | counts: [ , ] |
| S1 Okubo–Weiss fraction (mandatory) | 0.507 | 0.456 | 0.656 | does not count: [ , ] |
| S1 log dissipation | 0.176 | 0.134 | 0.334 | does not count: [ , ] |
| S1 | 3 of 5, incl. Okubo–Weiss | fails (3 of 5, Okubo–Weiss not among them) |
| Quantity | JEPA non-physical JEPA | JEPA static VICReg | JEPA pooled JEPA |
|---|---|---|---|
| log enstrophy | [ , ] | [ , ] | [ , ] |
| log palinstrophy | [ , ] | [ , ] | [ , ] |
| log(1 + vortex count) | [ , ] | [ , ] | [ , ] |
| Okubo–Weiss fraction | [ , ] | [ , ] | [ , ] |
| log dissipation | [ , ] | [ , ] | [ , ] |
| , frame-pair (no CI) |
| Hypothesis | Rule | Result | Outcome |
|---|---|---|---|
| H1, velocity autoencoder | E1 0.70; control + 0.25; CI > 0 | E1 0.860; [ , ] | pass |
| H1, vorticity autoencoder | same | E1 0.799; [ , ] | pass |
| H1 | both pass | holds | |
| H2, velocity: plain VICReg | 0.20; CI > 0 | [ , ] | pass |
| H2, vorticity: plain VICReg | same | [ , ] | fail |
| H2 | both pass | fails |
| Content shape | Decoder | Shape-knob (median) | Both dials, all criteria | error | |
|---|---|---|---|---|---|
| 0 | 0.97 | sub-pixel | 0.07 | 0 of 3 | 0.015 |
| 0.2 | 0.61 | resize | 0.88 | 3 of 3 | 0.022 |
| 0.5 | 0.29 | sub-pixel | 0.93 | 0 of 3 (artefacts) | 0.041 |
| 0.5 | 0.29 | resize | 0.96 | 3 of 3 | 0.039 |
| 1 | 0.18 | sub-pixel | 0.88 | 0 of 3 | 0.125 |
| 2 | 0.07 | sub-pixel | 0.93 | 0 of 3 | 0.374 |
| Seed | Loudness dial | Shape dial | Both | loudness / shape, general | loudness / shape, matched energy | What failed |
|---|---|---|---|---|---|---|
| 100 | yes | yes | yes | 0.00 / 0.94 | 0.03 / 0.94 | — |
| 101 | yes | yes | yes | 0.00 / 0.98 | 0.04 / 0.78 | — |
| 102 | yes | no | no | 0.00 / 0.41 | 0.05 / 0.46 | shape, both sets: effect, monotonicity, cycle correlation and gain, grid artefacts (3.5% / 4.7% of frames); envelope on matched energy |
| 103 | yes | yes | yes | 0.00 / 1.00 | 0.05 / 0.81 | — |
| 104 | yes | no | no | 0.00 / 0.96 | 0.03 / 0.96 | shape, matched energy only: envelope (tolerance ) |
| Count | 5 of 5 | 3 of 5 | 3 of 5 | pass 4 partially supported |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Quantity (in-distribution test, raw pooled latent) | Untrained | JEPA | Pixel | Pooled JEPA | Static VICReg | Non-physical JEPA |
|---|---|---|---|---|---|---|
| log enstrophy, | 0.727 | 0.930 | 0.890 | 0.862 | 0.973 | 0.956 |
| log palinstrophy, | 0.437 | 0.746 | 0.733 | 0.665 | 0.900 | 0.802 |
| log(1 + vortex count), | 0.531 | 0.752 | 0.521 | 0.637 | 0.842 | 0.789 |
| Okubo–Weiss fraction, | 0.456 | 0.507 | 0.270 | 0.423 | 0.673 | 0.504 |
| log dissipation, | 0.134 | 0.176 | 0.156 | 0.148 | 0.191 | 0.178 |
| , frame pair, | 0.481 | 0.457 | 0.710 | 0.317 | 0.699 | 0.522 |
| Item | Value |
|---|---|
| Exploration data | 1,006 trajectories (378 inviscid); matched-energy set of 300 inviscid trajectories at = 0.2893 |
| JEPA enstrophy by readout: pooled / map mean std / map PCA-512 | 0.353 / 0.303 / 0.295 |
| Kill gate (a), exploration set: pixel vs untrained | 0.829 vs 0.418, gap [ , ]: pass |
| Kill gate (b), matched energy: pixel vs untrained | 0.979 vs 0.177, gap [ , ]: pass |
| E1 by horizon, imaginer / summarizer ( , 4, 16, 32) | 0.68 (two seeds) / 0.31, 0.80 / 0.37, 0.69 (one seed) / 0.38, 0.76 / 0.46 |
| Imaginer summarizer gap by k | , , , (all intervals above zero) |
| Stage | What changed | What it showed |
|---|---|---|
| 0 | Measure 21 existing models | 0 of 84 directions are dials (best effect 0.086); the supervised -direction is readable ( 0.97–0.99) but moves nothing |
| 1 | Channel-split latent, two render heads, FiLM variant | No dials; normalized content still carries loudness ( 0.99) |
| 2 | Separate branches, scale-free content | Loudness dial in 3 of 3 seeds (later withdrawn by the artefact criterion); the dial code holds at 0.95 but steering it does not move |
| 2b | Whitened content | PCA shape dial in 2 of 3 seeds (1 of 3 after the artefact criterion); no clean loudness dial; rendering 3.8 times the VICReg autoencoder |
| A | No training: loudness by symmetry; grid-artefact criterion | Pure loudness dial in 2 of 3 seeds; earlier loudness dials fail on tails (checkerboards in 5–14% of frames at ) |
| A ′ | No training: shape = top PC after projecting out loudness in the latent | 50/50 mixtures ( 0.50–0.54) |
| Seed | Shape dial: per unit , exploration / matched | Loudness dial: per unit | Artefact frames | ||
|---|---|---|---|---|---|
| 0 | / | 0.99 / 0.95 | / | 0.00 / 0.03 | 0% |
| 1 | / | 0.87 / 0.82 | / | 0.00 / 0.02 | 0% |
| 2 | / | 0.88 / 0.86 | / | 0.00 / 0.05 | 0% |
| Item | Value |
|---|---|
| PC2 shape rule: seeds with both dials | 3 of 5 (100, 102, 104) |
| PC2 failures | 101: 0.63 on the general set; 103: 0.36 / 0.30 (mostly loudness) |
| between the loudness-null and PC2 directions, seeds 100–104 | 0.99, 0.95, 0.35, 0.87, 0.96 (computed after the run) |
| , loudness-null / PC2, seeds 100–104 | 5.12 / 5.12, 5.22 / 5.16, 2.11 / 4.47, 5.34 / 5.05, 3.70 / 3.84 |
| control: loudness dial; loudness-null and PC2 effects | 3 of 3 (effect 0.78–0.85, 0.01); 0.02–0.10 and 0.12–0.23, all failing the cycle check |
| Untrained control | no dials; artefacts in 100% of frames; rendering error 1.9–16.2 |
| Seed | Fraction of PC2’s shape effect removed by projecting out (linear / steered) | Variance after projection: top direction / along PC2 | , loudness-null / PC2 | |
|---|---|---|---|---|
| 100 | 0.009 | / | 2.91 / 2.90 | 5.12 / 5.12 |
| 101 | 0.057 | / | 2.44 / 2.43 | 5.22 / 5.16 |
| 102 | 0.000 | / | 2.39 / 1.48 | 2.11 / 4.47 |
| 103 | 0.117 | / | 2.32 / 2.17 | 5.34 / 5.05 |
| 104 | 0.001 | / | 3.08 / 2.94 | 3.70 / 3.84 |
| When | Event |
|---|---|
| Before Study 1 | About 70 CPU planning pilots, summarized in the log, not reproducible from the repository |
| Study 1 | Pre-registration committed before training; amendment A1 seven minutes later (no-collapse rule limited to collapse-prone objectives) |
| Study 1 | Validation run crashed at its final console print after writing complete outputs; output-only fix |
| Study 1 | Validation results seen before the test; afterwards only the encoding fix and an unscored traversal readout changed |
| Exploration | Stage 2 launch aborted by a line-ending bug before any training; restarted from scratch |
| Exploration | Budget raised four times (from 14 to 30 GPU-hours), each before the work it covered |
| Line of work | Logged hours |
|---|---|
| Study 1 | 11.5 |
| Exploration and Study 2 | 29.6 |
| Dial build (exploration) | 17.8 |
| Study 3 confirmation | 8.8 |
| Post hoc analysis | 2.27 |
| Total | 70 |
Representation Learning
Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.
See all 855 papers