cs.LGOct 7, 2026
SaveDSReg: Provably Recovering Individual World Latents without Reconstruction
Organizations: University of Illinois Urbana-Champaign · Cold Spring Harbor Laboratory · Brown University · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen
Abstract
Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity. Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed. We close this gap: individual world latents can be provably recovered with no reconstruction, no decoder, and no labels. The key condition is Structural Diversity: different latents leave distinct dependency footprints on observations, just as no two snowflakes are alike. Building on the linear identifiability that LeJEPA provides, we prove that under Structural Diversity, DSReg (Dependency-Sparsity Regularization) recovers individual world latents up to signed permutation, without reconstruction or a decoder. It applies post hoc to any linearly identified representation, reusing trained checkpoints at no loss over joint training, and establishes the first fully identifiable JEPA that recovers every world latent. Moreover, as a condition on dependency footprints, Structural Diversity is strictly weaker than all structural conditions of prior identifiable latent variable models. Across synthetic regimes, world model probes, learned visual encoders, and external renderers, DSReg preserves dense prediction while improving individual-latent recovery and downstream use with scales.
Figures & tables
Figure 1: DSReg turns a mixed latent state into individual latents. World latents generate observations . LeJEPA identifies the latent state up to a linear transformation, , so each learned variable can remain a mixture of ground-truth latents. DSReg recovers individual latents up to signed permutation.
Figure 2: Dependency footprints. Each column of marks which observed variables one latent affects.
Figure 3: DSReg recovers individual latents wherever Structural Diversity holds, and only there. (a) DSReg (solid) is near ceiling at every , twenty runs. LeJEPA (dashed) stays mixed. (b) On analytic orbits , recovery succeeds wherever the condition holds and falls back to the shared pair subspace where it fails.
Figure 4: DSReg improves all six sparse-use probes. DSReg in teal, LeJEPA in gray. Tasks are (1) editing (top) and (2) control, prediction, and monitoring (bottom).
Figure 5: DSReg matches the supervised Procrustes oracle on encoders trained from pixels. (a) Latent-only rotations leave the variables mixed. DSReg closes the gap to the label-fitted oracle (fifteen seeds). (b) With fixed, the match holds within as the estimate widens to (five seeds).
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| MCC | step time (ms) | peak memory (GB) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| mat. | fact. | samp. | mat. | fact. | samp. | mat. | fact. | samp. | ||
| 128 | k | |||||||||
| 256 | k | |||||||||
| 512 | k | |||||||||
| 1024 | k | |||||||||
| 2048 | k | |||||||||
Table 1 : Scaling across latent dimension. The full estimated procedure, three seeds each. At the sampled columns use a compute-matched budget whose wall-clock stays at or below the materialized one; at smaller the standard budget already suffices, and sampling holds no advantage there since its per-step cost is nearly -independent while the materialized cost is what grows. All three modes optimize the same objective, and the materialized and factored columns agree on MCC to within everywhere, which is a live check that the factorization is exact. The factored mode trades time for memory, roughly halving peak memory at at about the step time, since it recomputes the neighbor differences on each pass. Axes without a panel of their own, each measured on its own sweep with its own baseline rather than on the rows above: raising the sample count to leaves recovery and step time flat to within seed noise ( at , ms throughout; at ), and growing the observed dimension to holds the sampled step at ms while MCC improves from to across the full doubling sweep.
Figure 6 : Scaling the full estimated procedure. (a) Recovery declines gently with ; open markers are the single-GPU frontier, where anchor count shrinks to fit memory. (b) Recovery against the neighborhood ratio at three scales; dashed lines mark the analytic score of an unrotated mixing, . (c) Sixteen anchors already match at . (d) The sampled step cost is nearly flat in while the materialized cost grows two orders of magnitude, with the curves crossing between and .
Figure 7 : Footprint regimes across dimension. Recovery follows Structural Diversity at every dimension: diverse, nested, and minimal-difference footprints stay at ceiling while identical footprints sit at the shared-subspace level, far above the decaying baseline of the unrotated representation; the identical pair’s span is still recovered, with CCA at least at every dimension of the full regime sweep.
Figure 8 : Recovery is insensitive to the optimizer’s choices. Recovery is insensitive to the sparsity surrogate and to the anchor budget; only the minimal-difference regime rewards more anchors, exactly where the evidence is scarcest. The two rightmost bars replace the relaxation by a hard support count at , minimized by coordinate search from the same six starts, and by that count used to refine the solution afterwards. Both series are DSReg selections, with color denoting the underlying footprint regime.
| Schedule | Latent MCC |
|---|---|
| LeJEPA (no rotation) | |
| Joint | |
| Joint, then decoupled fit | |
| Decoupled (default) |
Table 2 : Training-schedule ablation on Gaussian 3DShapes. Latent MCC (mean std), twenty runs per arm on a shared encoder trajectory; the rotation head never feeds back into the encoder.
Figure 9 : Corrupted Jacobians leave recovery at ceiling. Diverse footprints stay above through noise , and identical footprints stay at their predicted shared-subspace level. Nonlinear observation rows, so Assumption 2 holds; , five seeds. LeJEPA applies no rotation, so its curve is flat by construction.
Figure 10 : Sparse access improves while dense content is unchanged. At identical dense recovery ( for both methods), the rotation transforms individual-latent recovery, few-shot readout from eight labels, and sparse editing, each moving decisively in DSReg’s favor on the MLP benchmark at .
Figure 11 : Success strictly contains the guarantee. Success fraction of the thresholded criterion over the grid, five seeds. Solid curve: the guarantee boundary of Theorem 3 , with seed-range shading; circled cells satisfy the guarantee for every seed, and all of them succeed. The rotation used throughout the paper reproduces this map on all but four of the cells, again succeeding on every guaranteed cell.
Figure 12 : The excluded linear stratum is thin. MCC against the nonlinearity share with the measured no-cancellation margin overlaid (dotted, right axis). Where Assumption 2 fails ( , ) the criterion returns a mixed rotation; recovery crosses the level (dashed) at and reaches the ceiling by at every dimension, in both regimes, with exact and estimated Jacobians in close agreement.
| Benchmark | Dense | MCC | Few-shot | Sparse error | |
|---|---|---|---|---|---|
| Visual factors | 8 | ||||
| Object scenes | 8 |
Table 3 : Split-audited learned visual encoders. The analytic signature survives learning: dense recovery is unchanged while individual-latent recovery, few-shot readout, and sparse use improve, with every score computed on a held-out split (the rotation is fit on a separate split of estimated latents and fixed pooled-pixel features). Arrows are ten-run means for LeJEPA DSReg; sparse error is lower better.
Figure 13 : The analytic signature survives pixel training. The gain concentrates where individual latents matter: component-wise recovery and single-latent readout improve sharply, and sparse use follows. A negative few-shot value underperforms the held-out mean; the sparse-use score is normalized so that LeJEPA sits at one. On the pixel probes, sparse control error drops from to , five-step sparse rollout rises from to , and transition-surprise AUROC from to (LeJEPA / DSReg, ten runs).
Figure 14 : Training imbalance does not break the rotation. Long-tail and correlated training distributions leave the recovery gain intact, with the dense span fully preserved throughout.
| Method | Latent MCC | DCI | MIG | SAP | Info. | Dense |
|---|---|---|---|---|---|---|
| LeJEPA | ||||||
| -VAE | ||||||
| -TCVAE | ||||||
| DSReg |
Table 4 : DSReg improves latent MCC on every run of Gaussian 3DShapes while matching LeJEPA’s dense recovery. Scores are latent MCC, DCI, MIG, SAP, and informativeness (test of a boosted readout), twenty runs, on an external renderer. The LeJEPA row is the same frozen representation DSReg rotates, which is why the two share dense ; the -VAE/ -TCVAE rows are external references with the matched trunk, trained on i.i.d. images at an untuned ; their low dense reflects a linear readout only, with nonlinear informativeness staying high, so the gap measures axis alignment rather than lost information.
Figure 15 : Per-factor recovery on Gaussian 3DShapes. DSReg improves every factor and alone recovers scale, shape, and orientation, where the VAE baselines collapse almost entirely.
Figure 16 : The rotation turns entangled responses into one selective response per factor. One run of Gaussian 3DShapes. Each row sweeps one factor with all others fixed (rendered frames on the left); the right panels show every learned latent’s standardized response in LeJEPA and in DSReg coordinates, on a shared scale per row. The flat gray curves carry the no-mixing claim across all six factors.
Figure 17 : Matched correlation heatmaps. For the run of Figure 16 , columns are permuted by each method’s own best assignment. The same encoder yields a smeared matrix in the unrotated coordinates and a near-diagonal one after rotation, in agreement with the per-factor responses of the figure above.
Figure 18 : DSReg beats both VAE baselines under a partially attained premise. Quarter-orientation dSprites, twenty runs, dense as the rotation-invariant ceiling; each VAE at its best . One-sided Mann–Whitney tests are against -VAE and -TCVAE at their own best ; higher MCC is better.
| Method | Latent MCC | Dependency sparsity | |
|---|---|---|---|
| LeJEPA | |||
| Random orbit | – | ||
| PCA | – | ||
| Varimax | – | ||
| FastICA | – | ||
| DSReg |
Table 5 : Latent-only rotation baselines. The observation-side signal rather than the span is what does the work: all methods start from the same linearly identifiable representation, but PCA, Varimax, and FastICA see only estimated-latent samples and cannot break the rotational symmetry, while the dependency criterion can, because it also sees how the observed variables respond to the applied rotation.
| Task | Retrieval cost LeJEPA / DSReg | Sparse err. LeJEPA / DSReg |
|---|---|---|
| TwoRoom | ||
| PushT | ||
| FetchSlide |
Table 6 : Dense versus sparse-use sanity check. Everything the rotation gains, it gains in sparse access: dense readout and ordinary retrieval are unchanged, with dense at (LeJEPA / DSReg) for every task, while sparse-control error drops whenever only a few estimated latents may be used.
| Criterion | Gaussian 3DShapes | Quarter-orientation dSprites |
|---|---|---|
| Column-support sparse ICA | ||
| DSReg |
Table 7 : Sparse ICA on the same dependency signal recovers less. Latent MCC, mean std over ten fresh runs per renderer benchmark; both criteria consume the same estimated anchor Jacobians, orthogonal parameterization, and restarts. Both rows share every setting except the criterion.