cs.LGJul 15, 2026

The SIGReg Objective as Variational Free Energy: A Theoretical Active-Inference Account of JEPA World Models

Authors: Fabio Arnez, Alexandra Gomez-Villa

Abstract

Joint-Embedding Predictive Architectures (JEPAs) are the dominant design for latent world models, yet they are usually justified by empirical performance rather than a normative principle. We show that the choice of anti-collapse regulariser determines whether a JEPA's training objective, a prediction loss plus a weighted embedding regulariser, is a valid Active Inference (AIF) variational free energy. We organise four non-contrastive regularisers (VICReg, LogDet, PairDist, and SIGReg) into an entropy-estimator hierarchy indexed by a prior-miscalibration gap, and show that the gap's sign, whether the estimator bounds the latent entropy from above or below, decides whether the AIF surprise bound survives: VICReg and LogDet are unsafe upper bounds, PairDist a safe lower bound, and SIGReg eliminates the gap. We then prove a correspondence theorem: under the standard constant-noise encoder model and successful SIGReg enforcement (isotropic-Gaussian embeddings), the gap vanishes, the objective becomes an exact information bottleneck, the surprise bound is preserved, and the latent goal cost becomes an exact proxy for AIF pragmatic value, whereas VICReg leaves an irreducible second-order anisotropy term. We extend the correspondence to multi-step expected free energy, ensemble epistemic value, and a learned-policy regime, and we identify the one AIF term no current JEPA world model computes: the state-epistemic value, a future-state coverage signal. The predictions differ in kind, not degree, and are stated here as theoretical consequences left for empirical test in separate work; full proofs are in Appendix A, and the algebraic core of every result is machine-verified in Lean 4 (Appendix D).

Explore similar work

CardsList
Jun 25, 2026cs.LG

A Generalization Theory for JEPA-Based World Models

Joint Embedding Predictive Architectures (JEPAs) have recently emerged as a promising paradigm for world modeling by learning predictive dynamics in a latent space rather than generating future observations at the input level. Despite their empirical success, the theoretical understanding of JEPA-based world models remains limited. In this paper, we develop the first generalization theory for JEPA-based world models. We formulate JEPA pretraining as a conditional spectral graph learning problem and show that the JEPA objective is equivalent to a low-rank factorization of an action-conditioned co-occurrence matrix. Building on this characterization, we establish a connection between JEPA pretraining error and downstream planning regret, leading to a finite-sample generalization bound for JEPA-based world models. Our analysis reveals an inherent trade-off between approximation and sample errors with respect to the latent dimension, providing theoretical insights into the advantages and limitations of latent predictive models compared with input-level predictive approaches.
Jingyi Cui, Qi Zhang, Hongwei Wen +1
World ModelsModel PretrainingCo-Occurrence
AskPDF
May 29, 2026cs.LG

Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models

Joint-Embedding Predictive Architectures (JEPAs) learn compact latent world models by predicting future embeddings, but no single coordinate of the latent is designated to encode task progression. We carve the JEPA latent into two orthogonal subspaces with disjoint roles: a low-dimensional progression subspace shaped by a cosine-margin triplet loss, and a high-dimensional content subspace regularised by the existing SIGReg objective of LeWM. We prove that the two anti-collapse forces act on disjoint coordinates, so they compose additively rather than competing on the same dimensions. Our method, SD-JEPA improves over the LeWM baseline on the majority of its control benchmarks at matched compute, and outperforms the strongest non-LeWM JEPA baseline on Push-T; a subspace-ablation falsifier confirms the split is the load-bearing ingredient. Beyond planning, the resulting 1-D angular progression coordinate functions as a scene-aware compass on the latent. It advances with task progress, regresses when the agent backtracks, and under controlled perturbations both spikes and relocalises to a semantically appropriate new task-phase sector, separating the moment of surprise from its meaning in a way that prediction-error scalars cannot.
Lucas Thil, Jesse Read, Rim Kaddah +1
Video Joint Embedding Predictive ArchitectureLatent PredictionJoint-Embedding Predictive ArchitecturesLatent World Models
AskPDF
May 31, 2026cs.LG

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse. LeJEPA addresses this by enforcing an isotropic Gaussian target on the embeddings via Sketched Isotropic Gaussian Regularization (SIGReg). This target is in tension with the manifold hypothesis, which expects embeddings to concentrate on a low-dimensional subset of the ambient space. We propose \emph{UR-JEPA}, which targets a uniformly nn-rectifiable measure of local tangent dimension nn at small scales, realized through a Gaussian-kernel smoothed Carleson-type square function LCGLT\mathcal{L}^{\text{CGLT}}, with a complementary Jones ββ-number formulation. On Inet10, UR-JEPA(LCGLT\mathcal{L}^{\text{CGLT}}) attains 0.9141±0.00140.9141 \pm 0.0014 for a +0.83+0.83,pp gain over LeJEPA(LSIGReg\mathcal{L}^{\text{SIGReg}}) with ∼30%\sim 30\% lower seed standard deviation; on matched-recipe Galaxy10~SDSS, a single-seed ImageNet-100100 run, and a 33-seed EuroSAT remote-sensing run, the two methods lie in the same peak-accuracy band at convergence, with UR-JEPA retaining its lower-seed-variance signature. On EuroSAT the in-domain pair is competitive at 96.096.0 to 96.1%96.1\% with large remote-sensing foundation-model transfer at a 25×25\times smaller backbone. The distinction is geometric: direct visualization of the projector output distribution shows that on all four datasets UR--JEPA(LCGLT\mathcal{L}^{\text{CGLT}}) produces a global PCA spectrum with a 44 to 55 order-of-magnitude drop at index ∼20\sim 20 to 2525 out of D=32D = 32, while LeJEPA's spectrum is near-flat (top-to-bottom ratio at most 3.63.6). Per-dimension marginals are simultaneously near-Gaussian for both methods (mean Shapiro-Wilk W∈[0.992,0.996]W \in [0.992, 0.996]) as a Diaconis-Freedman consequence. At matched accuracy the two regularizers therefore yield structurally distinct projected representations.
Triet M. Le
Joint-Embedding Predictive ArchitecturesSphere-JepaRegularization MethodsHyperspherical Embedding GeometryPrincipal Component AnalysisRegularity+2
AskPDF