cs.LGNov 6, 2026

A Stochastic Latent Variable Generalization of Joint-Embedding Predictive Architectures

Authors: Nikos Giakoumoglou, Paschalis Giakoumoglou, Tania Stathaki

Organizations: Imperial College London · Information Technologies Institute, CERTH

Abstract

Joint-Embedding Predictive Architectures (JEPAs) predict the latent embedding of a target signal from a context signal, with an auxiliary variable z conditioning the predictor. Despite its prominent role in the original JEPA blueprint, the content of z has been carried by a fixed deterministic token in every prior instantiation, collapsing the genuinely multi-modal distribution over plausible completions to a point estimate. We present S-JEPA, which realises z as a proper stochastic latent variable for masked image prediction: a learned posterior infers z from context and target during training, while a learned context-conditioned prior approximates the distribution of plausible completions at inference, with no decoders and no reconstruction. In its deterministic limit the objective reduces to the I-JEPA loss in functional form, generalising that objective rather than strictly containing the I-JEPA model. Our central finding is that stochasticity adds capabilities, error-correlated predictive uncertainty and diverse completions, at no cost to representation quality: under matched architectures and schedules S-JEPA improves over the deterministic I-JEPA baseline and exceeds LeJEPA at matched architecture, while additionally producing spatially structured uncertainty maps and semantically diverse completions, neither of which the deterministic predictor produces.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Mar 20, 2026

Var-JEPA: A Variational Formulation of the Joint-Embedding Predictive Architecture - Bridging Predictive and Generative Self-Supervised Learning

The Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning, emphasizing prediction in representation space rather than reconstruction in observation space. We argue that the resulting separation from probabilistic generative modeling is largely rhetorical rather than structural: the canonical JEPA design (coupled encoders with a context-to-target predictor) mirrors the variational posteriors and learned conditional priors obtained when variational inference is applied to a particular class of coupled latent-variable models, and standard JEPA can be viewed as a deterministic specialization in which regularization is imposed via architectural and training heuristics rather than an explicit likelihood. Building on this view, we derive the Variational JEPA (Var-JEPA), which makes the latent generative structure explicit by optimizing a single Evidence Lower Bound (ELBO). This yields meaningful representations without ad-hoc anti-collapse regularizers and allows principled uncertainty quantification in the latent space. We instantiate the framework for tabular data (Var-T-JEPA) and achieve strong representation learning and downstream performance, improving over T-JEPA across real-world tabular benchmarks while remaining competitive with strong raw-feature baselines.
May 29, 2026

Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models

World models used for planning learn to predict how a scene evolves, yet they rarely hold a global view of how far a task has progressed. We argue that such a view should be part of the representation, and that it can be compact: describing a scene takes many dimensions, but where the scene stands on its way to the goal can be summarised by only a few. We propose SD-JEPA, a joint-embedding predictive world model whose latent is split into a small progression subspace and a large content subspace. A temporal triplet loss keeps consecutive frames of a trajectory close in the progression coordinates and separates different trajectories, while a distributional regulariser keeps the content coordinates from collapsing. On four control benchmarks, the model plans as well as the same architecture without the split. In return, its latent becomes easier to read. A few percent of its dimensions are enough to predict progress within an episode, and the angle they form follows the task, moves backward when the agent backtracks and changes fastest around events such as grasps and releases. Surprise also becomes anchored in the trajectory: when the scene changes mid-episode, the angle spikes and then relocalizes on a new part of the progression, anchoring surprise in the agent's own map of the task.
May 31, 2026

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse. LeJEPA addresses this by enforcing an isotropic Gaussian target on the embeddings via Sketched Isotropic Gaussian Regularization (SIGReg). This target is in tension with the manifold hypothesis, which expects embeddings to concentrate on a low-dimensional subset of the ambient space. We propose UR-JEPA, which targets a uniformly nn-rectifiable measure of local tangent dimension nn at small scales, realized through a Gaussian-kernel smoothed Carleson-type square function LCGLT\mathcal{L}^{\text{CGLT}}, with a complementary Jones ββ-number formulation. On Inet10, UR-JEPA(LCGLT\mathcal{L}^{\text{CGLT}}) attains 0.9141±0.00140.9141 \pm 0.0014 for a +0.83+0.83,pp gain over LeJEPA(LSIGReg\mathcal{L}^{\text{SIGReg}}) with ∼30%\sim 30\% lower seed standard deviation; on matched-recipe Galaxy10~SDSS, a single-seed ImageNet-100100 run, and a 33-seed EuroSAT remote-sensing run, the two methods lie in the same peak-accuracy band at convergence, with UR-JEPA retaining its lower-seed-variance signature. On EuroSAT the in-domain pair is competitive at 96.096.0 to 96.1%96.1\% with large remote-sensing foundation-model transfer at a 25×25\times smaller backbone. The distinction is geometric: direct visualization of the projector output distribution shows that on all four datasets UR--JEPA(LCGLT\mathcal{L}^{\text{CGLT}}) produces a global PCA spectrum with a 44 to 55 order-of-magnitude drop at index ∼20\sim 20 to 2525 out of D=32D = 32, while LeJEPA's spectrum is near-flat (top-to-bottom ratio at most 3.63.6). Per-dimension marginals are simultaneously near-Gaussian for both methods (mean Shapiro-Wilk W∈[0.992,0.996]W \in [0.992, 0.996]) as a Diaconis-Freedman consequence. At matched accuracy the two regularizers therefore yield structurally distinct projected representations.