cs.CVNov 6, 2026

LEF-JEPA: Denoising-Trained Latent Energy Fields for Spatial Representation Learning

Authors: Zhiyuan Liu, Zhihao Xu, Sicheng Liu, Huanhuan Huang

Organizations: Qinous Technology Co., Ltd · Tsinghua University

Abstract

A joint-embedding predictive architecture (JEPA) that applies prediction supervision only at masked positions struggles to clearly capture object shapes and boundaries in its latent representations. We further observe that some existing methods improve spatial structure while also retaining rich variations in texture and color. However, preserving more low-level visual information does not necessarily make representations better suited to prediction and action. This raises a key question: how can we strengthen the spatial structure in latent representations that supports prediction and action without over-encoding these details? We introduce LEF-JEPA, which learns spatial structure through target-guided latent recovery and transfers this supervision to the visual encoder through a relative-energy constraint; the energy module is used only during pretraining and adds no inference overhead. To reduce the cost of energy optimization during pretraining, we further construct a conditional Ising Hamiltonian that allows low-energy-state search to be performed by specialized Ising solvers. On physical hardware, we validate the feasibility of achieving lower forward latency with fewer local updates. Under a unified CIFAR-10 pretraining and frozen-encoder evaluation protocol, both realizations outperform V-JEPA2.1 in image classification, foreground segmentation, and visual planning.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MotionJEPA: Preventing Temporal Feature Collapse by Capturing Visual Changes in Latent Space

    Sep 20, 2026Markus Karmann, Shile Li, Christian Internò +10Joint-Embedding Predictive ArchitectureLatent World Models

  2. Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models

    May 29, 2026Lucas Thil, Jesse Read, Rim Kaddah +1Disentangled Representation LearningRepresentation Learning

  3. LocaLeJEPA: Maximum Entropy Representations under Geometric Priors

    Oct 20, 2026Kartik Tandon, Alessio Borgi, Pavel Snopov +8Representation LearningJoint-Embedding Predictive Architecture