MoP-JEPA: Hard-Assigned Predictor Mixtures for Stochastic JEPA World Models

Zhi Song; Ximing Xing; Zhenchao Tang; hanbo Huang; Weilong Yan; Tianxu Lv; Minghao Yang; Zhongzheng Niu

cs.AIJul 6, 2026

MoP-JEPA: Hard-Assigned Predictor Mixtures for Stochastic JEPA World Models

Authors: Zhi Song, Ximing Xing, Zhenchao Tang, hanbo Huang, Weilong Yan, Tianxu Lv, Minghao Yang, Zhongzheng Niu, +3 more

Organizations: City University of Hong Kong, China · Tencent, China

Abstract

JEPA world models commonly predict the next latent state with one regressor. Under stochastic transitions, squared and cosine regression return the conditional mean and its normalized direction, respectively: a single compromise that may match no valid successor. MoP-JEPA instead uses $K$ hard-assigned heads and a context-only router to produce a finite candidate set in one pass. On held-out OGBench transitions, graph search with single-output predictors succeeds on $0.02$ -- $0.09$ of queries, whereas MoP-JEPA reaches $0.85$ . To distinguish useful successors from indiscriminate coverage, we also measure verified-route success (\emph{realroute}), which checks after graph construction whether the proposal contains a path of real transitions. MoP-JEPA leads this same-protocol metric on all three mazes; an MDN attains high raw coverage but predicts many nonexistent edges.

Explore similar work

Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
Jul 28, 2026Jiaxin Bai, Jiaxuan XiongJoint-Embedding Predictive Architectures Classical Planning
Orthogonal JEPA: Factorized Predictive States for Latent World Models
Aug 20, 2026Taoyong Cui, Pheng Ann Heng, Wanli OuyangJoint-Embedding Predictive Architectures
A Generalization Theory for JEPA-Based World Models
Jun 25, 2026Jingyi Cui, Qi Zhang, Hongwei Wen +1Joint-Embedding Predictive Architectures World Models

Jul 28, 2026 · cs.CLJ/K move · Enter open · S save

Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control

Jiaxin Bai, Jiaxuan Xiong

Hong Kong Baptist University Hong Kong, China

Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting in representation space rather than reconstructing pixels, making them a natural backbone for latent model predictive control from offline demonstration logs. JEPA-style training optimizes short-horizon latent prediction, whereas planning requires a multi-step ranking of imagined futures by goal progress. Prior JEPA planners often inherit that ranking from embedding geometry, typically latent Euclidean distance, which arises as a byproduct of representation learning rather than as a progress cost mined from the logs. We propose Temporal-Distance-JEPA, which retains the LeWM encoder--predictor backbone and mines a directed temporal cost from reward-free trajectories: same-trajectory step order supplies positive targets, cross-trajectory pairs act as heuristic negatives, and a rollout-consistency term matches the planner horizon. The mined supervision serves two roles: as the deployed planning cost when progress is topological, and as a representation signal that improves Euclidean planning when contact geometry dominates. Under locked evaluation, deploying the mined cost raises Two-Room success to 100.0% versus LeWM's 97.4%, while shared Euclidean planning on the same temporally trained checkpoint raises OGB-Cube by 14.2 points over LeWM and improves Push-T. Against LeWM and the concurrent RC-aux baseline under locked evaluation, Temporal-Distance-JEPA matches or exceeds both methods on every environment. Ablations show that the directed head, cross-trajectory negatives, and rollout consistency each contribute. Temporal-Distance-JEPA narrows the train--plan gap for JEPA world-model planners by discovering temporal progress structure in offline logs and co-designing cost form with plan-time deployment. Code is available at https://github.com/HKBU-KnowComp/Temporal-Distance-JEPA.

Joint-Embedding Predictive Architectures Classical Planning Model Predictive Control Representation Learning

Open paper Ask about it PDF

Abstract

Explore similar work

Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control

Orthogonal JEPA: Factorized Predictive States for Latent World Models

A Generalization Theory for JEPA-Based World Models