cs.RONov 18, 2026

JEPA Dots: Dual-Latent World Model for Language-Guided Robot Control

Authors: Xianhao Pan, Chi Li, Hanchao Li, Yunxin Liu, Jiankai Sun, Luhui Hu

Organizations: Z Robotics · Tsinghua University · Stanford University

Abstract

Predictive world models for language-guided robot control face a fundamental conditioning challenge: instructions should influence action selection without altering the predicted consequences of fixed actions under identical observations. To address this challenge, we introduce JEPA Dots (Dynamics-Oriented Task Separation), a compact dual-latent framework that jointly learns visual dynamics and language-guided control through separate dynamics and task representations. Language-conditioned task features guide a flow-matching policy, while masked attention restricts world prediction to instruction-independent dynamics history and supplied actions. In closed-loop evaluation on 40 trained LIBERO tasks, JEPA Dots achieves 84.9% overall success with approximately 39.1M parameters, compared with the published 78.7% of NanoVLA-S with 161.0M parameters under a different evaluation protocol. In the reference experiment, JEPA Dots also improves overall success from 76.0% for our task-ID-conditioned adaptation baseline to 84.9%, with LIBERO-Goal success increasing from 67.1% to 88.5%. Instruction-switching experiments further confirm that task representations and policy outputs respond to language changes, while dynamics representations and world predictions remain invariant under fixed observations and supplied actions.

Figures & tables

Explore similar work

CardsList
  1. La-JEPA: Compact Robot Policies with Latent World Modeling

    Nov 5, 2026Mateo Clémente, George Z. Xue, Amin Abyaneh +5Robot Policy LearningWorld Action Models

  2. D-JEPA: A Decision-Aligned Latent World Model

    Sep 21, 2026Shuaijun Liu, Chengyu Wu, Qifu Wen +5World ModelsWorld Model-Based Planning

  3. CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization

    Sep 30, 2026Morgan Byrd, Robert Wright, Sehoon HaRepresentation LearningRepresentation Collapse