cs.AINov 7, 2026

WARP-JEPA: World-Model-in-the-Loop Amortized Rollout Planning for Frozen Hierarchical Joint-Embedding Predictive Architectures

Authors: Junjie Li, Zhaojiacheng Zhou, Kaixiang Wang, Zihan Wang, QingSong Zhao, Jiong Lou, Jie Li

Organizations: Shanghai Jiao Tong University

Abstract

Hierarchical Joint-Embedding Predictive Architectures (H-JEPA) enable long-horizon visual control but rely on costly test-time gradient descent for planning. Our analysis reveals that iterative optimization consumes up to 97.7% of episode runtime and that low predicted rollout costs do not necessarily translate into successful control. We further identify a planning-horizon asymmetry between long-horizon top-level composition and short two-step subordinate planning. To address these challenges, we propose WARP-JEPA, a hierarchical amortized planning framework that replaces online gradient optimization with learned multi-candidate trajectory proposers while keeping the world models frozen. WARP-JEPA combines a lightweight world-model-in-the-loop causal Transformer at the top level with parallel residual MLPs at subordinate levels. The proposers are trained using model-based rollout costs and best-of-K behavior cloning, enabling efficient candidate selection without test-time gradient optimization. Across 30 frozen H-JEPA models on four benchmarks, WARP-JEPA achieves 9.6-18.8x faster episode inference on long-horizon tasks (2.9-3.0x on Cube) and improves task success by up to 33.3 percentage points. These results demonstrate that amortized planning can substantially improve both inference efficiency and control performance without retraining the world models. Code is available at https://github.com/hideinsewer/WARP-JEPA.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning

    Oct 5, 2026Wancong Zhang, Basile Terver, Michael Rabbat +2World Models for RoboticsWorld Models

  2. LeWAM: End-to-End World and Action Modeling with JEPAs

    Oct 8, 2026Minghao Fu, Tavis Siebert, Eryk Halicki +1World Model LearningWorld Model-Based Planning

  3. Hierarchical Planning with Latent World Models

    Apr 3, 2026Wancong Zhang, Basile Terver, Artem Zholus +7Hierarchical PlanningHierarchical Robot Control