cs.ROSep 24, 2026

Representation World Model: Learning States, Transition and Executable Plans in Representation

Authors: Yijun Yuan, Weicheng Zheng, Weibang Wang, Minghui Qin, Chang Sun, Junhao Huang, Kenan Li, Anmin Liu, +2 more

Organizations: IIIS, Tsinghua University

Abstract

We propose the Representation World Model (RWM), which learns states, transitions, and executable plans directly in representation space. Unlike existing world models that typically learn latent representations together with explicit dynamics models and perform planning through search, optimization, or policy-based prediction, RWM directly incorporates planning into the learned representation geometry. RWM learns the representation geometry by applying inverse-dynamics supervision locally along latent paths constructed from endpoint representations, requiring these paths to preserve task-relevant state and transition information. At inference, planning is performed by directly constructing a latent path between the current and goal representations, with inverse dynamics used to recover the corresponding actions, without recursive rollouts or action-space search. Experiments on continuous-control benchmarks demonstrate the effectiveness of RWM for direct planning, while results on robotic manipulation further show its potential to extend to more complex embodied control tasks. These results suggest that planning directly in representation space provides a promising alternative to conventional world-model planning.

Explore similar work

CardsList
  1. Sensorimotor World Models: Perception for Action via Inverse Dynamics

    Jun 18, 2026Petr Ivashkov, Randall Balestriero, Bernhard SchölkopfLatent World ModelsInverse Dynamics

  2. Latent Geometry Beyond Search: Amortizing Planning in World Models

    May 9, 2026Hoang Nguyen, Xiaohao Xu, Xiaonan HuangLatent World ModelsWorld Models

  3. Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning

    Sep 3, 2026Muyuan Liu, Yue Huang, Zheng Liang +1Inverse DynamicsRobot Planning