cs.CVNov 8, 2026

ForeAct: Geometry-Guided Action Learning from World Representations

Authors: Wei Lu, Jiaxi Zhang, Yunheng Wang, Taowen Wang, Yuetong Fang, Zhenhui Zhang, Shuning Zhang, Zizhao Yuan, 

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · JD Explore Academy

Abstract

Robotic manipulation requires representations that capture both scene geometry and its evolution during interaction. Existing approaches have largely treated these two aspects separately. World action models capture such dynamics by coupling action prediction with future video generation, but their reliance on iterative diffusion or rectified-flow inference leads to high computational cost, while their representations lack explicit spatial structure. Explicitly modeling scene geometry typically requires auxiliary modules that increase both parameter count and computational cost. Conversely, geometry-centric policies primarily recover observed spatial structure and often lack explicit supervision for anticipating future geometric changes during interaction. To address these limitations, we introduce ForeAct, which repurposes a pretrained video world model as a single-pass representation backbone for low-latency control. ForeAct directly processes the clean latent of the current observation in one forward pass, eliminating future-frame synthesis and iterative denoising while retaining the backbone's pretrained spatiotemporal priors. Rather than introducing a separate geometry backbone, ForeAct uses learnable tokens supervised by next-frame relative depth to shape the shared video representation through bidirectional self-attention. This compact interface turns the spatial knowledge already encoded in the backbone into a future-aware representation for action generation, with little parameter overhead. The lightweight geometry readout is used only during training and removed at deployment, enabling low-latency, geometry-aware control without depth input or an external geometry backbone. Without additional large-scale pretraining, ForeAct achieves 96.6% average success on LIBERO, 63.5% on SimplerEnv-Bridge, and a 74.4% success rate across 90 real-world rollouts.

Figures & tables

Explore similar work

CardsList
  1. From Predictive Geometry to Action: Learning Complementary Predictive Representations for Embodied Manipulation

    Oct 20, 2026Xiaoxu Xu, Hao Li, Hongtao Zhang +6Robotic ManipulationPredictive Representation Learning

  2. ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies

    Oct 3, 2026Zhe Tao, Feiran Wang, Gaowen Liu +2World Models for RoboticsVision-Language-Action Models