Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation

Junjin Xiao; Dongyang Li; Yandan Yang; Shuang Zeng; Tong Lin; Xinyuan Chang; Feng Xiong; Mu Xu

cs.ROMay 12, 2026

Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation

Authors: Junjin Xiao, Dongyang Li, Yandan Yang, Shuang Zeng, Tong Lin, Xinyuan Chang, Feng Xiong, Mu Xu, +4 more

Abstract

This paper tackles spatial perception and manipulation challenges in Vision-Language-Action (VLA) models. To address depth ambiguity from monocular input, we leverage a pre-trained multi-view diffusion model to synthesize latent novel views and propose a Geometry-Guided Gated Transformer (G3T) that aligns multi-view features under 3D geometric guidance while adaptively filtering occlusion noise. To improve action learning efficiency, we introduce Action Manifold Learning (AML), which directly predicts actions on the valid action manifold, bypassing inefficient regression of unstructured targets like noise or velocity. Experiments on LIBERO, RoboTwin 2.0, and real-robot tasks show our method achieves superior success rate and robustness over SOTA baselines. Project page: https://junjxiao.github.io/Multi-view-VLA.github.io/.

Explore similar work

GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning
Aug 7, 2026Yanping Zhao, Hang Yu, Yiwei Wang +8Vision-Language-Action Models Latent Actions
GeomVLA: Unifying Scene, Motion, and Action in 3D
Sep 12, 2026Ziyin Xiong, Nikolaos Gkanatsios, Moritz Reuss +13D Motion Perception-Action Loop
GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation
Jun 7, 2026Yuan Zhang, Shiqi Zhang, Yedong Shen +11Vision-Language-Action Models Generalizable Robotic Manipulations

Aug 7, 2026 · cs.ROJ/K move · Enter open · S save

GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning

Yanping Zhao, Hang Yu, Yiwei Wang, Chen Ye+7

Tongji University

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but often degrade under visual and environmental shifts. Latent world modeling offers a promising approach to improving robustness, yet existing methods commonly encode camera views independently and predict holistic scene dynamics without explicitly modeling their geometric relationships. We propose GWM-VLA, a geometry-aware latent world modeling framework for VLA learning. GWM-VLA combines geometry-aware multi-view state encoding, global context-conditioned target-view prediction, and shared latent-action representations grounded by robot-action supervision. Specifically, VGGT-

Ω

jointly aggregates multi-view observations at each timestep to construct geometry-aware multi-view states. The latent world model predicts the next-step patch tokens of a selected target view using patch and register tokens obtained after multi-view aggregation, thereby retaining multi-view geometric information without predicting the complete multi-view state. We use the wrist view as the target in our experiments, placing greater emphasis on end-effector motion and local gripper-object interactions. Finally, the shared latent-action representations condition both the latent world model and the flow-matching action head, allowing latent-prediction supervision and ground-truth robot-action supervision to jointly shape the same latent-action representations. Experiments across both simulation and real-world environments demonstrate the effectiveness and robustness of GWM-VLA.

Vision-Language-Action Models Latent Actions Latent World Models

Open paper Ask about it PDF

Abstract

Explore similar work

GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning

GeomVLA: Unifying Scene, Motion, and Action in 3D

GEAR-VLA: Learning Geometry-Aware Action Representations for Generalizable Robotic Manipulation