World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.
Figures & tables
Figure 1: Geometry transfer paradigms for WAMs. (a) Explicit geometric modeling adds a geometry branch that predicts depth or 4D outputs. (b) Pointwise geometric alignment regresses intermediate features toward dense geometric targets, which benefits the video expert but degrades action learning. (c) RGF aligns pairwise feature relations instead, benefiting both the video and action experts.
Figure 2: Cross-view correspondence and LIBERO performance. (a) Wrist-view similarity maps for a query patch in the agent view, where RGF peaks near the ground-truth correspondence. (b) Comparison of average success rate on the same WAM: extending pointwise alignment to action features degrades performance below the baseline.
Figure 3: Overview of Relational Geometry Forcing (RGF). The WAM predicts future multi-view frames and an action chunk through its video and action experts, while a frozen 3D foundation model encodes the same frames into per-timestep 3D features. RGF aligns pairwise relations among intermediate video and action features with those induced by the 3D foundation model through intra-frame, cross-view, and action-temporal objectives. All auxiliary components are removed during inference.
Method
Geo.
Spat.
Obj.
Goal
Long
Avg.
Vision-Language-Action Models
OpenVLA-OFT
×
97.6
98.4
97.9
94.5
97.10
π0.5
×
98.8
98.2
98.0
92.4
96.90
Spatial Forcing
✓
99.4
99.6
98.8
96.0
98.50
World Action Models
LingBot-VA
×
98.5
99.6
97.2
98.5
98.50
Table 1: Success rates (%) on the four LIBERO suites. “Geo.” denotes the use of geometric inputs or supervision.
Method
P.T.
Geo.
Clean
Rand.
Avg.
Vision-Language-Action Models
π0.5
✓
×
82.74
76.76
79.75
World Action Models
Motus
✓
×
88.66
87.02
87.84
LingBot-VA
✓
×
92.90
91.50
92.20
X-WAM
✓
✓
89.80
90.70
90.25
Table 2: Success rates (%) on RoboTwin 2.0 under clean and randomized evaluation settings. “P.T.” denotes embodied-policy pretraining, and “Geo.” denotes the use of geometric inputs or supervision.
Foundation Model
Alignment Type
Average SR (%)
Δ
Video
Action
WAM Variant: Direct Action (Uncond)
No alignment
97.60
–
DINOv2
Pointwise
–
98.10
+0.50
DINOv2
Relational
Relational
98.35
+0.75
VGGT
Pointwise
–
97.90
+0.30
Table 3: Analysis of geometry transfer on LIBERO. Δ denotes the gain over the corresponding variant baseline.
Lintra
Lcross
Ltemporal
Avg.
Δ
×
×
×
97.60
–
✓
×
×
97.65
+0.05
×
✓
×
97.70
+0.10
×
×
✓
97.45
− 0.15
✓
✓
×
97.90
+0.30
✓
×
✓
97.35
− 0.25
Table 4: Ablation of relational objectives and loss types on LIBERO. Δ denotes the gain over no alignment.
Figure 4: Intra-frame, cross-view, and action-temporal relations learned by RGF. (a) Intra-frame similarity maps for the marked query patch, with the ground-truth 3D distance map as reference. (b) Cross-view similarity maps in the wrist view. The green plus marks the ground-truth correspondence, and the circle marks the highest-similarity patch. (c) Action-temporal similarity matrices for a rollout with nonuniform motion, compared with the scene-evolution reference. Across all three settings, RGF follows the geometric reference more closely than the unaligned WAM.
Figure 5: Real-world setup, tasks, and generalization settings. Left: a Franka Research 3 arm observed by three ZED 2i cameras. Middle: the four evaluation tasks. Right: Bowl-to-Rack generalization settings with an unseen object color, background distractors, and altered placement heights.
Method
Basic Tasks
Generalization
Avg. Succ. (%)
Toy → Drawer
Bowl → Rack
Bowls → Plate
Lids → Blocks
Unseen Object
Distractors
Height
π0.5
65
70
65
50
60
30
40
54.29
Fast-WAM
55
85
60
40
65
40
55
57.14
RGF-WAM (ours)
60
85
70
55
70
45
75
65.71
Table 5: Real-world success rates (%) over 20 rollouts per setting. Generalization settings are Bowl-to-Rack variants with an unseen object color, background distractors, or altered placement heights. Best results per column are in bold.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
Video Expert
Action Expert
Transformer layers
30
30
Hidden dimension
3072
1024
FFN dimension
14336
4096
Attention heads
24
24
Appendix
Table A1: Shared architecture of the video and action experts.
Foundation Model
Video
Action
Spatial
Object
Goal
Long
Avg.
Δ
WAM Paradigm: Direct Action (Uncond)
No alignment
98.2
100.0
97.0
95.2
97.60
–
DINOv2
Pointwise
–
99.2
100.0
97.8
95.4
98.10
+0.50
DINOv2
Relational
Relational
98.4
100.0
99.0
96.0
98.35
+0.75
VGGT
Pointwise
–
98.4
99.8
98.0
95.4
97.90
+0.30
VGGT
Pointwise
Pointwise
99.2
98.6
98.2
94.2
97.55
− 0.05
Appendix
Table A2: Suite-level success rates (%) for geometry-transfer controls on LIBERO. Each suite uses 500 rollouts, following the training and evaluation protocol of the main paper. Δ denotes the gain over the corresponding paradigm baseline.
Lintra
Lcross
Ltemporal
Spatial
Object
Goal
Long
Avg.
Δ
×
×
×
98.2
100.0
97.0
95.2
97.60
–
✓
×
×
98.0
99.8
98.6
94.2
97.65
+0.05
×
✓
×
98.4
99.6
97.6
95.2
97.70
+0.10
×
×
✓
98.8
99.0
98.2
93.8
97.45
− 0.15
✓
✓
×
97.8
100.0
97.8
96.0
97.90
+0.30
✓
×
✓
97.2
99.6
98.2
94.4
97.35
− 0.25
Appendix
Table A3: Suite-level ablation of the three relational objectives and loss types on LIBERO. Each suite uses 500 rollouts, and Δ denotes the gain over no alignment.
Task
π0.5
Motus
LingBot
Fast-WAM
X-WAM
GeoSem
WAM4D
RGF (Ours)
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Adjust Bottle
100
99
89
93
90
94
100
100
100
99
100
100
100
99
100
99
Beat Block Hammer
96
93
95
88
96
98
99
97
98
96
98
98
99
97
100
97
Blocks Ranking RGB
92
85
99
97
99
98
100
100
99
95
100
98
100
97
100
99
Blocks Ranking Size
49
26
75
63
94
96
94
98
76
82
91
96
98
84
92
94
Click Alarmclock
98
89
100
100
99
100
100
100
98
99
100
100
100
100
100
100
Appendix
Table A4: Per-task success rates (%) on RoboTwin 2.0 under clean and randomized evaluation settings. The average row follows the aggregate values reported in the main paper.
Layer
Spatial
Object
Goal
Long
Avg.
No alignment
98.2
100.0
97.0
95.2
97.60
L8
98.6
99.8
97.8
95.6
97.95
L15
99.4
100.0
98.6
96.6
98.65
L22
99.0
100.0
98.4
95.4
98.20
Appendix
Table A5: Ablation of the supervised WAM layer on LIBERO. L15 is the default setting.
λRGF
Spatial
Object
Goal
Long
Avg.
0 (off)
98.2
100.0
97.0
95.2
97.60
0.25
98.4
100.0
98.8
96.4
98.40
0.5
99.4
100.0
98.6
96.6
98.65
1.0
98.0
100.0
98.0
94.2
97.55
Appendix
Table A6: Ablation of the RGF loss weight on LIBERO. λRGF=0.5 is the default setting.
Figure A1: Complete real-world task executions. Each row shows eight uniformly sampled frames from one successful rollout. From top to bottom: Toy to Drawer , Bowl to Rack , Two Bowls to Plate , and Two Lids Cover Blocks .
World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.
World-Action Models (WAMs) couple visual dynamics with action prediction, bringing the rich priors of pretrained video models to robotic manipulation. However, their multi-view interfaces typically tile images or concatenate tokens, leaving the geometric relationships among synchronized cameras implicit. This makes it harder to connect global scene context with the local geometry required for interaction. We introduce the Multi-View Geometry-Aware World-Action Model (MVG-WAM), which organizes these observations as related projections of one physical world rather than separate images on a canvas. Our model combines an epipolar-constrained global state with view-indexed geometric states jointly inferred from synchronized observations. Camera-aware routing supplies each video region with its corresponding geometric context and the shared global state, explicitly structuring the representation used for action prediction. We further ground the geometry-aware representation in metric scale through multi-horizon future-depth supervision, without requiring depth decoding during action rollout. MVG-WAM achieves average success rates of 99.1% on LIBERO and 92.07% on RoboTwin 2.0, demonstrating competitive performance across both benchmarks. Real-world experiments on Cobot Magic further demonstrate a 91.3% success rate across 150 trials spanning three manipulation tasks.
Wenbo Chen, Tianfu Li, Haoxuan Xu +9
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · ETH Zurich +4
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.