Relational Geometry Forcing for World Action Models

Yuan Xu; Yixiang Chen; Qisen Ma; Jiabing Yang; Peiyan Li; Kai Wang; Jianhua Yang; Jianlou Si

cs.ROOct 21, 2026

Relational Geometry Forcing for World Action Models

Authors: Yuan Xu, Yixiang Chen, Qisen Ma, Jiabing Yang, Peiyan Li, Kai Wang, Jianhua Yang, Jianlou Si,

Organizations: CASIA · UCAS · Alibaba Group · FiveAges

Abstract

World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.

Figures & tables

Figure 1: Geometry transfer paradigms for WAMs. (a) Explicit geometric modeling adds a geometry branch that predicts depth or 4D outputs. (b) Pointwise geometric alignment regresses intermediate features toward dense geometric targets, which benefits the video expert but degrades action learning. (c) RGF aligns pairwise feature relations instead, benefiting both the video and action experts.

Figure 2: Cross-view correspondence and LIBERO performance. (a) Wrist-view similarity maps for a query patch in the agent view, where RGF peaks near the ground-truth correspondence. (b) Comparison of average success rate on the same WAM: extending pointwise alignment to action features degrades performance below the baseline.

Figure 3: Overview of Relational Geometry Forcing (RGF). The WAM predicts future multi-view frames and an action chunk through its video and action experts, while a frozen 3D foundation model encodes the same frames into per-timestep 3D features. RGF aligns pairwise relations among intermediate video and action features with those induced by the 3D foundation model through intra-frame, cross-view, and action-temporal objectives. All auxiliary components are removed during inference.

Vision-Language-Action Models
Method	Geo.	Spat.	Obj.	Goal	Long	Avg.
OpenVLA-OFT	$\times$	97.6	98.4	97.9	94.5	97.10
$\pi_{0.5}$	$\times$	98.8	98.2	98.0	92.4	96.90
Spatial Forcing	$\checkmark$	99.4	99.6	98.8	96.0	98.50
World Action Models
LingBot-VA	$\times$	98.5	99.6	97.2	98.5	98.50

Table 1: Success rates (%) on the four LIBERO suites. “Geo.” denotes the use of geometric inputs or supervision.

Method	P.T.	Geo.	Clean	Rand.	Avg.
Vision-Language-Action Models
$\pi_{0.5}$	$\checkmark$	$\times$	82.74	76.76	79.75
World Action Models
Motus	$\checkmark$	$\times$	88.66	87.02	87.84
LingBot-VA	$\checkmark$	$\times$	92.90	91.50	92.20
X-WAM	$\checkmark$	$\checkmark$	89.80	90.70	90.25

Table 2: Success rates (%) on RoboTwin 2.0 under clean and randomized evaluation settings. “P.T.” denotes embodied-policy pretraining, and “Geo.” denotes the use of geometric inputs or supervision.

Foundation Model	Alignment Type		Average SR (%)	$\Delta$
Foundation Model	Video	Action	Average SR (%)	$\Delta$
WAM Variant: Direct Action (Uncond)
No alignment			97.60	–
DINOv2	Pointwise	–	98.10	+0.50
DINOv2	Relational	Relational	98.35	+0.75
VGGT	Pointwise	–	97.90	+0.30

Table 3: Analysis of geometry transfer on LIBERO.

\Delta

denotes the gain over the corresponding variant baseline.

$\mathcal{L}_{\mathrm{intra}}$	$\mathcal{L}_{\mathrm{cross}}$	$\mathcal{L}_{\mathrm{temporal}}$	Avg.	$\Delta$
$\times$	$\times$	$\times$	97.60	–
$\checkmark$	$\times$	$\times$	97.65	+0.05
$\times$	$\checkmark$	$\times$	97.70	+0.10
$\times$	$\times$	$\checkmark$	97.45	$-$ 0.15
$\checkmark$	$\checkmark$	$\times$	97.90	+0.30
$\checkmark$	$\times$	$\checkmark$	97.35	$-$ 0.25

Table 4: Ablation of relational objectives and loss types on LIBERO.

\Delta

denotes the gain over no alignment.

Figure 4: Intra-frame, cross-view, and action-temporal relations learned by RGF. (a) Intra-frame similarity maps for the marked query patch, with the ground-truth 3D distance map as reference. (b) Cross-view similarity maps in the wrist view. The green plus marks the ground-truth correspondence, and the circle marks the highest-similarity patch. (c) Action-temporal similarity matrices for a rollout with nonuniform motion, compared with the scene-evolution reference. Across all three settings, RGF follows the geometric reference more closely than the unaligned WAM.

Figure 5: Real-world setup, tasks, and generalization settings. Left: a Franka Research 3 arm observed by three ZED 2i cameras. Middle: the four evaluation tasks. Right: Bowl-to-Rack generalization settings with an unseen object color, background distractors, and altered placement heights.

Method	Basic Tasks				Generalization			Avg. Succ. (%)
Method	Toy $\to$ Drawer	Bowl $\to$ Rack	Bowls $\to$ Plate	Lids $\to$ Blocks	Unseen Object	Distractors	Height	Avg. Succ. (%)
$\pi_{0.5}$	65	70	65	50	60	30	40	54.29
Fast-WAM	55	85	60	40	65	40	55	57.14
RGF-WAM (ours)	60	85	70	55	70	45	75	65.71

Table 5: Real-world success rates (%) over 20 rollouts per setting. Generalization settings are Bowl-to-Rack variants with an unseen object color, background distractors, or altered placement heights. Best results per column are in bold.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Video Expert	Action Expert
Transformer layers	30	30
Hidden dimension	3072	1024
FFN dimension	14336	4096
Attention heads	24	24

Appendix

Table A1: Shared architecture of the video and action experts.

Foundation Model	Video	Action	Spatial	Object	Goal	Long	Avg.	$\Delta$
WAM Paradigm: Direct Action (Uncond)
No alignment			98.2	100.0	97.0	95.2	97.60	–
DINOv2	Pointwise	–	99.2	100.0	97.8	95.4	98.10	+0.50
DINOv2	Relational	Relational	98.4	100.0	99.0	96.0	98.35	+0.75
VGGT	Pointwise	–	98.4	99.8	98.0	95.4	97.90	+0.30
VGGT	Pointwise	Pointwise	99.2	98.6	98.2	94.2	97.55	$-$ 0.05

Appendix

Table A2: Suite-level success rates (%) for geometry-transfer controls on LIBERO. Each suite uses 500 rollouts, following the training and evaluation protocol of the main paper.

\Delta

denotes the gain over the corresponding paradigm baseline.

$\mathcal{L}_{\mathrm{intra}}$	$\mathcal{L}_{\mathrm{cross}}$	$\mathcal{L}_{\mathrm{temporal}}$	Spatial	Object	Goal	Long	Avg.	$\Delta$
$\times$	$\times$	$\times$	98.2	100.0	97.0	95.2	97.60	–
$\checkmark$	$\times$	$\times$	98.0	99.8	98.6	94.2	97.65	+0.05
$\times$	$\checkmark$	$\times$	98.4	99.6	97.6	95.2	97.70	+0.10
$\times$	$\times$	$\checkmark$	98.8	99.0	98.2	93.8	97.45	$-$ 0.15
$\checkmark$	$\checkmark$	$\times$	97.8	100.0	97.8	96.0	97.90	+0.30
$\checkmark$	$\times$	$\checkmark$	97.2	99.6	98.2	94.4	97.35	$-$ 0.25

Appendix

Table A3: Suite-level ablation of the three relational objectives and loss types on LIBERO. Each suite uses 500 rollouts, and

\Delta

denotes the gain over no alignment.

Task	$\pi_{0.5}$		Motus		LingBot		Fast-WAM		X-WAM		GeoSem		WAM4D		RGF (Ours)
	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.
Adjust Bottle	100	99	89	93	90	94	100	100	100	99	100	100	100	99	100	99
Beat Block Hammer	96	93	95	88	96	98	99	97	98	96	98	98	99	97	100	97
Blocks Ranking RGB	92	85	99	97	99	98	100	100	99	95	100	98	100	97	100	99
Blocks Ranking Size	49	26	75	63	94	96	94	98	76	82	91	96	98	84	92	94
Click Alarmclock	98	89	100	100	99	100	100	100	98	99	100	100	100	100	100	100

Appendix

Table A4: Per-task success rates (%) on RoboTwin 2.0 under clean and randomized evaluation settings. The average row follows the aggregate values reported in the main paper.

Layer	Spatial	Object	Goal	Long	Avg.
No alignment	98.2	100.0	97.0	95.2	97.60
L8	98.6	99.8	97.8	95.6	97.95
L15	99.4	100.0	98.6	96.6	98.65
L22	99.0	100.0	98.4	95.4	98.20

Appendix

Table A5: Ablation of the supervised WAM layer on LIBERO. L15 is the default setting.

$\lambda_{\mathrm{RGF}}$	Spatial	Object	Goal	Long	Avg.
0 (off)	98.2	100.0	97.0	95.2	97.60
0.25	98.4	100.0	98.8	96.4	98.40
0.5	99.4	100.0	98.6	96.6	98.65
1.0	98.0	100.0	98.0	94.2	97.55

Appendix

Table A6: Ablation of the RGF loss weight on LIBERO.

\lambda_{\mathrm{RGF}}=0.5

is the default setting.

Figure A1: Complete real-world task executions. Each row shows eight uniformly sampled frames from one successful rollout. From top to bottom: Toy to Drawer , Bowl to Rack , Two Bowls to Plate , and Two Lids Cover Blocks .

World Models for Robotics

World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.

See all 380 papers

Learning 4D Geometric Priors for Inference-Efficient World Action Models
Jul 6, 2026Jianjun Zhang, Jian Zhu, Taiyi Su +4World Models for Robotics Action-Conditioned World Models
Tongji University · AIRC, Midea Group
MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation
Sep 29, 2026Wenbo Chen, Tianfu Li, Haoxuan Xu +9Multi-View Geometry World Models for Robotics
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · ETH Zurich +4
Geometric Action Model for Robot Policy Learning
Jun 15, 2026Jisang Han, Seonghu Jeon, Jaewoo Jung +7World Models for Robotics Language-Conditioned Robot Manipulation
KAIST AI · ETH Zurich · ETH AI Center

Jul 6, 2026J/K move · Enter open · S save

Learning 4D Geometric Priors for Inference-Efficient World Action Models

Jianjun Zhang, Jian Zhu, Taiyi Su, Chong Ma+3

Tongji University · AIRC, Midea Group

World Models for Robotics Action-Conditioned World Models Geometric Representation Learning Robotic Manipulation World Action Models

Open paper AskPDF arXiv

World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.

Abstract

World Models for Robotics

Explore similar work

Learning 4D Geometric Priors for Inference-Efficient World Action Models

MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation

Geometric Action Model for Robot Policy Learning