World-action models (WAMs) offer a promising framework for embodied intelligence, yet mainstream approaches remain centered on video foundation models or vision-language models. Even when geometric features extracted by 3D models such as VGGT are incorporated, they are typically passed into a video model or VLM that serves as the primary backbone, leaving 3D geometry as an auxiliary input or prediction target rather than the native foundation for world modeling. We present E0.5, a geometry-native world-action model built directly on 3D foundation models. E0.5 introduces two key innovations. First, it adopts a 3D foundation model as the primary backbone, making 3D geometry the native representation for both world modeling and action prediction. Second, it explicitly predicts future 3D states together with actions, enabling the model to capture geometric world dynamics beyond action-only modeling or future video prediction. Experiments on RoboTwin 2.0 first demonstrate the effectiveness of E0.5 without embodied-data pretraining, achieving success rates of 89.90% and 50.09% under clean and randomized settings, respectively. Building on these results, we further pretrain E0.5 on more than 160k robot trajectories from AgiBotWorld-Beta, jointly learning robot actions and future geometry within the pretrained 3D foundation. This embodied pretraining improves mean real-world manipulation success rates by approximately 15 and 7 percentage points on AgileX Piper and Zerith H1, respectively. Together, these results demonstrate the potential of geometry-native foundations to acquire robot interaction knowledge and transfer it to physical manipulation. E0.5 embodies this geometry-native approach as a member of the VWA family of world-action models.
Figures & tables
Figure 1: Positioning E0.5. Representative modeling choices differ in their primary foundation and prediction target. Video and VLM-centered models can incorporate geometric features without making a geometric model their primary backbone; the future-video branch applies to video-predictive WAMs. E0.5 jointly learns actions and future geometry on fully trainable 3D backbones. Its geometric target is the frame aligned with the final step of the action chunk.
Figure 2: E0.5 architecture. (a) Current images and language instruction are processed by a language-conditioned 3D foundation backbone, whose geometric representations condition flow-matching action generation. (b) Native geometric prediction heads predict future depth, point maps, and camera parameters at the endpoint of the action chunk. (c) Layer-wise geometry–action attention enables action tokens to attend to geometric representations. All policy modules are jointly optimized, while future geometry serves exclusively as training supervision and is not required during inference.
Method
Policy core params. (B) ↓
Embodied pretrain.
Clean ↑
Randomized ↑
Average ↑
VLA
StarVLA ( Community, 2026 )
–
✗
46.52
3.16
24.84
GalaxeaVLA ( Jiang et al., 2026 )
–
∼ 2k h
62.72
12.72
37.72
XR-0 ( Cai et al., 2026b )
4.70
∼ 1.9k h a
62.86
18.22
40.54
EventVLA ( Yang et al., 2026a )
–
✗
65.58
15.74
40.66
ABot-M0 ( Yang et al., 2026c )
–
∼ 9.5k h
57.40
30.36
43.88
Table 1: RoboTwin 2.0 success rate (%) under the clean-to-random protocol. RoboTwin post-training uses clean demonstrations only; evaluation uses clean and randomized scenes. Pretraining scales follow the reported model or base-model recipes.
Figure 3: Model Efficiency and Generalization on RoboTwin. Models are post-trained on clean demonstrations and evaluated in (a) clean and (b) randomized scenes. Translucent circles centered on the model markers encode embodied pretraining data, with circle area proportional to duration and a shared area scale across panels; models without embodied pretraining have no background circle.
Figure 4: Future geometry reconstruction for stack_blocks_two . Rows show the high, left-wrist, and right-wrist cameras. Columns show current RGB, future reference RGB, future GT depth, predicted future depth, and absolute relative depth error. Future RGB is used only as a reference and is not a model input. GT and predicted depth share a color scale; error visualization is clipped at 0.5, without affecting metric computation.
Metric
High
Left wrist
Right wrist
Overall
Depth MAE ↓
0.01259
0.00686
0.00831
0.00925
Depth RMSE ↓
0.05263
0.02021
0.02702
0.03329
Depth AbsRel ↓
0.01674
0.02606
0.02665
0.02315
δ1 (%) ↑
98.08
97.80
97.98
97.95
δ2 (%) ↑
99.14
99.18
99.12
99.15
δ3 (%) ↑
99.68
99.61
99.55
99.61
Table 2: Future geometry reconstruction on the RoboTwin training set diagnostic : 50 tasks, 200 samples, and 600 views, with a prediction horizon of 32 frames (2.13 s). Inputs are the current three-view RGB and the current instruction. Results are averaged over views within each sample, samples within each task, and then tasks; per-view columns omit the first averaging step. No post-hoc alignment or scale fitting is applied.
Figure 5: Real-world tasks on AgileX Piper (top) and Zerith H1 (bottom), showing initial configurations, execution, and successful terminal states for mouse placement, trash disposal, and charger packing.
Setting
Method
Pretrain
Mouse
Charger
Trash
Mean
SR
CS
SR
CS
SR
CS
SR
CS
AgileX Piper
ID
π0.5
50.0
0.500
50.0
0.625
40.0
0.550
46.7
0.558
E0.5 (Ours)
✗
90.0
0.900
70.0
0.775
100.0
1.000
86.7
0.892
π0.5
100.0
1.000
80.0
0.850
60.0
0.850
80.0
0.900
E0.5 (Ours)
✓
100.0
1.000
90.0
0.925
100.0
1.000
96.7
0.975
Table 3: Real-world manipulation performance under in-distribution (ID) and out-of-distribution (OOD) settings. We report success rate (SR, %) and completion score (CS) over each task and setting. Pretrain indicates large-scale robot-data pretraining (For π0.5 ( Intelligence et al., 2025 ) , it’s the official pre-training data, while ours refers to pre-training conducted on AgiBot-beta data). Bold denotes the best value for each metric within each matched baseline–E0.5 pair.
Figure 6: Real-world manipulation performance on AgileX Piper and Zerith H1. Mean success rate (SR, %) and completion score (CS) are reported under in-distribution (ID) and out-of-distribution (OOD) settings, averaged across three manipulation tasks. We compare π0.5 and E0.5 with (w/) and without (w/o) large-scale robot-data pretraining. For π0.5 , pretraining uses the official pretrained checkpoint, while E0.5 is pretrained on AgiBotWorld-Beta.
Adapter depth
Total action layers
Clean
Randomized
0
24
69.14
14.34
3
30
73.22
19.84
6
36
69.44
14.66
Table 4: Ablation on action adapter depth. All variants retain 24 layers that interact with VGGT-Omega and are trained without geometric reconstruction supervision. Adapter depth denotes the number of action-only layers at each end. We report mean success rates (%) across 50 tasks.
Table 11
Reconstruction strategy
Clean (%)
Δ (pp)
Random (%)
Δ (pp)
No reconstruction
73.22
—
19.84
—
Auxiliary DepthDiT
72.60
−0.62
17.20
−2.64
Native geometry heads
82.72
+9.50
29.20
+9.36
Table 7: Comparison of future geometry reconstruction strategies on RoboTwin. Success rates are averaged across tasks. Δ denotes the change relative to the baseline without reconstruction, measured in percentage points (pp).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Qualitative visualization of the Make a salad task in AgiBotWorld-Beta. Columns correspond to the head, left-wrist, and right-wrist cameras. From top to bottom, rows show the original RGB images, available sensor depth, DA3-predicted depth, and RGB-colored point clouds back-projected from the predictions. Wrist sensor depth is unavailable. Sensor depth is displayed in raw codes, while DA3 depth is displayed in estimated metres.
Figure 8: Qualitative visualization of the Brew coffee with a capsule machine task in AgiBotWorld-Beta. Columns correspond to the head, left-wrist, and right-wrist cameras. Rows show the original RGB images, available sensor depth, DA3-predicted depth, and the corresponding RGB-colored point clouds. Wrist sensor depth is unavailable. Sensor depth and DA3 depth use raw-code and estimated-metre scales, respectively.
Figure 9: Qualitative visualization of the Water Pouring in Restaurant task in AgiBotWorld-Beta. From left to right, the views are captured by the head, left-wrist, and right-wrist cameras. From top to bottom, the rows present RGB images, available sensor depth, DA3-predicted depth, and RGB-colored point clouds reconstructed from the predictions. Wrist sensor depth is unavailable; sensor and predicted depth are displayed in raw codes and estimated metres, respectively.
Figure 10: Qualitative visualization of the Insert a book into the bookshelf task in AgiBotWorld-Beta. Columns show the head, left-wrist, and right-wrist views, and rows show RGB images, available sensor depth, DA3-predicted depth, and the corresponding RGB-colored point clouds. Wrist sensor depth is unavailable. Sensor depth is visualized in raw codes, whereas DA3 depth is visualized in estimated metres.
Clean
Random
Task
S/N
SR (%)
S/N
SR (%)
adjust_bottle
100/100
100.00
72/100
72.00
beat_block_hammer
99/100
99.00
42/100
42.00
blocks_ranking_rgb
100/100
100.00
69/100
69.00
blocks_ranking_size
80/100
80.00
31/100
31.00
click_alarmclock
89/100
89.00
75/100
75.00
Appendix
Table 8: Per-task RoboTwin results for E0.5 at training step 109,000. S/N denotes successful episodes divided by evaluated episodes; SR is reported in percent. † marks tasks with fewer than the planned 100 Random evaluation episodes. Macro-average SR assigns equal weight to each of the 50 tasks.
Task
Task objective
OOD condition
Mouse placement
Move the mouse onto the target mouse pad.
Unseen initial mouse poses.
Charger packing
Place the charger body and the entire cable inside the receptacle.
Unfamiliar chargers and cables.
Trash disposal
Deposit all designated objects into the bin.
Six objects on each platform, including unseen objects. ID uses four objects on Piper and three on H1.
Appendix
Table 9: Real-world task objectives and OOD conditions. EASY and HARD correspond to ID and OOD, respectively.
World action models (WAMs) aim to answer a coupled physical question: given a task instruction, what motion should the robot execute, and how will that motion change the surrounding world? Most existing WAMs build on pretrained video generators and represent world evolution through images or visual latents. Robotic interaction, however, takes place in metric three-dimensional space, while images are view-dependent projections whose pixel distances do not directly encode physical distances. We introduce UNITAS, to our knowledge the first 3D-native world action model that unifies observations, actions, and scene dynamics in a shared metric 3D frame within each interaction, using a common representation across robot embodiments and human hands. Action flow represents human hands and robot grippers as 3D point trajectories, while scene flow describes scene-point displacements conditioned on these trajectories. World-aligned 3D positional embeddings ground visual tokens with or without depth input, and a physical-time trajectory tokenizer encodes each point trajectory as one token anchored at its current 3D position. This interface supports both direct action execution and action-conditioned scene prediction. With 1.7B parameters, UNITAS achieves the best action-conditioned scene prediction on RoboTwin among the compared methods, with up to 49% lower displacement errors than PointWorld, and state-of-the-art manipulation success, including 99.8% on LIBERO and an average of 85% across real-world tasks. The code is available at https://github.com/DexForce/UNITAS.
Ruixiang Wang, Yongyi Su, Wenlve Zhou +14
The Chinese University of Hong Kong, Shenzhen · DexForce Technology Co., Ltd. · Foshan University +2
World action models jointly learn to forecast world dynamics and predict robot actions, such that the learned internal world dynamics guide accurate actions. Existing approaches typically represent the world as RGB frames or latent counterparts while predicting actions as end-effector poses or joint angles, but they often struggle to capture the 3D spatial structure and contact geometry central to dexterous manipulation. We introduce Point World Action Model (PointWAM), a 3D world action model that decomposes the world into a scene (i.e., environment) and hands (i.e., actor), and jointly forecasts both as 3D point trajectories within a shared space-time coordinate frame. This explicit, disentangled representation enables effective pre-training on large-scale human demonstration videos without requiring any task-specific object or keypoint selection. Given a colored point cloud and a language instruction, PointWAM predicts how the scene and hands co-evolve in 3D space over time, then retargets the forecast hand motion to robot actions. Pre-training on human videos improves average DexJoCo success by 56.9 percentage points, and scene-trajectory supervision adds 10.9 points over forecasting the hands alone. With both, PointWAM surpasses the prior state of the art on ten DexJoCo tasks by 11.7 points and outperforms strong VLAs on a real robot.
Action-conditioned video world models predict future observations from an initial observation and an action signal. In robotics, actions influence future observations through two distinct processes: they are first realized into robot motion by the robot body and controller, and the scene then responds through contact and object motion. Conditioning directly on action commands asks the world model to learn the realization process itself, while conditioning on logged future states leaks the interaction outcomes it is meant to predict. We propose robot-factored world models, which move two robot-specific factors outside the world model. First, action realization: each command is rolled through the robot's own controller and kinematics into a deployment-available nominal trajectory, a middle signal that avoids both action-realization learning and future-state leakage. Second, robot rendering: this nominal trajectory is rendered through the robot URDF, factoring the robot's geometry, kinematics, and appearance out of the model and into explicit rendered robot geometry. To resolve depth ambiguity, we pair end-effector depth with scene depth, giving geometric cues for contact and occlusion beyond image-plane overlap. Together, camera-aware static RGB/depth context and rendered robot geometry form a shared visual world-model interface that stays consistent across viewpoints and robot embodiments, so the model sees the action only as visible robot geometry and learns how objects respond to it. Our experiments show that the rendered interface outperforms vector-conditioned baselines and generalizes to unseen robot embodiments at inference. We further demonstrate that our model generates robot manipulation videos from human demonstrations by retargeting and rendering the hand motion as robot geometry.