Organizations: Institute of Automation, Chinese Academy of Sciences · Anyverse Dynamics · Tsinghua University · National University of Singapore · Fudan University
World-action models jointly predict future observations and actions, yet reconstruction-oriented visual representations do not explicitly emphasize the spatial relationships and object-state changes needed for manipulation. We introduce Phoenix, which learns representations of scene structure for world-action modeling. Specifically, Phoenix Encoder enriches DINOv3's general visual semantics through dual-teacher distillation of geometric and spatiotemporal semantic knowledge. Next, CS-VAE causally aggregates and compresses Phoenix Encoder features into generative latents while preserving scene information and spatial layout. Finally, Phoenix jointly predicts these latents, complementary appearance latents, and robot actions, linking scene evolution with action generation. To study pretraining data efficiency, we curate approximately 39,000 hours of egocentric and robot manipulation data for representation learning and use the robot demonstration subset for WAM pretraining. Following benchmark-specific post-training, Phoenix achieves state-of-the-art performance across simulation and real-robot benchmarks spanning single-arm, bimanual, and dexterous manipulation. It demonstrates strong robustness to perturbations, achieving success rates of 94.8% on LIBERO-Plus and 72.4% on RoboTwin 2.0 C2R, alongside strong cross-embodiment transfer and real-world manipulation performance on Bench2Dex and RoboDojo-RealWorld, respectively. Pretraining-scale analysis shows larger performance gains from data scaling than a video-VAE-only baseline, indicating stronger scaling benefits and more effective data utilization. Moreover, Phoenix maintains high success rates when adapted with limited downstream demonstrations, demonstrating strong few-shot adaptation. Our code, pretraining data, and models are available on the project page to help the community reproduce our results.
Figures & tables
Figure 1: Phoenix generalizes across environments and embodiments. We report success rates (%) on six simulation and real-world benchmarks after benchmark-specific adaptation.
Figure 2: Phoenix architecture. (a) VGGT- Ω and V-JEPA 2.1-L distill geometry and spatiotemporal semantics into a DINOv3-based Phoenix Encoder. (b) CS-VAE reconstructs frozen Phoenix Encoder features while causally aggregating and compressing nine frames into three latent time steps, retaining the patch layout. (c) Action tokens attend to visual tokens; visual tokens cannot attend to actions. The experts jointly predict scene and appearance latents and action chunks. Soft prompt tokens condition the shared control interface.
LIBERO-Plus
RoboTwin 2.0
Method
Camera
Robot
Language
Light
Background
Noise
Layout
Total ↑
Clean2Random
VLA
OpenVLA [ 22 ]
0.8
3.5
23.0
8.1
34.8
15.2
28.5
15.6
–
π0 [ 3 ]
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
16.3
OpenVLA-OFT [ 20 ]
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
–
StarVLA [ 9 ]
47.0
60.1
87.0
96.3
95.3
73.1
79.2
75.0
10.6
Table 1: LIBERO-Plus and RoboTwin 2.0 Clean2Random success rates.
Method
None
Equi.
Inv.
Full
SR
Visuomotor Policy
ACT [ 76 ]
29.5
15.4
14.0
13.0
18.0
Diffusion Policy [ 8 ]
12.9
6.3
5.2
3.8
7.1
VLA
π0.5 [ 16 ]
35.5
21.2
26.7
25.5
27.2
GR00T N1.5 [ 41 ]
48.5
18.8
26.0
19.8
28.3
Table 2: Bench2Dex success rate.
Method
ARX X5
Piper
Piper X
Avg.
VLA
StarVLA [ 9 ]
0.0
5.0
0.0
1.7
π0 [ 3 ]
0.0
5.0
0.0
1.7
X-VLA [ 78 ]
1.7
8.3
0.0
3.3
π0.5 [ 16 ]
13.3
21.7
3.3
12.8
WAM
Table 3: RoboDojo-RealWorld success rate.
Spatial readout
Predictive readout
Rank
Model
Depth ↓
Instance ↓
View ↑
Direct ↑
Path ↑
Avg. ↓
DINOv2-L [ 42 ]
0.1422
1.3828
95.57
23.73
9.00
2.60
Fast3R [ 59 ]
0.1386
1.6072
98.08
1.33
1.90
3.40
VideoMAE-L [ 47 ]
0.2968
4.1571
91.18
5.31
6.85
5.60
V-JEPA 2-L [ 1 ]
0.2708
2.9211
92.56
6.25
15.88
3.80
Qwen2.5-VL [ 2 ]
0.2244
2.9476
93.85
17.12
9.03
3.80
Table 4: Spatial and predictive probing results.
Figure 3: Depth (top; 0–6 m) and instance probing (bottom).
Representation
LIBERO-Plus
Appearance
Encoder
Compression
Camera
Robot
Language
Light
Background
Noise
Layout
Total ↑
VAE
–
–
75.5
69.0
81.9
84.7
55.0
88.3
79.0
76.9
VAE
VGGT- Ω
PCA48
81.9
60.4
64.5
94.2
85.0
90.0
77.9
78.3
VAE
V-JEPA 2.1-L
PCA48
77.1
72.3
76.9
98.7
86.9
92.8
83.0
83.2
VAE
DINOv3
PCA48
61.7
66.2
68.3
88.2
79.2
86.1
76.6
74.4
VAE
Phoenix Encoder
PCA48
79.0
70.2
82.8
98.5
86.4
94.4
83.3
84.4
Table 5: Visual representation and feature compression ablations. All models are post-trained without WAM pretraining.
Figure 4: Pretraining data scaling. Phoenix gains more from scaling pretraining data than the VAE-only baseline (Fast-WAM).
Figure 5: Post-training data efficiency of pretrained Phoenix.
Figure 6: Future-video predictions on LIBERO-Plus (left) and RoboTwin 2.0 C2R (right). Phoenix preserves clearer object structure and more coherent interaction dynamics.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
DINO-GT ↑
CLIP-GT ↑
LIBERO-Plus
Fast-WAM [ 70 ]
20.75
0.77
0.19
85.33
91.50
Phoenix (w/o. PT)
20.96
0.78
0.17
86.06
91.46
Phoenix (Ours)
23.31
0.81
0.12
90.28
93.09
RoboTwin 2.0 Clean2Random
Fast-WAM [ 70 ]
16.51
0.61
0.39
60.33
83.27
Table 6: Video prediction quality (PT: WAM pretraining).
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : Metric-depth probing. The Phoenix Encoder preserves both gradual changes in depth across a room and sharp depth discontinuities between adjacent surfaces.
Figure S2 : Instance-identity probing. Object boundaries remain distinct in the Phoenix representation as objects change position and apparent scale in the image.
Figure S3 : View-overlap classification. Views of the same workspace remain recognizable as overlapping even when shared objects appear at different image locations.
Figure S4 : Path integration. The Phoenix representation retains distinctions between viewpoints along a camera path, including views of the same room facing in different directions.
Figure S5 : Direct future-feature prediction. Given the camera motion, predicted Phoenix features can distinguish the target viewpoint among different views of the same scene.
Figure S6 : Additional future-video predictions on LIBERO-Plus (left) and RoboTwin 2.0 Clean2Random (right). Pretrained Phoenix produces clearer, more coherent predictions across six tasks.
Component
Configuration
Backbone
DINOv3 ViT-L/16 initialized from VGGT- Ω
Training input
256×256 , patch size 16
Feature map
16×16×1024 per frame
Adapters
Two branches; eight blocks each, width 1024, 16 heads, MLP ratio 4
Auxiliary tokens
One camera token and 16 register tokens
Adapter taps
Blocks {1,3,5,7}
Appendix
Table S6 : Phoenix Encoder architecture. Adapters, predictor, and projection heads are used only during distillation.
Table S8 : CS-VAE architecture. Encoder and decoder spatial blocks use the same configuration.
Setting
Video expert
Action expert
Transformer blocks
30
30
Residual width
3072
1024
FFN width
14336
4096
Attention heads
24
24
Head width
128
128
Q/K/V width
3072
3072
Appendix
Table S9 : World–action expert configurations. Q/K/V width differs from the action expert’s residual width.
Module
Parameters (M)
Status
Video expert
5000.97
Trainable
Action expert, including prompts
1021.26
Trainable
State projection and type embedding
0.34
Trainable
Phoenix Encoder backbone
303.23
Frozen
CS-VAE, encoder and decoder
72.33
Frozen
Wan VAE, encoder and decoder
704.69
Frozen
Appendix
Table S10 : Parameter counts for the reported configuration. M denotes 106 parameters; status refers to WAM training.
Benchmark
Updates
Execution steps
LIBERO-Plus / LIBERO-Pro
25k
10
RoboTwin 2.0 C2R
25k
32
RoboCasa-GR1
30k
12
Bench2Dex
20k
8
RoboDojo-RealWorld
30k
32
Appendix
Table S11 : Post-training and action execution. Execution steps are the number of predicted actions applied before replanning; each prediction contains 32 actions.
Figure S7 : Composition of the pretraining data pool. Original and retained durations are reported for each source. Italicized dataset names indicate non-public datasets.
Figure S8 : Robot data processing pipeline. Collected demonstrations undergo format conversion, quality filtering, annotation, and review. Coordinate-frame alignment and instruction augmentation prepare synchronized observations, states, actions, and instructions for sample construction.
Figure S9 : Unified 80-dimensional state–action interface. Slot boundaries use zero-based indexing. The two rows show the left-arm and right-arm portions together with body-control and reserved slots. Single-arm data use the right-arm slots. End-effector states use all nine pose dimensions, whereas end-effector actions use the first six and mask the remaining three. Missing and reserved dimensions are masked out of action supervision.