Effective manipulation requires a policy to capture not only about the current scene geometry, but also about how that geometry will evolve under interaction. Existing approaches either augment policies with geometric information from current observations or introduce future prediction as auxiliary supervision, leaving open how predictive geometry should be represented for action generation. Therefore, we propose a two-stage framework, which learns complementary predictive representations from multi-view video clips. During pretraining, a frozen 3D foundation model provides dense geometric features that are compressed into predictive geometric representations and optimized to preserve scene structure and its temporal evolution. Action supervision is then applied to geometric transitions to extract a complementary motor representation that emphasizes action-relevant transitions. During downstream post-training, predictive geometry is transferred through geometry queries, while the motor representation guides intermediate features along the action-generation pathway. Future observations are used only as privileged supervision during training and are not required at inference time. Across LIBERO, LIBERO-Plus, SimplerEnv, and real-world bimanual and single-arm tasks, our approach consistently improves downstream policies, including an absolute gain of 10.4 percentage points on the WidowX evaluation. We further demonstrate transfer across multiple VLA architectures and WAMs, with improvements in both future-world and action prediction. The code will be available at https://github.com/xuxiaoxxxx/CPR.
Figures & tables
Figure 1: Comparison of geometry-guidance paradigms for policy learning. (a) Input-centric methods augment policies with spatial information derived from current observations. (b) Prediction-centric methods use future-world or predictive targets as additional supervision. (c) Our approach learns a predictive geometric representation from dense geometric features and derives a complementary motor representation from the evolution of scene geometry. The two representations are transferred through role-specific guidance pathways during training and are not required at inference time.
Figure 2: Overview of complementary predictive representation pretraining. A frozen 3D foundation model extracts dense geometric features from multi-view video observations, which are compressed by the Hierarchical Geometric Tokenizer and temporally modeled by a causal Slot-Temporal Mixer to produce the predictive geometric representation. Geometry reconstruction, masked geometric representation prediction, and geometric transition prediction jointly shape this representation to preserve scene structure and capture its temporal evolution. The resulting geometric transitions are used with action supervision to extract an action-aware motor representation, providing complementary predictive and motor information for policy learning.
Figure 3: Overview of representation-guided policy post-training. The frozen pretrained model provides the predictive geometric representation G and motor representation Z as privileged supervision. The two representations are transferred through role-specific pathways: G supervises dedicated geometry queries, while Z aligns with intermediate features along the action-generation pathway.
Table 4
Figure 4: Performance comparison on real-robot manipulation tasks.
Figure 5: Transfer of CPR to different downstream VLA models on the WidowX robot.
Methods
Object
Spatial
Goal
Long
Avg.
FastWAM wo/ CPR
100.0
98.2
97.2
95.2
97.6
FastWAM w/ CPR
100.0
99.0
97.4
96.8
98.5
Table 3: Transfer of CPR to FastWAM on the LIBERO.
Figure 6: Probing analysis of representation organization on LIBERO-90. The top compares action-error distributions between the CPR motor and unified representations, while the bottom shows depth-probing results.
Table 9
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Out-of-Distribution success rates on real-world tasks under visual and spatial perturbations.
Methods
Cam.
Rob.
Lang.
Light
Back.
Noise
Layout
Total
OpenVLA-OFT ( Kim et al., 2025 )
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
π0 ( Black et al., 2024 )
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
WorldVLA ( Cen et al., 2025 )
0.1
27.9
41.6
43.7
17.1
10.9
38.0
25.0
UniVLA ( Bu et al., 2025 )
1.8
46.2
69.6
69.0
81.0
21.2
31.9
43.9
4D-WAM ( Yang et al., 2026a )
45.2
64.3
90.6
94.3
57.7
69.1
79.2
71.0
ELAN4D ( He et al., 2026 )
63.7
70.7
77.8
89.8
91.4
79.9
81.4
78.2
Appendix
Table 6: Comparison of different VLA models on LIBERO-Plus. Success rates (%) are reported under the seven perturbation types. Total denotes the official task-count-weighted mean over all 10,030 perturbed tasks, rather than an unweighted mean of the seven categories.
Figure 8: Visualization of the real-world out-of-distribution perturbations. From left to right: Remove Stopper , with randomly shifted flask positions; Store Tubes , with randomly varied target-box heights; Transfer Liquid , with a randomly moving toy car as a dynamic visual distractor; and Make Sandwich , with additional irrelevant objects added to the workspace. These perturbations introduce changes in object placement, target geometry, dynamic visual context, and scene clutter while preserving the underlying manipulation tasks.
Model
PSNR ↑
SSIM ↑
LPIPS ↓
Action ℓ1↓
FastWAM w/o CPR
19.67
0.764
0.229
0.124
FastWAM w/ CPR
22.91
0.803
0.186
0.075
Appendix
Table 7: Zero-shot future-world and action prediction performance of FastWAM on LIBERO-90. PSNR and SSIM are higher-is-better, while LPIPS and action ℓ1 error are lower-is-better.
Figure 14
Figure 10: Qualitative comparison of zero-shot future world prediction on LIBERO-90. Each example contains a main camera view (upper half) and a wrist camera view (lower half), with columns progressing over nine predicted future frames. Within each view, the three rows from top to bottom correspond to ground-truth observations (GT), FastWAM without CPR guidance (w/o CPR), and FastWAM with CPR guidance (w/ CPR).
Figure 11: Distribution of action-probing experiment.
Figure 17
Figure 13: Qualitative visualization of the depth-probing analysis for representation organization. Each example contains a main camera view (upper half) and a wrist camera view (lower half), with columns progressing over nine future frames. Within each view, the rows from top to bottom show the corresponding RGB observations, depth predictions decoded from the unified representation, depth predictions decoded from the predictive geometric representation, and the pseudo-depth references extracted by Depth Anything ( Lin et al., 2025 ) . The predictive geometric representation produces depth maps that more faithfully recover object geometry, relative depth, and overall scene structure than the unified representation.
Methods
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
WoG ( Su et al., 2026 )
83.3
62.5
37.5
75.0
64.6
FutureVLA ( Xu et al., 2026 )
79.2
70.8
41.7
79.2
67.7
RepWAM ( Wang et al., 2026a )
87.5
75.0
54.2
58.3
68.8
Ours
95.8
70.8
70.8
50.0
71.9
Appendix
Table 8: Comparison with different predictive representation learning methods under a unified downstream policy framework on the WidowX evaluation.
Lrec
Lmask
Ltrans
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
(a)
83.3
58.3
54.2
50.0
61.5
(b)
✓
87.5
66.7
58.3
50.0
65.6
(c)
✓
✓
91.7
70.8
66.7
45.8
68.8
(d)
✓
✓
✓
95.8
70.8
70.8
50.0
71.9
Appendix
Table 9: Ablation of the pretraining objectives for the predictive geometric representation on the WidowX evaluation.
Figure 14: Attention-map comparison between the unguided baseline (middle) and the CPR-guided policy (right). CPR guidance produces more concentrated responses around task-relevant objects and interaction regions.
Motor Input
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
Predictive States G
87.5
66.7
66.7
54.2
68.8
Geometric Transitions ΔG
95.8
70.8
70.8
50.0
71.9
Appendix
Table 10: Effect of the input used to learn the motor representation on the WidowX evaluation.
Alignment Layer
6
12
18
24
30
36
SR (%)
69.8
71.9
67.7
64.6
65.6
61.5
Appendix
Table 11: Effect of the motor representation alignment depth on the WidowX evaluation. The alignment is applied to intermediate features from different layers.
Methods
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
DINO
87.5
75.0
54.2
62.5
69.8
VGGT
95.8
70.8
70.8
50.0
71.9
Appendix
Table 12: Effect of the foundation encoder used during predictive representation pretraining on the WidowX evaluation.
λgeo
0.1
0.3
0.5
0.7
0.9
SR (%)
69.8
71.9
70.8
68.8
65.6
Appendix
Table 13: Ablation study of the geometry-guidance loss weight λgeo on the WidowX evaluation.
Guidance Paradigm
Put Spoon
Put Carrot
Stack Cube
Put Eggplant
Avg.
Input-centric
83.3
66.7
58.3
45.8
63.5
Prediction-centric
87.5
62.5
62.5
50.0
65.6
Ours
95.8
70.8
70.8
50.0
71.9
Appendix
Table 14: Controlled comparison of different geometry-guidance paradigms under the same downstream VLA framework on the WidowX evaluation. The input-centric variant uses current-frame VGGT features as policy input, while the prediction-centric variant predicts future VGGT features as auxiliary guidance.
Figure 27
Figure 16: Qualitative visualization of predictive geometry transferred through the geometry-query pathway. The first panel shows examples from LIBERO, where each example contains a main camera view (upper half) and a wrist-camera view (lower half); within each view, the rows from top to bottom show the RGB observation, the depth prediction decoded from the predicted geometry-query features, and the pseudo-depth reference generated by Depth Anything. The second panel shows real-world manipulation examples with three camera views, corresponding to the main view, the left-arm wrist view, and the right-arm wrist view. For each view, the rows from top to bottom follow the same order: RGB observation, predicted depth, and pseudo-depth reference. The results illustrate that the geometry-query pathway can recover meaningful object geometry and relative spatial structure across both simulated and real-world settings.
Dataset
Ratio
UTAustin Mutex ( Shah et al., 2023 )
2.3
DLR EDAN Shared Control ( Quere et al., 2020 )
0.7
Taco Play ( Rosete-Beas et al., 2023 ; Mees et al., 2022 )
2.3
Austin Sirius Dataset ( Liu et al., 2025 )
1.5
Roboturk ( Mandlekar et al., 2019 )
1.6
Austin Sailor Dataset ( Nasiriany et al., 2022 )
1.7
Appendix
Table 15: Our training data mixture used during the pretraining phase.
Figure 17: Attention mask of FastWAM. F denotes video frames, G indicates predictive geometric representation and A represents actions to be denoised.
Figure 18: Bimanual and single-arm manipulation platforms.
Figure 19: Visualization of four real-robot manipulation tasks.
Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions. However, existing methods typically rely on 2D features with limited spatial awareness, or require explicit point clouds that are difficult to obtain reliably in real-world settings. At the same time, recent 3D geometric foundation models show that accurate and diverse 3D structure can be reconstructed directly from RGB images in a fast and robust manner. We leverage this opportunity and propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model. Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap. By explicitly predicting how the 3D scene will evolve together with the action sequence, the policy gains strong spatial understanding and predictive capability using only RGB observations. We evaluate our method both in simulation on the RoboTwin benchmark and in real-world robot executions. Our approach consistently outperforms 2D-based and point-cloud-based baselines, achieving state-of-the-art performance in manipulation success, inter-arm coordination, and 3D spatial prediction accuracy. Code is available at https://github.com/Chongyang-99/GAP.git.
Chongyang Xu, Haipeng Li, Shen Cheng +4
College of Computer Science, Sichuan University, China · Dexmal · University of Electronic Science and Technology of China, China +1
Human videos provide rich manipulation experience, but extracting action representations that preserve useful motion remains challenging. Visual reconstruction alone can entangle manipulation-related motion with appearance changes and camera movement. We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human videos. GeoLAM combines future-frame reconstruction through a frozen geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher. The geometric representation provides a structural prior, while the teacher's predictions yield spatially pooled targets capturing 3D displacement, residual image-plane motion, and surface-orientation changes. Visibility and confidence weighting reduces the contribution of unreliable estimates, encouraging continuous latent actions to retain geometric motion without explicit hand-pose or hand-trajectory annotations. After video pretraining without action labels, the learned representation provides transition targets for a world-action model trained on action-labeled robot demonstrations. The model jointly denoises latent actions and executable action chunks, with future-video prediction used only as an auxiliary training task. Deployment therefore requires neither the geometry teacher nor future-video generation. Evaluations on a latent-action benchmark and robotic manipulation tasks demonstrate the strong performance of GeoLAM.
Yifan Xie, Hekun Tian, Jinkun Liu +3
Tsinghua University · Beijing Institute of Technology · Fudan University
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models, but they still operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation. We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy that directly repurposes a pretrained geometric foundation model (GFM) as a shared substrate for perception, temporal prediction, and action decoding. GAM splits the GFM at an intermediate layer: the shallow layers serve as an observation encoder, and a causal future predictor inserted at the split layer forecasts future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining GFM blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions. This design equips the GFM with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors. Across a broad suite of simulation and real-robot manipulation benchmarks, GAM is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines.