Organizations: School of Software, Shandong University, Jinan 250101, China · Shenzhen Loop Area Institute, Shenzhen 518038, China · School of Life Science and Technology, Xidian University, Xi’an 710126, China · School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), 518055, China
Fine-tuning a vision-language-action (VLA) model on a few dozen demonstrations is the standard way to put a manipulation policy on a robot, yet the resulting policy silently assumes that the camera and the robot base stay exactly where they were during data collection: a bumped tripod or a base re-docked a few centimetres away makes it reach for the wrong place. We show that both failures are one phenomenon, a change of the camera-to-base extrinsic that the policy has memorised, and that the robot carries everything needed to undo it. We introduce WACO, a world-anchored canonicalization framework for VLA models: the arm, placed by forward kinematics at the measured joint angles, serves as a metric calibration target, and the static scene serves as the world anchor that tells a moved camera from a moved robot. From a single RGB-D frame, WACO estimates both shifts without ground-truth extrinsics or extra data, canonicalises the image to the calibration viewpoint and the state, action and start pose to the calibration base frame, and fine-tunes the unmodified policy under the artefacts of its own estimator. We also introduce LIBERO-PoseShift, the first benchmark for RGB VLAs with graded camera, base and compound shifts. Across LIBERO-PoseShift, the LIBERO-Plus camera track and four real-world tasks on a UR5e robot, WACO consistently outperforms the standard few-shot recipe and prior viewpoint-robust methods, raising shifted success by 13.2 points in simulation and by 36.7 points on the real robot, recovering 90% of the simulation gap to a ground-truth oracle and 87% of the real-robot gap, and establishing a new state of the art among methods that use no perturbed training data, with no test-time calibration. Project page: https://peacelwh.github.io/WACO/.
Figures & tables
Fig. 1: Overview. A few-shot fine-tuned VLA memorises the camera-to-base extrinsic of its demonstrations and misses the target once the camera is bumped or the robot is re-docked (left: our UR5e setup at the calibration pose and after the table and camera were moved 10 cm away from the robot). WACO estimates both shifts from a single RGB-D frame, with the arm placed by forward kinematics as the metric anchor and the static scene as the world anchor, and canonicalises the observation back to the calibration viewpoint and base frame, so that an unmodified π0.5 sees the world it was trained in (right; yellow: outline of the calibration reference in the live view). We evaluate on LIBERO-PoseShift, the LIBERO-Plus camera track and four real-world UR5e tasks.
Shift handled
Requires
Method
Cam.
Base
Both
Extr.
Pert. data
Sim
Arch.
Calib.-free
OpenVLA-OFT+ [ 7 ]
✓
✗
✗
✗
✓
✓
✓
✓
RoVi-Aug [ 8 ]
✓
✗
✗
✗
✓
✗
✓
✓
VISTA [ 9 ]
✓
✗
✗
✗
✓
✗
✓
✓
Cross-View AC [ 11 ]
✓
✗
✗
✗
✗
✓
✓
✓
RayViT [ 13 ]
✓
✗
✗
✓
✓
✗
✗
✗
TABLE I: Related Work Summary. Which pose shifts a method handles, what it requires beyond the demonstrations, and whether the policy architecture is left unchanged. Extr. : ground-truth extrinsics at test time; Pert. data : perturbed or multi-view training trajectories; Sim : a simulator that re-renders states; Arch. : policy architecture unchanged; Calib.-free : no test-time calibration object or procedure.
Fig. 2: Model Pipeline. WACO wraps an unmodified π0.5 . At reset, a training-free front end registers the FK-placed arm model to the observed depth ( arm anchor ) and matches the static scene to an empty-workspace reference ( world anchor ); their fusion yields ΔTc , ΔTb and an out-of-plane residual that gates the estimate. Channel A warps the image with ΔTc only; channel B re-expresses state, action and start pose with ΔTb only. The policy is fine-tuned once on the canonicaliser’s own artefacts (bottom left).
Fig. 3: Two-Anchor Estimation. (a) Arm anchor: the visible part of the FK-placed arm cloud is registered to the observed depth by trimmed ICP and ranked by a render-and-compare score. (b) World anchor: the live frame is registered to the empty-workspace reference by SIFT, 3D RANSAC and point-to-plane ICP. (c) Fusion closes Tbc=ΔTb−1Tbc0ΔTc ; the discarded height, roll and pitch of the base shift form a ground-truth-free consistency check.
Fig. 4: Canonicalization and error-aware training. (a) Channel A forward-splats the shifted third-person image to the nominal camera with the measured depth; disocclusions and the out-of-frustum band are grey and are not hallucinated. (b) Channel B maps the reported end-effector pose into the nominal base frame with C=Twb0ΔTb(Twb0)−1 , rotates the policy’s deltas back before execution, and homes the end effector to the start pose the demonstrations began from; in the nominal frame a base shift has become an object displacement on the table. (c) Training splats a demonstration frame to a random virtual camera and back with a perturbed shift drawn from the estimator’s error model, which reproduces the holes, borders and residual misalignment of the test-time warp without changing the action label; the state receives a planar base-shift error about the base origin. Per-frame metric depth (right) removes the ghosting that a static reference depth leaves on the arm (middle).
Camera ΔTc
Base ΔTb
Level
trans.
rot./axis
trans.
yaw
px (t / yaw)
L1
≤ 2 cm
±3∘
≤ 2 cm
±3∘
6 / 12
L2
≤ 5 cm
±5∘
≤ 5 cm
±8∘
15 / 21
L3
≤ 10 cm
±10∘
≤ 10 cm
±15∘
31 / 42
TABLE II: Perturbation Levels of LIBERO-PoseShift. Camera translations are uniform in a ball and base translations in a disc, both between half and the full radius; rotations are uniform per axis. Last column: displacement of a target object in the 2562 third-person view for the largest translation / yaw.
Fig. 5: LIBERO-PoseShift Observations. Episode 0 of a LIBERO-Spatial task (“pick up the black bowl between the plate and the ramekin and place it on the plate”) and of a LIBERO-10 task (“pick up the book and place it in the back compartment of the caddy”): third-person and wrist views at the calibration pose, and the third-person view with the wrist view inset under camera, base and compound shifts of level 1–3. Yellow: outline of the calibration view.
Fig. 6: Shift Distribution and Further Scenes. Left: 80 draws per level from the benchmark’s sampler, seen from above: camera translations, and base translations with the sampled yaw drawn as a tick (rings at 2, 5 and 10 cm). Right: four further tasks at the calibration pose and under level-3 shifts, drawn as in Fig. 5 .
L1
L2
L3
all four suites (240 per level)
1.00
1.00
0.97
Spatial and LIBERO-10 (120 per level)
1.00
1.00
0.94
worst penetration (mm)
0
0
30
TABLE III: Feasibility of the Base Shifts : share of sampled base poses with no robot–environment penetration after reset and a stable arm after settling (six samples per task and level). Invalid samples are kept in the evaluation, so they bound the success at base and compound L3.
Fig. 7: Real-Robot Tasks. The four tabletop tasks form an increasing difficulty ladder: Lemon to Basket needs only coarse localisation; Color Matching adds an instruction-dependent referent and a cup opening only slightly wider than the block; Toy on Shelf requires carrying the toy over the shelf’s wire frame and releasing it on the elevated upper tier; Toast into Toaster requires an insertion whose clearance is far below the object size, into a toaster that stays at the same spot across the demonstrations.
Component
Time
World anchor (SIFT + RANSAC)
124 ms
World anchor refinement (point-to-plane ICP)
83 ms
Arm anchor (multi-hypothesis ICP)
524 ms
Whole front end, once per episode
771 ms
Channel A warp (per control step, when applied)
11 ms
π0.5 inference (per chunk of 10 steps)
≤ 320 ms
TABLE IV: Runtime of the front end (median over 36 resets, one CPU thread) and of the canonicaliser, and homing statistics (12 episodes per condition). Policy inference was timed on a GPU shared with a training run and is an upper bound.
Kind
None
World-frame protocol
WACO runtime
GT oracle
clean
96.0
–
97.5
–
camera
96.0 / 97.0 / 82.5
–
97.5 / 94.5 / 86.0
96.5 / 97.0 / 92.5
base
97.0 / 92.0 / 74.0
94.0 / 97.0 / 85.5
95.5 / 94.5 / 91.0
95.5 / 94.0 / 91.0
both
95.5 / 89.0 / 60.5
94.5 / 92.0 / 73.0
95.5 / 90.5 / 80.5
96.5 / 93.5 / 89.5
Avg over shifts
87.1
89.3 ∗
91.7 +4.6
94.0
TABLE V: Sensitivity and Oracle Bound. The public π0.5 checkpoint on LIBERO-Spatial and LIBERO-10 (200 episodes per cell; cells are L1 / L2 / L3; base-frame protocol unless noted). The WACO runtime is applied without retraining. ∗ World-frame average over base and compound rows only, since a camera shift is protocol-independent.
Requires
Camera
Base
Both
Method
extra data
run- time
Clean
L1
L2
L3
L1
L2
L3
L1
L2
L3
Avg
B1 (official few-shot recipe)
–
–
84.5
82.0
83.5
67.5
84.5
80.0
63.0
87.5
73.5
44.5
74.0
B1 + 2D aug (crop, rotate)
–
–
86.5
87.0
83.0
75.0
87.5
82.5
66.5
84.0
79.0
53.0
77.5 +3.5
B2 (W1 data, no runtime)
–
–
88.5
87.5
86.5
80.0
88.0
82.0
72.0
88.0
80.5
65.0
81.1 +7.1
Align-and-replay (B1 + front end, ch. B)
–
est.
86.0
82.5
80.0
65.5
86.5
90.0
83.0
86.0
77.0
63.0
79.3 +5.3
Base gauge (B1 + arm anchor only)
–
est.
86.5
85.0
80.5
64.0
87.5
74.0
56.0
82.5
73.0
46.0
72.1 − 1.9
TABLE VI: Pose-Shift Robustness on LIBERO-PoseShift (LIBERO-Spatial and LIBERO-10, 200 episodes per cell, base-frame protocol). All learned rows share checkpoint, demonstrations and budget; est. : shift estimated at test time; † : upper bound with ground-truth poses. Red: gain over B1. Base / camera gauge: the arm’s pose change read as a camera / base shift.
Method
Perturbed data
Released
Corrected
OpenVLA-OFT [ 7 ]
–
56.4
–
π0 -FAST [ 7 ]
–
65.1
–
CycleVLA ( π0.5 ) [ 63 ]
–
74.2 ‡
–
Cross-View AC [ 11 ]
sim re-render
87.2 ‡
–
TBD-VLA [ 64 ]
–
87.8 ‡
–
Flex- π [ 62 ]
–
–
91.6
TABLE VII: LIBERO-Plus Camera Track (1,599 tasks, one episode each). Released : the benchmark’s prompts, which append the variant’s parameters; corrected : the task’s own instruction [ 62 ] . ‡ Prompt handling not stated. Ours: evaluation seed 0 / mean over three evaluation seeds.
Demonstrations per task
10
20
50
B1 (standard recipe)
63.2
66.6
74.0
B1 + WACO runtime
73.3
76.3
81.4
W1 (error-aware training only)
71.7
73.3
81.1
WACO (W1 + runtime)
82.4
82.3
87.2
gain of WACO over B1
+19.2
+15.7
+13.2
TABLE VIII: Few-Shot Scaling : average shifted success (nine conditions) on LIBERO-Spatial and LIBERO-10 with the first 10, 20 or all 50 demonstrations per task, same recipe and budget.
Method
Nominal
Camera
Base
Both
Avg
π0.5 (B1)
92.5
60.0
40.0
30.0
43.3
B1 + 2D aug
93.8
72.5
42.5
32.5
49.2 +5.9
WACO runtime on B1
91.3
75.0
72.5
62.5
70.0 +26.7
WACO (W1 + runtime)
95.0
85.0
80.0
75.0
80.0 +36.7
GT oracle on W1 †
95.0
87.5
86.3
82.5
85.4
TABLE IX: Real-Robot Results (UR5e, success rate in %, 20 trials per task and condition, averaged over the four tasks; per-task numbers in Table S4 ). Avg is the mean over the three shifted conditions; red superscripts are gains over B1; † uses the re-calibrated ground-truth shifts and is an upper bound.
Fig. 8: Real-Robot Per-Task Performance. Success under pose shifts (mean of the camera, base and compound conditions) on the four UR5e tasks. WACO improves every task, most on Toy on Shelf (+46.7 points).
Block
Method
Camera
Base
Both
Avg
Gauges
Base gauge
55
25
20
33.3
Camera gauge
30
65
35
43.3
WACO
77.5
72.5
67.5
72.5
Few-shot (10 demos)
B1
45
25
15
28.3
WACO
72.5
67.5
62.5
67.5 +39.2
Occluded
WACO w/o gate
–
40
–
–
TABLE X: Real-Robot Analysis on Toy on Shelf and Toast into Toaster (20 trials per task and condition, averaged over the two tasks). Gauges : arm anchor only, the change read as a camera (base gauge) or base shift (camera gauge). Few-shot : the first 10 of 50 demonstrations per task. Occluded : a box hides about a third of the static background.
Block
Variant
Clean
Cam L3
Base L3
Avg
–
WACO (full)
87.0
84.5
86.5
87.2
Channels
A only (image)
87.0
82.5
67.5
82.0
B only (state, action)
87.0
77.5
81.5
83.6
B without homing
87.0
77.5
78.5
81.7
Anchors
arm only ( ΔTc=I )
87.0
78.5
86.5
82.6
scene only ( ΔTb=I )
87.0
84.0
66.5
81.7
TABLE XI: Ablations. Each block changes one component of the full method (blue row) on LIBERO-Spatial and LIBERO-10. The full row is the same evaluation as WACO in Table VI . Avg is the mean over the nine shifted conditions.
ΔTc error
ΔTb error
Gate
Condition
mm
deg
mm
deg
caught
false
clean
0.0 / 6.6
0.0 / 0.0
0.7 / 7.6
0.1 / 0.2
0/0
0
camera L1–L3
0.8 / 14.5
0.0 / 0.3
1.7 / 18.5
0.2 / 1.0
3/13
1
base L1–L3
0.0 / 3.9
0.0 / 0.0
2.9 / 10.6
0.1 / 0.3
4/4
0
both L1–L3
0.8 / 17.2
0.0 / 1.0
4.8 / 25.0
0.2 / 2.5
16/24
8
all
0.3 / 10.7
0.0 / 0.2
2.4 / 19.0
0.1 / 1.2
23/41
9
TABLE XII: Estimator Accuracy and Gate Behaviour over 240 resets (four suites, three tasks each, two episodes per condition): median / 90th-percentile error against the true shift. A catastrophic estimate exceeds 2 cm or 1.5∘ ; caught counts catastrophic estimates whose channel the gate disabled, false counts accurate estimates of a real shift that it disabled.
Fig. 9: Sensitivity to Calibration Error (base L3, LIBERO-Spatial, 200 episodes per point, 95% Wilson bands): yaw error injected into the ground-truth base canonicalisation, and the histogram of the yaw errors of the estimator’s applied base estimates.
ΔTc error
ΔTb error
Condition
mm
deg
mm
deg
Accepted
nominal
0.6 / 2.4
0.0 / 0.1
1.1 / 3.8
0.1 / 0.2
100%
camera shift
3.4 / 11.8
0.4 / 1.2
4.2 / 14.5
0.5 / 1.6
86.7%
base shift
1.9 / 5.7
0.2 / 0.4
3.6 / 10.2
0.3 / 0.9
96.7%
both
4.7 / 16.5
0.7 / 1.8
5.5 / 19.8
0.8 / 2.1
83.3%
all
2.8 / 11.2
0.3 / 0.9
3.8 / 13.7
0.4 / 1.2
91.7%
TABLE XIII: Real-Robot Estimator Accuracy against hand–eye re-calibration (median / 90th-percentile error; 10 poses per condition, three arm configurations each, 120 estimates). Accepted : share of estimates the gate applies.
Fig. S1: UR5e Setup. The workspace at the calibration pose (left) and after the base shift (right); the wrist D435i is mounted on the gripper.
ΔTc
ΔTb
Condition
cm
deg
cm
deg
Scale
camera
7.2
4.5
0
0
L3
base
0
0
10.0
0
L3
both
6.5
5.8
10.0
0
L3
TABLE S1: Real-Robot Shift Poses : translation and rotation of the camera relative to the table ( ΔTc ) and of the arm’s base relative to the table ( ΔTb ); the base shift as realised, the camera shifts as measured by hand–eye re-calibration. Last column: the corresponding level of LIBERO-PoseShift.
Front-end state
Tasks
WACO
no runtime
accurate, applied
1,423
94.3
64.4
inaccurate, applied
95
76.1
65.3
no usable estimate
67
64.2
46.3
correct but not trusted
14
97.6
71.4
TABLE S2: LIBERO-Plus Outcomes by Front-End State (corrected protocol, all 1,599 tasks; WACO: mean over three evaluation seeds; no runtime: the public checkpoint without canonicalisation). Accurate : within 2 cm and 2∘ of the true camera shift.
Fig. S2: Few-Shot Scaling. Average shifted success on LIBERO-PoseShift with the first 10, 20 and 50 demonstrations per task, same recipe and budget, per suite. The gap between WACO and the standard recipe widens as data shrinks, because the pose variation that the recipe must absorb from data is removed analytically; both the runtime and the error-aware training contribute, and their combination is better than either.
Method
Spatial
LIBERO-10
Mean
B1
79.6
68.4
74.0
B1 + 2D aug
83.6
71.4
77.5
B2
88.4
73.7
81.1
WACO runtime on B1
88.4
74.4
81.4
WACO
90.6
83.8
87.2
GT oracle on W1
91.0
86.3
88.7
TABLE S3: Per-Suite Results of the main comparison ( Table VI ); average over the nine shifted conditions (900 episodes per cell).
Method
Condition
Lemon to Basket
Color Matching
Toy on Shelf
Toast into Toaster
Mean
π0.5 (B1)
Nominal
95
95
90
90
92.5
Camera
80
65
50
45
60.0
Base
60
45
30
25
40.0
Both
50
35
20
15
30.0
Mean (shifted)
63.3
48.3
33.3
28.3
43.3
B1 + 2D aug
Nominal
95
95
95
90
93.8
TABLE S4: Real-Robot Per-Task Results (success rate in %, 20 trials per task and condition). Mean rows are highlighted; † uses the re-calibrated ground-truth shifts.
Fig. S3: Visual Result. LIBERO-Plus views far outside the training distribution: the live frame, the frame canonicalised by WACO, and the calibration reference, with the true viewpoint change and the stage of the front end that produced the estimate. Pixels the live camera did not see are filled from the reference, so objects appear only where the live view observed them.
Fig. S4: More Visual Results on the LIBERO-Plus camera track: live view, the view canonicalised by WACO, and the calibration reference.
Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.
Wenhao Li, Xueying Jiang, Quanhao Qian +4
1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1
Despite recent advances in Vision-Language-Action models (VLAs) for robotic manipulation, their performance remains sensitive to changes in camera configuration. The problem becomes more evident in cross-setup deployment, as reproducing the exact camera pose used for training is nearly impossible. Unlike fixed external views, wrist views are more challenging because the camera moves with the robot, causing even small mounting variations to alter fine-grained geometric cues. To address this, we propose WARP-VLA, a camera-view robust VLA for diverse wrist camera configurations. WARP-VLA adopts a Mixture-of-Experts (MoE) architecture where individual experts learn view-specific feature transformations, and a router combines them based on implicit view information. This allows the policy to be deployed without requiring camera extrinsic parameters as additional input. Through experiments on the LIBERO benchmark, WARP-VLA improves the average success rate of pi-0.5 from 39.2% to 78.3% under wrist-view perturbations. The real-robot experiments further show that the feature-level adaptation learned in simulation successfully transfers to diverse deployment settings. To facilitate reproducibility and future research, we release our wrist viewpoint robustness benchmark and a plug-and-play implementation.
Junmyeong Lee, Dongmin Shin, Min-Gyu Park +3
Yonsei University. · Korea Electronics Technology Institute(KETI).
Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
Hyeongjun Heo, Seungyeon Woo, Sang Min Kim +4
Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, South Korea · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA