AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
Authors: Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim
Organizations: Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, South Korea · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
Figures & tables
Fig. 1: We present a zero-shot camera adaptation framework for VLAs that operates in real-time under various camera configuration changes from those used during training. At each control step, images from a test-time camera are synthesized into the training camera viewpoint at 30 Hz, while the frozen VLA policy runs at 10 Hz. Without any policy fine-tuning, our framework enables robust task execution across diverse setups—including different extrinsics, intrinsics, and freely moving handheld cameras such as an iPhone, ZED, and RealSense.
Fig. 2: (a) Three perturbation levels on the agent and wrist camera. For the agent camera, we set the intersection point of the workspace surface and the camera z -axis as the center of spherical coordinates and apply perturbations to (r,θ,ϕ) on camera poses. For the wrist camera, we perturb the x , y coordinates and the pitch of the camera poses on the wrist camera frame. (b) Visualization of agent and wrist cameras in the viewpoint augmented LIBERO dataset.
LIBERO-Spatial
LIBERO-Object
LIBERO-Goal
LIBERO-Long
All Suites
Method
S
M
L
Avg
S
M
L
Avg
S
M
L
Avg
S
M
L
Avg
S
M
L
Avg
OpenVLA-OFT
98.2
92.4
91.0
93.9
78.6
42.4
15.4
45.5
84.4
75.8
68.0
76.1
79.6
8.8
10.4
32.9
85.2
54.9
46.2
62.1
π0.5
97.6
86.0
48.2
77.3
97.6
86.0
48.2
77.3
88.8
66.8
48.4
68.0
85.6
46.8
14.6
49.0
92.4
71.4
39.9
67.9
π0.5∗
93.6
95.4
94.2
94.4
94.0
95.2
94.0
94.4
86.4
86.0
85.0
85.8
76.4
75.0
71.6
74.3
87.6
87.9
86.2
87.2
GeoAwareVLA
76.6
84.2
78.2
79.7
97.4
94.6
93.4
95.1
88.4
86.2
85.2
86.6
86.0
83.6
79.0
82.9
87.1
87.2
84.0
86.1
Ours-OV
99.0
98.4
97.4
98.3
95.8
63.4
49.6
69.6
96.2
94.0
96.4
95.5
89.6
79.4
67.6
78.9
95.2
83.8
77.8
85.6
TABLE I: Success rate (%) on unseen agent camera viewpoint across LIBERO suites.
Method
Small
Medium
Large
Average
π0.5
40.8
39.8
5.2
28.6
π0.5∗
84.0
84.0
81.2
83.1
GeoAwareVLA
1.6
5.0
9.0
5.2
Ours- π
91.8
89.6
84.4
88.6
TABLE II: Success rate (%) on unseen wrist camera viewpoint on LIBERO-Long.
Fig. 3: Success rate on LIBERO-Long with agent view variations by fine-tuning steps: (Left) Average task success rate on original view (Right) Average task success rate on all three unseen views.
Fig. 4: Qualitative results of LVSM on simulation and real images. Fine-tuning reduces the domain gap for input views on simulation.
Success Rate (%)
PSNR (dB)
Method
Small
Medium
Large
Average
Average
π0.5 (original view)
–
92.4
–
π0.5 (no adaptation)
85.6
46.8
14.6
49.0
13.64
Homography
74.6
9.6
10.8
31.7
14.72
Depth
85.2
84.6
73.4
81.1
18.27
Ours- π (w/o FT)
65.4
20.0
14.2
33.2
16.54
TABLE III: Success rate (%) and image quality on viewpoint adaptation ablation.
Fig. 5: Experimental setup for real-world experiments. We use two input views for VLA fine-tuning. At test time, we change one input view.
Fig. 6: Results of performance improvement on novel viewpoint in real world. (a) Task success rate on novel viewpoints. Bar and marker each indicate progress and binary task success rate. (b) Real-world execution snapshots. Each column illustrates the sequential progression of a task from the initial state to the final state.