GS-VLA: Plug-and-Play Viewpoint Canonicalization for Frozen VLA Policies via Gaussian Splatting

Yechan Park; HyunJin Kim

cs.CVAug 19, 2026

GS-VLA: Plug-and-Play Viewpoint Canonicalization for Frozen VLA Policies via Gaussian Splatting

Authors: Yechan Park, HyunJin Kim

Organizations: Dankook University Yongin, Republic of Korea · Dept. of EEE, Dankook University Yongin, Republic of Korea

Abstract

This paper proposes a lightweight, plug-and-play framework that improves robustness to viewpoint shifts in Vision-Language-Action (VLA) policies without policy retraining. To our knowledge, this is the first approach to directly leverage 3D Gaussian-based novel-view synthesis for observation-space adaptation in VLA policies. Current VLA performance relies on the implicit assumption that training and deployment camera configurations are identical. Our experiments show that even a small displacement of the camera mount can reduce the success rate on the LIBERO benchmark from about 90% to about 10% in the worst case. Prior approaches, such as large-scale fine-tuning or generative data augmentation, are computationally expensive and risk catastrophic forgetting. To address this, viewpoint shifts are reformulated as a localized novel-view synthesis problem. Under a Locality assumption, that camera perturbations remain within a small bounded region relative to the workspace, viewpoint normalization reduces to a scene- and policy-independent disocclusion task. Our work implements this idea with a 4M-parameter 3D-Gaussian canonicalizer prepended to a frozen VLA policy. Without modifying policy weights, GS-VLA improves performance across three orthogonal axes: (1) Policy architectures, (2) Unseen task suites, and (3) Perturbation scales. These results show that a lightweight visual module can recover a large fraction of the performance lost under viewpoint shift, without policy retraining.

Figures & tables

(a) Cross-policy generalization ( $\varepsilon_{t}{=}100$ cm, $500$ ep)
Policy / Setting	Suite	No canon	Fwd-warp	GS-VLA (ours)	$\Delta$ (pp)
		SR (%)	SR (%)	SR (%)
XVLA	spatial	$1.4$	$19.7$	$\mathbf{81.0}$	$\mathbf{+79.6}$
OpenVLA-OFT (7 B)	object	$19.8$	$30.5$	$\mathbf{81.6}$	$\mathbf{+61.8}$
RynnVLA-002	spatial	$25.8$	$37.8$	$\mathbf{78.6}$	$+52.8$
$\pi_{0.5}$ (3.4 B)	spatial	$42.6$	$56.5$	$\mathbf{86.8}$	$+44.2$

Table 1: Three-axis generalization from a single checkpoint. GS-VLA is trained once on libero_spatial at

\varepsilon_{t}{=}100

cm, transfers without retraining across (a) policies, (b) suites, and (c) perturbation scales. Within each block,by descending

\Delta

in (a) and (b), by descending

\varepsilon_{t}

in (c). Boldface indicates the best result in each row, while bold

\Delta

highlights the headline cell of each block. The Fwd-warp column reports a depth-only forward-warp baseline where measured. Full per-cell matrices in Table 10 and Section 4.3 .

Method \ $\varepsilon_{t}$ (cm)	$5$	$15$	$30$	$50$	$100$	$200$
No canon	$85.2$	$82.7$	$80.6$	$73.5$	$42.6$	$35.7$
GS-VLA (ours)	$\mathbf{88.0}$	$\mathbf{88.6}$	$\mathbf{87.6}$	$\mathbf{87.2}$	$\mathbf{86.8}$	$\mathbf{78.5}$
$\Delta$ (pp)	$+2.8$	$+5.9$	$+7.0$	$+13.7$	$\mathbf{+44.2}$	$+42.8$

Table 2:

\varepsilon_{t}

-sweep on

\pi_{0.5}\!\times\!\texttt{libero\_spatial}

. The baseline drops

49.5

pp across the sweep; GS-VLA drops only

9.5

pp. Term

\Delta

grows approximately linearly in

\varepsilon_{t}

within the Locality bound, matching Appendix A.1 ; the largest cell (

\varepsilon_{t}{=}100

cm) is in bold.

Yaw $\theta$	$\varepsilon=100$ cm	$\varepsilon=200$ cm
$\theta=10^{\circ}$	$50.0\!\to\!\mathbf{81.8}$ ( $+31.8$ )	$35.8\!\to\!\mathbf{78.0}$ ( $\mathbf{+42.2}$ )
$\theta=20^{\circ}$	$45.5\!\to\!\mathbf{66.5}$ ( $+21.0$ )	$34.8\!\to\!\mathbf{69.3}$ ( $+34.5$ )

Table 3: Joint translation

\times

rotation. on

\pi_{0.5}\!\times\!\texttt{libero\_spatial}

. Entries: No canon

\to

GS-VLA (

\Delta

), in

\%

SR. The bold

\Delta

flags the largest gain (

+42.2

pp).

Calib. noise $(T,R)$	object	spatial	goal	libero_10
	SR (%)	SR (%)	SR (%)	SR (%)
$(0,0)$	$\mathbf{89.2}$	$\mathbf{79.9}$	$\mathbf{88.5}$	$\mathbf{65.4}$
$(1,1)$	$87.3$	$78.2$	$86.6$	$63.4$
$(2,2)$	$86.1$	$75.0$	$86.5$	$59.9$
$(3,3)$	$84.0$	$76.0$	$85.4$	$51.6$
$(5,5)$	$78.4$	$70.3$	$81.0$	$41.6$

Table 4: Calibration stress. Pose noise

(T\text{\, cm},R^{\circ})

injected on top of

\varepsilon{=}100

cm and applied only to the pose seen by the canonicalizer. The shaded zero-noise row is the GS-VLA reference; degradation is graceful up to

\sim\!3

cm/

3^{\circ}

, with long-horizon libero_10 the most sensitive at

5

cm/

5^{\circ}

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

libero_spatial ( $\rho\leq 0.5$ )
Mode	$\eta_{\mathrm{par}}$	$\eta_{\mathrm{fr}}$	$\eta_{\mathrm{tot}}$	$R^{2}$
Translation	$0.082$	$0.281$	$0.363$	$0.999$
Rotation	${<}0.001$	$1.084$	$1.084$	$0.995$
Joint	$0.038$	$0.607$	$0.645$	$0.994$
libero_object ( $\rho\leq 0.5$ )
Translation	$0.050$	–	–	–

Appendix

Table 5: Empirical slopes of

\eta(\rho)

. Per-mode slopes for parallax (

\eta_{\mathrm{par}}

), frame (

\eta_{\mathrm{fr}}

), and total (

\eta_{\mathrm{tot}}

). Silhouette-hugging: occluded pixels average

5.05

6.78

px from depth edges (spatial/object) vs.

26.77

33.10

px for uniform-random pixels (

5\!\times

tighter). Object suite uses translation- and rotation-only fits; joint mode pending.

Link	Quantity	Slope	$R^{2}$
$\varepsilon\to\eta$	joint $\eta/\rho$ slope (Tab. 5 )	${\approx}0.645$	$0.99$
$\eta\to\text{residual}$	predictor linearity	$c_{\mathrm{recon}}=0.049$	$0.81$
$\text{residual}\to\Delta$	policy Lipschitz	$L\approx 213$ pp/L 1	$1.00$
$\varepsilon\to\Delta$ (composite)	end-to-end ( $\varepsilon\leq 50$ )	$6.67$ pp/ $\rho$	$0.996$

Appendix

Table 6: Success-rate coupling chain (Appendix A.1 ). Each individual link and the end-to-end drop fit linearly with high

R^{2}

. The implied

L\approx 213

pp per

L_{1}

-residual unit lies inside the spectral-norm bound for

\pi_{0.5}

(

\log\prod_{l}(1+L_{l})=389

Component	Specification
Backbone	Symmetric U-Net, base width $C{=}32$ , 4 down-/up-sampling levels
Conv block	FiLM-modulated ConvBlock at every scale
Pose embedding	$z_{\mathrm{pose}}\!=\!\mathrm{MLP}([t_{s},q_{s},t^{\star},q^{\star}])\in\mathbb{R}^{128}$ , 2-layer
Input	$[I_{s}\,\\|\,D_{s}]\in\mathbb{R}^{4\times H\times W}$ , $H{=}W{=}256$
Per-pixel output	14-dim Gaussian descriptor $(\Delta\mu,\log s,q,\alpha,\Delta c)$
Centre clamp	$\mu=X_{w}+0.1\,D_{s}\,\tanh(\Delta\mu)$ ( $\leq 10\%$ of local depth)

Appendix

Table 7: Canonicalizer architecture. With the listed initialization,

f_{\phi}

starts at the closed-form forward warp at iteration 0 and learns only a residual.

Hyperparameter	Value
Optimizer	AdamW ( $\beta_{1}{=}0.9$ , $\beta_{2}{=}0.999$ , weight decay $10^{-4}$ )
Base learning rate	$5\!\times\!10^{-5}$
LR schedule	$500$ -step linear warmup (start factor $0.01$ ), then cosine to $0$
Batch size	$16$
Epochs	$50$
Gradient clipping	global norm $1.0$

Appendix

Table 8: Training configuration. Only the canonicalizer parameters are updated; the policy is never touched. No discriminator or perceptual GAN loss is used.

Stage	Per-frame cost (A5000)
U-Net forward	$\approx 5$ ms
Differentiable $\alpha$ -splat ( gsplat )	$\approx 8$ ms
Total canonicalizer $f_{\phi}$	$\approx\mathbf{13}$ ms ( $\approx 75$ Hz)
Rasterizer config	$45^{\circ}$ vertical FOV, bilinear splatting

Appendix

Table 9: Inference latency. The downstream policy keeps its native action-chunk schedule (e.g.,

50

for

\pi_{0.5}

), so the canonicalizer cost is amortized across many actions per frame. Softmax splatting is not used in the released checkpoint.

Policy	Suite	No canon	GS-VLA (ours)	$\Delta$ (pp)
$\pi_{0.5}$ (3.4 B)	spatial	$42.6$	$\mathbf{86.8}$	$+44.2$
	object	$64.0$	$\mathbf{90.2}$	$+26.2$
	goal	$66.3$	$\mathbf{92.2}$	$+25.9$
	libero_10	$9.3$	$\mathbf{72.1}$	$\mathbf{+62.8}$
OpenVLA-OFT (7 B)	spatial	$87.2$	$\mathbf{92.0}$	$+4.8$
	object	$19.8$	$\mathbf{81.6}$	$\mathbf{+61.8}$

Appendix

Table 10: Full cross-policy

\times

suite matrix with

\varepsilon_{t}=100

cm. Bold

\Delta

values denote the absolute increase in success rate (percentage points) for key configurations, such as XVLA (

+79.6

pp),

\pi_{0.5}

on libero_10 (

+62.8

pp), and OpenVLA-OFT on object-centric tasks (

+61.8

pp).

Suite (GS-VLA)	$\varepsilon_{t}{=}5$	$15$	$30$	$50$	$100$	$200$
spatial	$\mathbf{88.0}$	$88.6$	$87.6$	$87.2$	$86.8$	$78.5$
object	$\mathbf{92.2}$	$92.2$	$91.4$	$92.7$	$90.2$	$84.7$
goal	$\mathbf{93.9}$	$93.6$	$92.4$	$93.2$	$92.0$	$85.7$
libero_10	$71.9$	$72.7$	$73.4$	$70.8$	$72.1$	$52.9$

Appendix

Table 11: GS-VLA absolute SR (%) across suites as

\varepsilon_{t}

varies. object and goal stay within

\sim\!2

pp of their

\varepsilon_{t}{=}5

ceiling up to

\varepsilon_{t}{=}100

cm; the main drop appears at

\varepsilon_{t}{=}200

cm—the per-suite Locality boundary.

Condition	pos $\sigma$ (m)	lookat $\sigma$	No canon	GS-VLA (ours)	$\Delta$ (pp)
$\varepsilon_{t}{=}15$ (low) †	n/a	n/a	$82.7$	$\mathbf{88.6}$	$+5.9$
$1\!\times$ (default, $\varepsilon_{t}{=}100$ )	$(0.3,0.5,0.25)$	$0.06$	$42.6$	$\mathbf{86.8}$	$+44.2$
$2\!\times$ (bigrand) ‡	$(0.6,0.8,0.5)$	$0.06$	$41.6$	$\mathbf{80.8}$	$+39.2$
$3\!\times$ (extreme) ‡	$(0.9,1.2,0.75)$	$0.06$	$34.4$	$\mathbf{76.1}$	$\mathbf{+41.7}$

Appendix

Table 12: Magnitude sweep on

\pi_{0.5}\!\times\!\texttt{libero\_spatial}

. While the performance drop (

\Delta

) increases with perturbation magnitude, the canonicalizer exhibits significantly slower decay compared to the No canon baseline. This gap (utility) widens progressively until the Locality boundary. † The

\varepsilon_{t}{=}15

row follows the single-scalar protocol from Table 2 . ‡ Labels

2\!\times

3\!\times

are nominal; position

\sigma

is capped to remain within the workspace, and lookat

\sigma

is fixed at

1\!\times

Base $C$	Params	PSNR (dB)	SR (%)
$16$	$\sim\!1$ M	$25.77$	–
$24$	$\sim\!2$ M	$26.33$	–
$\mathbf{32}$ (ours)	$\mathbf{4}$ M	$26.84$	$\mathbf{86.8}$
$64$	$15$ M	$\mathbf{28.40}$	$85.2$

Appendix

Table 13: Capacity ablation. Reconstruction PSNR keeps improving with

C

, but SR peaks at

C{=}32

. Beyond

\sim\!4

M parameters, extra capacity goes to texture detail the policy does not consume.

Pairs $N$	PSNR (dB)	SR (%)
$10$ k	$26.06$	$\sim\!84$
$25$ k	$26.38$	$\sim\!85$
$\mathbf{50}$ k (ours)	$\mathbf{26.84}$	$\mathbf{86.8}$

Appendix

Table 14: Data-size ablation (

C{=}32

\varepsilon{=}100

cm). The signal saturates near

25

k pairs, consistent with the low intrinsic learning complexity that Locality predicts.

fov y	SR (%)	$\Delta$ vs. ref
$-10\%$ (zoom-in)	$86.4$	$-2.8$
$-5\%$	$87.9$	$-1.3$
ref ( $0\%$ ) (ours)	$\mathbf{89.2}$	–
$+5\%$	$89.1$	$-0.1$
$+10\%$ (zoom-out)	$89.0$	$-0.2$

Appendix

Table 15: Intrinsics stress on libero_object ,

1000

ep, atop

\varepsilon{=}100

cm extrinsic. Tolerance is

\sim\!4\!\times

tighter than for extrinsic noise in Table 4 .

Depth source	Encoder	val $L_{1}$	SR (%)
GT depth (upper bound) (ours)	–	–	$\mathbf{86.8}$
Calibration only (no fine-tune)
DA V2, raw inverse	ViT-S	–	$40.9$
DA V2, metric indoor	ViT-S	–	$44.5$
Fine-tuned on $50$ k LIBERO pairs (3 ep, $L_{1}$ + edge-aware)
DA V2-S, FT	ViT-S	$0.0092$	$68.6$

Appendix

Table 16: Depth-substitution ablation ( libero_spatial ,

\varepsilon=100

cm,

1000

ep unless marked). Calibration alone caps at

44.5\%

; a fine-tune reaches

71.7\%

. Larger encoders, a refinement head, and end-to-end gradients all fail to improve further — the residual

-15

pp gap is not a depth-accuracy bottleneck.

Robustness of VLA Models

Robustness of vision-language-action (VLA) models concerns reliable robot behavior under changes to language, vision, sensors, tasks, and hardware.

See all 221 papers

AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models
Mar 6, 2026Hyeongjun Heo, Seungyeon Woo, Sang Min Kim +4Test-Time Adaptation of VLA Models Robustness of VLA Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, South Korea · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies
Aug 7, 2026Bingqi Huang, Bingchuan Wei, Xuan Wang +2Multi-View Consistency Vision-Based Robot Control
Tsinghua University · Informatics Institute, University of Amsterdam & Faculty of Science, Vrije Universiteit Amsterdam
Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse
Sep 27, 2026Futa Waseda, Shuhei Kurita, Isao EchizenAdversarial Training Multi-View Learning
National Institute of Informatics Tokyo, Japan

Mar 6, 2026J/K move · Enter open · S save

AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models

Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim+3

Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, South Korea · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA

Test-Time Adaptation of VLA Models Robustness of VLA Models Vision-Language-Action Models

Open paper AskPDF arXiv

Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/

Abstract

Robustness of VLA Models

Explore similar work

AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models

Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies

Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse