InfraVLA: Extending Vision-Language-Action Navigation with Infrastructure Cameras

Lukas Vierling; Benjamin Ramtoula; Luke Robinson; Ronald Clark; Daniele De Martini

cs.ROSep 27, 2026

InfraVLA: Extending Vision-Language-Action Navigation with Infrastructure Cameras

Authors: Lukas Vierling, Benjamin Ramtoula, Luke Robinson, Ronald Clark, Daniele De Martini

Organizations: Dept. of Computer Science, University of Oxford · Oxford Robotics Institute, University of Oxford

Abstract

Many indoor environments in which robots operate, such as warehouses, offices, and hospitals, already have cameras installed. They observe parts of the building that the robot cannot see from where it stands, yet navigation policies, including recent vision-language-action (VLA) models, do not use them. We propose InfraVLA, an end-to-end method that adapts a pretrained navigation VLA to such static infrastructure views: a closed-circuit television (CCTV) encoder turns each external view into tokens of the input sequence. Because the views matter only at rare decision points, fine-tuning alone did not make the policy use them in our experiments; we therefore train in two stages, on demonstrations with upsampled counterfactual data and then on recovery data. We evaluate on two simulated warehouse tasks, finding an object named in the instruction and rerouting around blocked aisles, where the deciding information is often visible only to the infrastructure cameras. Tested in distribution, InfraVLA reached a success rate of 100% on both, against 34.0% and 73.6% for a baseline without CCTV input. On out-of-distribution test sets it reached 88.2% and 88.9%. On a real quadruped fine-tuned with under 10 minutes of demonstrations, the policy reached 83.3% against 29.2% for the on-board-only baseline.

Figures & tables

Panel of Figure 1 — Fig. 2: Architecture. Green: our extension, the infrastructure images, encoded per view, pooled from 256 to 64 tokens, and inserted after the on-board tokens. Orange: the on-board image pipeline; blue: the task specification; both as in OmniVLA [ 2 ] .

	Semantic (sim)	Spatial (sim)	Semantic (real)
CCTV cameras	4	4	2
Goal modality	language	2D pose	language
Success	$1\text{\,}\mathrm{m}$ to object	$1\text{\,}\mathrm{m}$ to goal	human-judged
Test rollouts	50	72	24
Trajectories (train / val.)	552 / 48	481 / 46	75 (all)
Training data (h)	3.03	6.89	0.16

TABLE I: Hyperparameters of the three experiments: data, training, recovery mining, snippet detection, and success criterion.

Panel of Figure 3 — Fig. 3: Semantic task. Left: two rollouts of one scene in plan view, both instructed to reach the box (brown square) and both reaching it. From the black start dot, InfraVLA after stage 1 (gray) passes close to a shelf (dark gray) and after stage 2 (green) keeps its distance. Right: what the four ceiling cameras see in that same scene at the start; the box is top left.

Policy	SR (%) $\uparrow$	Clearance (m) $\uparrow$
Fine-tuned OmniVLA [ 2 ] (no CCTV)	34.0	0.67
InfraVLA, no upsampling	36.0	0.62
InfraVLA, upsampling (stage 1)	100.0	0.71
$+$ recovery (stage 2)	100.0	1.27

TABLE II: Semantic task, 50 rollouts. SR: success within

1\text{\,}\mathrm{m}

of the object. Clearance: mean over rollouts of the closest distance to a shelf (demonstrations:

1.47\text{\,}\mathrm{m}

Object	Synonym 1	SR (%)	Synonym 2	SR (%)
Forklift	lift truck	91.7	vehicle	66.7
Barrel	canister	91.7	oil drum	75.0
Box	package	91.7	crate	83.3
Traffic cone	safety cone	100.0	witch’s hat	25.0

TABLE III: Semantic task, unseen instruction words: SR over 12 rollouts per word, for two synonyms of each training noun.

Evaluation set	Rollouts	SR (%) $\uparrow$
In-distribution	50	100.0
Held-out scene, other target	103	99.0
Held-out scene, forklift	34	88.2

TABLE IV: Semantic task, out-of-distribution tests. A separate policy trained through stage 2 with every scene that puts the forklift in the top-left aisle held out.

Panel of Figure 7 — Fig. 4: Spatial task. Left: one test scenario in plan view. From the black start dot both policies drive the same route to the middle area; the on-board-only baseline (orange) then enters the aisle sealed by a forklift (yellow), while InfraVLA (green) takes the open aisle and reaches the goal (star). Right: the four ceiling views of the same scenario at the start; the forklift that seals the aisle is visible in the bottom-right view.

Policy	SR (%) $\uparrow$	Route (%) $\uparrow$
Fine-tuned OmniVLA [ 2 ] (no CCTV)	73.6	55.6
InfraVLA, no upsampling	76.4	51.4
InfraVLA, upsampling (stage 1)	77.8	44.4
$+$ CF snippets only, no recovery	75.0	52.8
$+$ recovery (stage 2), $K=6$	100.0	91.7
$+$ recovery (stage 2), $K=12$	100.0	93.1

TABLE V: Spatial task, 72 systematic scenarios. Route: share of rollouts through the same aisles as A*.

+

rows: the stage-1 policy trained 5k more steps on the data named (stage 2). CF: counterfactual.

Policy	Seen (%) $\uparrow$	Held-out (%) $\uparrow$
InfraVLA, stage 1	75.9	33.3
$+$ recovery (stage 2), $K=30$	100.0	88.9

TABLE VI: Spatial task, out-of-distribution test. A separate policy trained with two of the nine blocking patterns held out; recovery data from the seen patterns only. SR on the 54 seen and 18 held-out scenarios.

Panel of Figure 10 — Fig. 5: The real setup, seen from the two fixed cameras at one instant of a demonstration. Left: CCTV 1 shows the traffic cone and the fire extinguisher. Right: CCTV 2 shows the other half of the area, with the robot and the plant. A logo on the robot has been masked for anonymity.

Policy	Overall	Plant	Fire ext.	Cone
Fine-tuned OmniVLA [ 2 ]	29.2	12.5	50.0	25.0
InfraVLA	83.3	100.0	62.5	87.5

TABLE VII: Real robot, 8 test arrangements with one rollout per object (24 rollouts). SR (%)

\uparrow

of reaching the named object, judged by the operator, overall and per object.

Panel of Figure 12 — Fig. 6: Obstacle avoidance preserved from pretraining, seen from CCTV 1. Top: instructed to reach the traffic cone, the policy walks around a blue bin. Bottom: instructed to reach the plant, it passes between two warning signs. The strip above each photograph holds four on-board frames in chronological order, each linked to the numbered disc marking roughly where it was taken; the disc positions are approximate. A logo in the background of two frames has been masked for anonymity.

Policy	SR (%) $\uparrow$	Clearance (m) $\uparrow$
InfraVLA, upsampling (stage 1)	100.0	0.71
16 CCTV tokens per view	28.0	0.77
Shared CCTV encoder	96.0	0.61
CCTV tokens first	94.0	0.65
Frozen on-board encoder	90.0	0.59
Frozen on-board encoder and LLM	26.0	0.61

TABLE VIII: Ablations of the architecture, on the semantic task with the stage-1 recipe, 50 rollouts each. First row: the stage-1 policy of Table II ; every other row changes one decision of Section III-B . SR and clearance as in Table II .

Explore similar work

Jul 6, 2026cs.CV

From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model

Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.

Wenhao Li, Xueying Jiang, Quanhao Qian +4

1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1

Diffusion-Based Vision-Language-Actions Camera End-Effector Pose

Ask PDF

Jul 13, 2026cs.RO

See like a Robot: Robot-Centric Pointmaps for VLA Models

Vision-language-action (VLA) models require 3D spatial reasoning, yet RGB observations encode robot-object geometry only implicitly. Lifting depth with camera intrinsics makes this geometry explicit as dense, image-aligned pointmaps, but their camera-frame coordinates depend on camera placement. We propose SeeR-VLA, which transforms pointmaps into a robot-centric frame with an end-effector origin and robot-base-aligned axes. An encoder initialized from pretrained RGB weights extracts pointmap features, which are added to corresponding RGB tokens without increasing the token count. Across 24 RoboCasa tasks and four real-world tasks, SeeR-VLA improves average success over RGB-only

π_{0.5}

by 6.4 and 32.5 percentage points, respectively. It also exceeds the strongest evaluated 3D-augmented baseline, PointVLA, by 3.5 and 23.7 percentage points, respectively. Beyond these gains, our ablations clarify how coordinate choices affect VLA performance, showing that end-effector centering is most effective with robot-base-aligned axes. The benefits grow as training viewpoints diversify, highlighting the importance of using robot-frame pointmaps when learning from diverse camera configurations.

Byungkun Lee, Dongyoon Hwang, Dongjin Kim +4

KAIST AI · Holiday Robotics

Robot Systems

Ask PDF

May 26, 2026cs.RO

FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies

Vision-Language-Action (VLA) models are increasingly expected to not only complete robot tasks, but also follow human instructions about how those tasks should be executed. However, existing robot datasets usually pair trajectories with coarse goal-level language, leaving execution-critical details such as active arm, approach direction, and contact region unspecified. This limits steerable policy learning and robotic video understanding. We introduce FineVLA, an open framework for action-aligned fine-grained VLA supervision. The framework includes: (1) a data construction tool that unifies 972,247 trajectories across 85K tasks from 10 open-source robot datasets and builds FineVLA-Data, a human-verified dataset of 47,159 fine-grained trajectories; (2) a held-out benchmark with 500 videos, 11,631 atomic facts, and 1,030 VQA questions; (3) a robotics-specialized VLM annotator for scalable fine-grained annotation; and (4) a steerable VLA policy trained with controlled mixtures of fine-grained and raw goal-level instructions. Our experiments yield three findings. First, fine-grained supervision does not sacrifice goal-level success: FG-only improves over Raw-only by +1.4 to +8.1 success-rate points across settings. Second, fine-grained and raw instructions are complementary, following a consistent inverted-U trend peaking at FG:Raw = 1:2 to 1:1. The best mixed setting reaches 86.8%/82.5% in RoboTwin simulation and 62.7/100 in real-world dual-arm manipulation (vs. 49.9 Raw-only). Third, fine-grained supervision improves steerable control: the largest real-world gains appear on pose (+23), color (+18), and approach direction (+18)--factors where goal-level instructions provide no guidance. Overall, fine-grained language should augment goal-level instructions: specifying how to execute alongside what to achieve. Project page: https://finevla.xlang.ai/

Xintong Hu, Xuhong Huang, Jinyu Zhang +11

XLANG Lab, The University of Hong Kong · Qwen Team, Alibaba Inc.

Topic pending

Ask PDF