Look Before You Act: Bridging Active Perception and Vision-Language-Action Models via Observation-Aligned Data Adaptation

Jiawei Geng; Zhenyang Liu; Yuzheng Wu; Yongchong Gu; Yanwei Fu

cs.ROOct 16, 2026

Look Before You Act: Bridging Active Perception and Vision-Language-Action Models via Observation-Aligned Data Adaptation

Authors: Jiawei Geng, Zhenyang Liu, Yuzheng Wu, Yongchong Gu, Yanwei Fu

Organizations: Fudan University · Shanghai Innovation Institute

Abstract

Robots manipulating objects in clutter must decide not only how to act, but also where to look. Although active camera motion can expose an occluded target, it may move the robot into an initial observation-state distribution that differs from the manipulation policy's training distribution. We identify this problem as first-frame distribution sensitivity: under perturbed starts in LIBERO, naively prepending camera repositioning to pi_0.5 reduces success from 46.3% to 11.5%. We introduce ActiveAlign, a data-adaptation framework that trains a Vision-Language-Action (VLA) policy from the states at which active perception terminates. A learned viewpoint module first generates a camera trajectory. Offline, ActiveAlign uses closed-loop alignment to connect the trajectory endpoint to an existing demonstration, replays the remaining manipulation trajectory, and retains successful continuations for fine-tuning. At deployment, the same viewpoint module precedes the adapted VLA without modifying its architecture. Across 40 LIBERO tasks, ActiveAlign achieves 70.7% success with pi_0 and 72.7% with pi_0.5, outperforming their standard-training baselines by 40.0% and 26.4% respectively. Matched-state and data-matched controls support gains beyond prefix shortening and generic viewpoint augmentation; OpenVLA and GR00T N1 also show the same trend. Success rate under physical occlusion in real world increases from 35/90 to 83/90. These results show that active perception benefits manipulation when policy training is aligned with the observation and robot-state distribution induced by viewpoint selection.

Figures & tables

Fig. 1: ActiveAlign overview. Wrist-camera repositioning changes the view and robot state, potentially degrading an unadapted VLA. ActiveAlign fine-tunes the policy on successful trajectories beginning at active endpoints. The distribution sketches illustrate the intended alignment.

Fig. 2: ActiveAlign pipeline. Left: RGBD and language condition a camera trajectory. Center: prefix matching, alignment, and successful replay produce adapted demonstrations. Right: VLA fine-tuning and deployment, with one viewpoint-selection phase followed by manipulation from fresh observations. Physical motion is continuous; IK repositioning is simulation-only. Distribution sketches are schematic.

VLA	Setting	Spatial	Object	Goal	Long	Avg.
$\pi_{0}$	Standard	34.0	32.2	28.4	28.0	30.7
	Naive Active	6.6	0.0	13.6	7.6	7.0
	ActiveAlign	73.8	78.2	66.6	64.2	70.7
$\pi_{0.5}$	Standard	48.0	39.0	49.4	48.8	46.3
	Naive Active	22.0	2.2	18.6	3.0	11.5
	ActiveAlign	78.2	78.0	68.8	65.8	72.7

TABLE I: Perturbed-start LIBERO success rate(%).

Backbone	Standard	Naive Active	ActiveAlign
$\pi_{0}$	30.7	7.0	70.7
$\pi_{0.5}$	46.3	11.5	72.7
OpenVLA	56.4	24.8	69.6
GR00T N1	51.8	21.6	67.8

TABLE II: Backbone transfer on LIBERO (suite-average success rate, %).

Start noise $\sigma$ (m/rad)	Benchmark init	0.02	0.05	0.10
Standard	96.8	85.2	70.4	46.3
Naive Active	48.0	38.2	23.1	11.5
ActiveAlign	96.6	90.4	83.0	72.7
ActiveAlign $-$ Standard	$-$ 0.2	+5.2	+12.6	+26.4

TABLE III: Start-state shift sweep on LIBERO with

\pi_{0.5}

(suite-average success rate, %).

Variant	Success (%)
Standard	48.0
Naive Active	22.0
Matched-state control	54.6
Random-view adaptation	58.0
Broad viewpoint jitter	61.4
ActiveAlign	78.2

TABLE IV: Viewpoint and data controls on LIBERO Spatial with

\pi_{0.5}

Method	Spatial	Object	Goal	Long
Standard	20.0	22.6	21.4	7.8
Naive Active	5.8	2.2	8.6	0.2
ActiveAlign	48.6	47.2	43.8	42.4

TABLE V: Initial-occlusion success (%) on LIBERO-Occ with

\pi_{0.5}

Fig. 3: Perception trajectories. Top: LIBERO. Bottom: perception-only zero-shot RLBench transfer. The examples illustrate view selection; end-to-end success is evaluated separately.

Fig. 4: First-frame features. Shared t-SNE embedding of CLIP ViT-H/14 features: red circles, original data; blue squares, adapted data; green triangles, inference after AP. Feature overlap is a qualitative illustration.

Task	Primary variation	Standard	Naive Active	ActiveAlign
Grasp a cup on a cluttered tabletop	Target identity and clutter	9/20 [25.8,65.8]	4/20 [8.1,41.6]	17/20 [64.0,94.8]
Retrieve puzzles from a container	Object and container geometry	6/20 [14.5,51.9]	3/20 [5.2,36.0]	15/20 [53.1,88.8]
Observe, open, grasp, and place	Longer manipulation horizon	5/20 [11.2,46.9]	3/20 [5.2,36.0]	13/20 [43.3,81.9]

TABLE VII: Physical task diversity. Entries are successes/trials followed by Wilson 95% confidence intervals in percentage points.

Occlusion	Standard	Naive Active	ActiveAlign
Partial	16/30 (53.3)	8/30 (26.7)	29/30 (96.7)
Medium	11/30 (36.7)	5/30 (16.7)	28/30 (93.3)
Severe	8/30 (26.7)	4/30 (13.3)	26/30 (86.7)
Overall	35/90 (38.9)	17/90 (18.9)	83/90 (92.2)

TABLE VIII: Physical occlusion success with

\pi_{0.5}

. Entries are successes/trials (rate in %).

Fig. 5: Physical rollouts with $\pi_{0.5}$ + ActiveAlign. > Rows show different tasks at varying occlusion levels, with active observation preceding manipulation.

Active Vision for Robot Manipulation

Active vision for robot manipulation studies how robots move cameras, gaze, or objects to gather useful visual evidence while performing tasks.

See all 34 papers

SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics
Mar 12, 2026Mengzhen Liu, Enshen Zhou, Cheng Chi +6Active Vision for Robot Manipulation Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence · School of Software, Beihang University
ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardware
Sep 16, 2026Shuai Zhou, Kaisheng Pang, Wenxuan Song +3Active Vision for Robot Manipulation Robotic Manipulation
Robotics Institute, Carnegie Mellon University · The Hong Kong University of Science and Technology (Guangzhou)
ActiveWAM: Evidence-Aware Active Vision for World-Action Models
Oct 1, 2026Renjun Wu, Luzhou Ge, Xuesong LiWorld Action Models Multi-Task Robotic Manipulation
Beijing Institute of Technology

Mar 12, 2026J/K move · Enter open · S save

SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics

Mengzhen Liu, Enshen Zhou, Cheng Chi, Yi Han+5

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence · School of Software, Beihang University

Active Vision for Robot Manipulation Robotic Manipulation Vision-Language-Action Models Data-Efficient Robot Learning Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Active perception and manipulation are crucial for robots to interact with complex scenes. Existing methods struggle to unify semantic-driven active perception with robust, viewpoint-invariant execution. We propose SaPaVe, an end-to-end framework that jointly learns these capabilities in a data-efficient manner. Our approach decouples camera and manipulation actions rather than placing them in a shared action space, and follows a bottom-up training strategy: we first train semantic camera control on a large-scale dataset, then jointly optimize both action types using hybrid data. To support this framework, we introduce ActiveViewPose-200K, a dataset of 200k image-language-camera movement pairs for semantic camera movement learning, and a 3D geometry-aware module that improves execution robustness under dynamic viewpoints. We also present ActiveManip-Bench, the first benchmark for evaluating active manipulation beyond fixed-view settings. Extensive experiments in both simulation and real-world environments show that SaPaVe outperforms recent vision-language-action models such as GR00T N1 and

π_0

, achieving up to 31.25% higher success rates in real-world tasks. These results show that tightly coupled perception and execution, when trained with decoupled yet coordinated strategies, enable efficient and generalizable active manipulation. Project page: https://lmzpai.github.io/SaPaVe

Abstract

Active Vision for Robot Manipulation

Explore similar work

SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics

ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardware

ActiveWAM: Evidence-Aware Active Vision for World-Action Models