Robots manipulating objects in clutter must decide not only how to act, but also where to look. Although active camera motion can expose an occluded target, it may move the robot into an initial observation-state distribution that differs from the manipulation policy's training distribution. We identify this problem as first-frame distribution sensitivity: under perturbed starts in LIBERO, naively prepending camera repositioning to pi_0.5 reduces success from 46.3% to 11.5%. We introduce ActiveAlign, a data-adaptation framework that trains a Vision-Language-Action (VLA) policy from the states at which active perception terminates. A learned viewpoint module first generates a camera trajectory. Offline, ActiveAlign uses closed-loop alignment to connect the trajectory endpoint to an existing demonstration, replays the remaining manipulation trajectory, and retains successful continuations for fine-tuning. At deployment, the same viewpoint module precedes the adapted VLA without modifying its architecture. Across 40 LIBERO tasks, ActiveAlign achieves 70.7% success with pi_0 and 72.7% with pi_0.5, outperforming their standard-training baselines by 40.0% and 26.4% respectively. Matched-state and data-matched controls support gains beyond prefix shortening and generic viewpoint augmentation; OpenVLA and GR00T N1 also show the same trend. Success rate under physical occlusion in real world increases from 35/90 to 83/90. These results show that active perception benefits manipulation when policy training is aligned with the observation and robot-state distribution induced by viewpoint selection.
Figures & tables
Fig. 1: ActiveAlign overview. Wrist-camera repositioning changes the view and robot state, potentially degrading an unadapted VLA. ActiveAlign fine-tunes the policy on successful trajectories beginning at active endpoints. The distribution sketches illustrate the intended alignment.
Fig. 2: ActiveAlign pipeline. Left: RGBD and language condition a camera trajectory. Center: prefix matching, alignment, and successful replay produce adapted demonstrations. Right: VLA fine-tuning and deployment, with one viewpoint-selection phase followed by manipulation from fresh observations. Physical motion is continuous; IK repositioning is simulation-only. Distribution sketches are schematic.
VLA
Setting
Spatial
Object
Goal
Long
Avg.
π0
Standard
34.0
32.2
28.4
28.0
30.7
Naive Active
6.6
0.0
13.6
7.6
7.0
ActiveAlign
73.8
78.2
66.6
64.2
70.7
π0.5
Standard
48.0
39.0
49.4
48.8
46.3
Naive Active
22.0
2.2
18.6
3.0
11.5
ActiveAlign
78.2
78.0
68.8
65.8
72.7
TABLE I: Perturbed-start LIBERO success rate(%).
Backbone
Standard
Naive Active
ActiveAlign
π0
30.7
7.0
70.7
π0.5
46.3
11.5
72.7
OpenVLA
56.4
24.8
69.6
GR00T N1
51.8
21.6
67.8
TABLE II: Backbone transfer on LIBERO (suite-average success rate, %).
Start noise σ (m/rad)
Benchmark init
0.02
0.05
0.10
Standard
96.8
85.2
70.4
46.3
Naive Active
48.0
38.2
23.1
11.5
ActiveAlign
96.6
90.4
83.0
72.7
ActiveAlign − Standard
− 0.2
+5.2
+12.6
+26.4
TABLE III: Start-state shift sweep on LIBERO with π0.5 (suite-average success rate, %).
Variant
Success (%)
Standard
48.0
Naive Active
22.0
Matched-state control
54.6
Random-view adaptation
58.0
Broad viewpoint jitter
61.4
ActiveAlign
78.2
TABLE IV: Viewpoint and data controls on LIBERO Spatial with π0.5 .
Method
Spatial
Object
Goal
Long
Standard
20.0
22.6
21.4
7.8
Naive Active
5.8
2.2
8.6
0.2
ActiveAlign
48.6
47.2
43.8
42.4
TABLE V: Initial-occlusion success (%) on LIBERO-Occ with π0.5 .
Fig. 4: First-frame features. Shared t-SNE embedding of CLIP ViT-H/14 features: red circles, original data; blue squares, adapted data; green triangles, inference after AP. Feature overlap is a qualitative illustration.
Task
Primary variation
Standard
Naive Active
ActiveAlign
Grasp a cup on a cluttered tabletop
Target identity and clutter
9/20 [25.8,65.8]
4/20 [8.1,41.6]
17/20 [64.0,94.8]
Retrieve puzzles from a container
Object and container geometry
6/20 [14.5,51.9]
3/20 [5.2,36.0]
15/20 [53.1,88.8]
Observe, open, grasp, and place
Longer manipulation horizon
5/20 [11.2,46.9]
3/20 [5.2,36.0]
13/20 [43.3,81.9]
TABLE VII: Physical task diversity. Entries are successes/trials followed by Wilson 95% confidence intervals in percentage points.
Occlusion
Standard
Naive Active
ActiveAlign
Partial
16/30 (53.3)
8/30 (26.7)
29/30 (96.7)
Medium
11/30 (36.7)
5/30 (16.7)
28/30 (93.3)
Severe
8/30 (26.7)
4/30 (13.3)
26/30 (86.7)
Overall
35/90 (38.9)
17/90 (18.9)
83/90 (92.2)
TABLE VIII: Physical occlusion success with π0.5 . Entries are successes/trials (rate in %).
Fig. 5: Physical rollouts with π0.5 + ActiveAlign. > Rows show different tasks at varying occlusion levels, with active observation preceding manipulation.
Active perception and manipulation are crucial for robots to interact with complex scenes. Existing methods struggle to unify semantic-driven active perception with robust, viewpoint-invariant execution. We propose SaPaVe, an end-to-end framework that jointly learns these capabilities in a data-efficient manner. Our approach decouples camera and manipulation actions rather than placing them in a shared action space, and follows a bottom-up training strategy: we first train semantic camera control on a large-scale dataset, then jointly optimize both action types using hybrid data. To support this framework, we introduce ActiveViewPose-200K, a dataset of 200k image-language-camera movement pairs for semantic camera movement learning, and a 3D geometry-aware module that improves execution robustness under dynamic viewpoints. We also present ActiveManip-Bench, the first benchmark for evaluating active manipulation beyond fixed-view settings. Extensive experiments in both simulation and real-world environments show that SaPaVe outperforms recent vision-language-action models such as GR00T N1 and π0, achieving up to 31.25% higher success rates in real-world tasks. These results show that tightly coupled perception and execution, when trained with decoupled yet coordinated strategies, enable efficient and generalizable active manipulation. Project page: https://lmzpai.github.io/SaPaVe
Mengzhen Liu, Enshen Zhou, Cheng Chi +6
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Academy of Artificial Intelligence · School of Software, Beihang University
Active perception is essential for robotic manipulation when fixed viewpoints leave task-relevant information occluded or unobserved. However, enabling vision-language-action (VLA) models to reason across changing viewpoints and actively acquire informative observations remains challenging. We present ActiveScale, a framework that advances active perception through coordinated model, data, and hardware designs. Our model augments a VLA with historical video observations and explicit camera-pose supervision, using per-frame pose tokens and a lightweight prediction head to associate observations across viewpoints and support a coherent understanding of the scene. To learn from the camera motion naturally present in human activity, we introduce a scalable human--robot mid-training recipe using 1000 hours of egocentric and robotic data, adapting the model to temporal inputs and pose supervision. We further introduce Active-perception Mobile-manipulation Platform (AMP), a robotic platform that supports active perception and mobile manipulation through single-operator teleoperation, enabling scalable collection of demonstrations that coordinate viewpoint changes and manipulation. Experiments demonstrate improved success rates on active-perception tasks, while ablation studies validate the contributions of camera-pose-aware modeling and egocentric mid-training. Together, these components provide an integrated foundation for studying and developing active perception in robotic manipulation.
Shuai Zhou, Kaisheng Pang, Wenxuan Song +3
Robotics Institute, Carnegie Mellon University · The Hong Kong University of Science and Technology (Guangzhou)
Active vision manipulation requires a policy to control both its camera and its end-effectors, yet camera motion determines which evidence remains visible within finite observation windows. Acquiring a new view can displace task-critical cues, while retaining a view forgoes potentially useful observations. We formulate this as an evidence-aware retain--acquire problem and present ActiveWAM, a unified world--action model that learns observation and manipulation jointly. To this end, we propose training-time inversion which constrains a frozen video prior by task-bearing source evidence and visible temporal changes, eliminating the need for test-time inversion or candidate ranking. At deployment, the policy generates bimanual and pan/tilt actions-including stay and reacquisition behaviors-from view-aware history, and updates context from newly measured RGB observations. Future-video prediction serves as a co-training signal, while action generation requires neither future-video decoding nor optimal viewpoint annotations. We introduce RoboTwin-AV, a 50-task benchmark with executable pan/tilt control and automatically generated demonstrations. ActiveWAM improves TAVIS out-of-distribution success by up to 17.0 percentage points over the strongest baselines, achieves 20.0 additional points over Fast-WAM on RoboTwin-AV, and outperforms it by 26.7 points on real-world physical kitchen tasks.