Recovering the View: Benchmarking Physical Active Vision for Occlusion Recovery in Robotic Manipulation
Organizations: Sun Yat-sen University · University of Electronic Science and Technology of China · X-Era AI Lab · Pengcheng Laboratory
Abstract
Physical active vision allows robots to change their viewpoint when task-relevant observations become unreliable, yet existing manipulation benchmarks provide limited support for studying how policies recover from occlusion during execution. We introduce BAVO-Bench (Bimanual Active Vision under Occlusion), a bimanual active-vision benchmark that systematically controls external visibility through Clean, Stage Occlusion, and Random-time Occlusion conditions, enabling evaluation of both manipulation performance and active visual recovery. Building on this setting, we present A-FAR (Active Future-Aware Recovery), an active-vision policy for joint viewpoint and manipulation control. A-FAR represents moving-camera observations in a unified robot-centric 3D frame and distills relational structure together with its future evolution from a pretrained 4D model, providing the policy with future-aware geometric guidance without requiring future observations at deployment. Experiments across multiple manipulation tasks show that A-FAR improves robustness to both structured and temporally shifted occlusions while maintaining strong performance under clean observations.
Figures & tables
| Clean | Stage Occlusion | Random-time Occlusion | ||||
| Method | SR (%) | TPS (%) | SR (%) | TPS (%) | SR (%) | TPS (%) |
| ACT | ||||||
| 13.0 | 55.2 | 42.0 | 73.4 | 4.0 | 40.8 | |
| ManiFlow | ||||||
| A-FAR | ||||||
| Task | ACT | ManiFlow | A-FAR | |
|---|---|---|---|---|
| Cube | ||||
| Drawer | ||||
| Slide | ||||
| Stack | ||||
| Sweep | ||||
| Avg. |
| Variant | C | S | R | Avg. |
| ManiFlow (base) | 44 | 60 | 15 | 41.0 |
| + Future Evo. | 47 | 64 | 16 | 42.3 |
| + Future Fusion | 47 | 63 | 18 | 43.0 |
| A-FAR (base) | 50 | 66 | 19 | 44.7 |
| ManiFlow (camera) | 26 | 44 | 14 | 28.0 |
| A-FAR (camera) | 22 | 43 | 16 | 27.0 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | Objective | Ordered milestones |
|---|---|---|
| Cube Handoff | Handoff a cube from the right arm to the left arm and place it on the target plate. | : the right gripper contacts the cube; : the right arm lifts the cube; : both grippers hold the cube during handoff; : the left gripper takes over and the right gripper releases; : the left arm places the cube on the target plate. |
| Block Stacking | Move the base block to the central workspace and stack the top block on it. | : the right gripper contacts the base block; : the base block is placed in the stacking region and released; : the left gripper contacts the top block; : the top block is lifted into the pre-stacking region; : a stable stack is formed with both grippers released. |
| Drawer Stowing | Open the drawer, stow an object inside, and close the drawer. | : the right gripper contacts the drawer handle; : the drawer is opened; : the left gripper contacts the object; : the object is placed and released inside the drawer; : the drawer is closed with the object remaining inside. |
| Slide Retrieval | Slide the object to expose its handle, then retrieve it with the left arm. | : the right gripper contacts the blade; : the handle is exposed by sliding; : the right gripper releases the object; : the left gripper contacts the exposed handle; : the left arm lifts the object from the support surface. |
| Dustpan Sweeping | Position the dustpan and sweep the target object into it. | : the left gripper contacts the dustpan handle; : the dustpan reaches a valid sweeping configuration; : the right gripper contacts the broom; : the broom contacts the target object during sweeping; : the target object reaches the dustpan entrance. |
| Task | Initial target | Transitions | Subsequent targets |
|---|---|---|---|
| Cube Handoff | cube | cube plate | |
| Block Stacking | base block | top block stacking region | |
| Drawer Stowing | drawer front | object drawer front | |
| Slide Retrieval | support platform | blade handle | |
| Dustpan Sweeping | dustpan | broom target object |
| Configuration | Value |
| Point-cloud size | 256 |
| Teacher query points | 256 |
| Observation horizon | 2 |
| Action horizon | 16 |
| Point feature dimension | 128 |
| State feature dimension | 64 |
| Task | Method | C | S | R |
|---|---|---|---|---|
| Cube Handoff | ACT | |||
| ManiFlow | ||||
| A-FAR | ||||
| Drawer Stowing | ACT | |||
| Task | Method | C | S | R |
|---|---|---|---|---|
| Cube Handoff | ACT | |||
| ManiFlow | ||||
| A-FAR | ||||
| Drawer Stowing | ACT | |||