Many everyday objects are difficult to directly grasp (e.g., a flat iPad) or manipulate functionally (e.g., opening the cap of a pen lying on a desk). Such tasks require sequential, asymmetric coordination between two arms, where one arm performs preparatory manipulation that enables the other's goal-directed action - for instance, pushing the iPad to the table's edge before picking it up, or lifting the pen body to allow the other hand to remove its cap. In this work, we introduce Collaborative Preparatory Manipulation, a class of bimanual manipulation tasks that demand understanding object semantics and geometry, anticipating spatial relationships, and planning long-horizon coordinated actions between the two arms. To tackle this challenge, we propose a visual affordance-based framework that first envisions the final goal-directed action and then guides one arm to perform a sequence of preparatory manipulations that facilitate the other arm's subsequent operation. This affordance-centric representation enables anticipatory inter-arm reasoning and coordination, generalizing effectively across various objects spanning diverse categories. Extensive experiments in both simulation and the real world demonstrate that our approach substantially improves task success rates and generalization compared to competitive baselines.
Figures & tables
Figure 2: Overview of the BiPreManip framework for bimanual preparatory manipulation tasks. (a) The system predicts an anticipatory affordance map to infer the goal-directed interaction of the primary arm. (b) Guided by this prediction, the assistant arm performs preparatory actions, establishing favorable conditions for manipulation. (c) The primary arm then executes the goal-directed manipulation. This anticipatory reasoning enables effective and coordinated dual-arm manipulation.
Figure 3: The BiPreManip pipeline. Given the point cloud observation and a language instruction, the Goal Affordance Network predicts an anticipatory affordance for the primary arm. Conditioned on this, the Pre-Affordance Network infers how the assistant arm should act to establish favorable object conditions. The Anticipatory Object Pose Predictor and Reorient Actor estimate and execute the object reconfiguration required for collision-free access. Finally, the Goal Affordance Network is re-invoked on the updated scene to execute the goal-directed manipulation. This framework design enables anticipatory, collaborative, and geometrically consistent bimanual reasoning.
(1) Edge-Pushing
Method
Bowl
Cap
Keyboard
Laptop
Phone
Remote
Scissors
Switch
Window
W2A
0% / 0%
2% / 4%
0% / 0%
5% / 10%
2% / 0%
0% / 0%
5% / 5%
0% / 0%
0% / 0%
ACT
32% / 27%
22% / 36%
2% / 1%
7% / 3%
0% / 0%
1% / 0%
0% / 0%
1% / 1%
1% / 1%
3DA
0% / 0%
0% / 0%
0% / 0%
1% / 2%
2% / 0%
0% / 0%
0% / 0%
0% / 0%
1% / 0%
3DFA
3% / 0%
5% / 14%
1% / 3%
0% / 2%
1% / 0%
0% / 0%
0% / 3%
0% / 1%
8% / 9%
Heuristic
15% / 21%
31% / 37%
58% / 62%
35% / 33%
30% / 21%
20% / 23%
15% / 30%
20% / 23%
56% / 56%
Table 1: Quantitative comparison of task success rates across different object categories and methods in simulation. For each category, results are reported on training (before slash) and unseen (after slash) objects.
Figure 4: Simulation experiments. Qualitative results of the predicted affordance maps and corresponding actions from our method. In each example, the second column presents the predicted affordances: the top subfigure depicts the anticipatory affordance for the primary arm, while the bottom subfigure shows the pre-manipulation affordance for the assistant arm.
Method
Bottle
Dispenser
Lighter
Pen-button
Pen-cap
Pliers
Stapler
USB
w/o Ant-Aff
27% / 13%
39% / 48%
39% / 48%
48% / 58%
23% / 10%
8% / 12%
21% / 23%
3% / 3%
w/o ObjPosePred
24% / 15%
38% / 45%
31% / 52%
51% / 50%
21% / 8%
20% / 31%
14% / 14%
10% / 6%
ours
30% / 26%
45 % / 56%
43% / 58%
67% / 72%
26% / 32%
25% / 29%
38% / 30%
13% / 14%
Table 2: Ablation study results on the Articulated Manipulation Tasks.
Figure 5: Real-world results across different objects and task types.
Edge
Articulated
Handover
Method
Book
Hat
Bottle
Dispenser
Bowl
Bottle
W2A
0/10
1/10
0/10
1/10
0/10
0/10
3DFA
1/10
0/10
0/10
2/10
0/10
2/10
ours
7/10
8/10
6/10
8/10
5/10
6/10
Table 3: Real-world success rate in different tasks.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Simulation experiments. We present additional qualitative visualizations of the predicted affordance maps and corresponding actions produced by our method. In each example, the second column displays the predicted affordances: the top subfigure depicts the anticipatory affordance for the primary arm, while the bottom subfigure shows the pre-manipulation affordance for the assistant arm.
Task Type
Category
Train
Unseen
Total
Edge-Pushing Tasks
Bowl
140
46
186
Cap
42
14
56
Keyboard
49
16
65
Laptop
42
13
55
Phone
14
4
18
Remote
37
12
49
Appendix
Table 4: Number of object instances per category in the training and unseen test sets.
Figure 8: An example where the functional part is occluded in the initial observation.
Figure 9: Representative failure cases. These examples illustrate scenarios in which the robot struggles to determine appropriate actions, reflecting the inherent difficulty of the tasks. The first column shows the input observations, and the second column presents the resulting failures.
Department of Computer Science, Technical University of Darmstadt · Hessian Center for Artificial Intelligence (Hessian.AI), Darmstadt · Robotics Institute Germany (RIG) +2