cs.ROSep 13, 2026

VLBiMan++: Expanding the Generalization Boundary of Vision-Language Anchored One-Shot Bimanual Manipulation

Authors: Huayi Zhou, Wei Gao, Yiyang Han, Kui Jia, Hui Huang

Organizations: Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, College of Computer Science and Software Engineering, Shenzhen University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generalizable bimanual robotic manipulation requires a reusable task prior that can persist across increasingly diverse tasks, objects, scenes, embodiments, and execution conditions, thus avoiding the prohibitive cost of large-scale teleoperated demonstrations and policy retraining. In this work, we present VLBiMan++, an extended framework that expands the generalization boundary of vision-language anchored one-shot bimanual manipulation. Starting from a single human demonstration, VLBiMan++ performs task-aware decomposition to identify reusable and adaptable skill components, and employs vision-language grounded geometric adaptation to transfer these skills to novel configurations without retraining. Building on this foundation, we systematically extend generalization along five dimensions: task generalization through diverse and long-horizon skill compositions; object generalization across unseen categories, varying geometries, and more complex articulated or deformable objects; scene generalization under clutter, occlusion, and dynamic interference; embodiment generalization across heterogeneous dual-arm robotic platforms; and deployment generalization through prolonged closed-loop execution under repeated external perturbations. To support this broader scope, we further introduce object-state-aware adaptation and lightweight trajectory optimization mechanisms that accommodate changes beyond simple rigid 6-DoF pose variations while preserving reliable bimanual coordination. Extensive real-world experiments demonstrate that VLBiMan++ maintains strong task success and adaptation capability across these increasingly challenging settings. Overall, VLBiMan++ advances one-shot bimanual manipulation from demonstrating isolated transferability toward a more systematic and scalable framework for generalization across tasks, objects, scenes, embodiments, and long-term deployment conditions.

Figures & tables

Fig. 1: Left : Taking pouring water as an example, we sketch the entire process of VLBiMan++ based on the one-shot demonstration. Right : Diagram illustration of the proposed Generalization Pyramid : Task →\rightarrow Object →\rightarrow Scene →\rightarrow Embodiment →\rightarrow Deployment . VLBiMan++ aims to investigate how far a single human demonstration can serve as a reusable manipulation prior across increasingly diverse tasks, objects, scenes, embodiments, and execution conditions.
Fig. 2: Framework of the expanded V ision- L anguage Anchored Bi manual Man ipulation ( VLBiMan++ ). Taking the pouring water as an example, the paradigm consists of three stages ( e.g. , decomposition, adaptation, and composition) based on a given demonstration. VLBiMan++ can achieve generalization of unseen spatial placements, category-level new instances, non-rigid object state variations, and heterogeneous novel embodiments under the same task.
Fig. 3: Illustrations of representative points for target objects in four tasks including pouring , reorient+unscrew , tool-use:spoon , and zip penbag . The first three tasks involve only rigid objects, while the last task involves an articulated object. These anchor points will be used to calculate the change in object position and orientation (not always required).
 pi=(xi−xˉ)ax+(yi−yˉ)ay,∀i∈[1,N]p_{i}=(x_{i}-\bar{x})a_{x}+(y_{i}-\bar{y})a_{y},\hskip 9.24994pt\forall i\in[1,N] 
Algorithm 1 Orientation Estimation for Rigid Objects
Fig. 4: Illustrations of extracted various anchor points for deformable objects in two tasks including coil cable (top) and fold towel (bottom). These can serve as a supplement to Alg. 2 . We present the computational results for four test cases of each task. In addition, we also present intermediate results for handling linear or rectangular deformable objects.
 primary bimanual tasks/skillslong-horizon multi-stage tasksnew rearrangement tasksnew non-rigid targets
Task Types & Namesplugpeninsertingunscrewpouringpressinghandoverreorient +unscrewunscrew +pouringtool-use scooptool-use funnelflattingreorientflippinguprightplace bottle_mugplace fork_spoonzip penbagcoil cablefold towel
Dual-Arm Platform(s)P1 ∣| P2P1 ∣| P2P1 ∣| P2P1 ∣| P2P1P1 ∣| P2P1P1 ∣| P2P1P1P2P2P2P2P2P2P2P2P2
Object Number2212211233111122211
Object Names /Statespen body +cappen +cupupward bottleupward bottle +mugcup+ pump bottlespoon or shovellying bottleupward bottle +mugspoon+ bowl_1+ bowl_2funnel +bottle +mugupside down bottlelying bottleupside down muglying mugbottle +mugfork+ spoonpenbag +penlinear cablerectan -gular towel
Object PropertiesArt.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Rig.Art.Def.Def.
Orientation Estimation?✓✓✓✗✗✗✗✗✗✓✓✗✗✓✗✗✓✗✗✗✓✗✓✓✓✓✓✓✓✗✗
TABLE I: Detailed statistics on all bimanual manipulation tasks, including 10 previous tasks and 9 newly added tasks. In the Dual-Arm Platform(s) , P1 and P2 represent the task has been instantiated on a contralateral robot and a humanoid robot, respectively. In the Object Properties , Rig., Art., and Def. represent rigid, articulated, and deformable objects, respectively. Among them, deformable objects require Alg. 2 to extract anchor points. And the Orientation Estimation is based on Alg. 1 .
Figure 7
   six primary bimanual tasks/skillsfour long-horizon multi-stage tasks
Test SettingDyna -mic Inter- ferenceManipulation MethodplugpeninsertingunscrewpouringpressinghandoverAverage Success Ratereorient +unscrewunscrew +pouringtool-use scooptool-use funnelAverage Success Rate
IDNoMechanisms11/2509/2505/2505/2507/2503/2526.7%05/2504/2502/2501/2512.0%
MAGIC16/2515/2510/2510/2509/2507/2544.7%09/2508/2504/2503/2524.0%
Robot-ABC14/2510/2509/2507/2508/2506/2536.0%06/2506/2503/2503/2518.0%
ReKep14/2511/2510/2512/2510/2508/2543.3%07/2508/2505/2503/2523.0%
ReKep+19/2518/2513/2517/2517/2511/2563.3%11/2510/2507/2506/2534.0%
TABLE II: Quantitative comparison results of success rates on six primary bimanual tasks/skills and four long-horizon multi-stage tasks under the in-distribution (ID) setting (new placements + same objects ) or the out-of-distribution (OOD) setting (new placements + novel instances ) using the contralateral dual-arm robot ( P1 ).
Fig. 7: Visualizations of ten tasks executed on the contralateral dual-arm robots platform P1 . They are designed to validate different aspects, including (a) six dual-arm primary skills, (b) combination of basic skills for two long-horizon tasks, and (c) exploration of two multi-stage tool-use tasks.
   six new rearrangement tasksthree new non-rigid targets
Test SettingDyna -mic Inter- ferenceManipulation Methodflattingreorientflippinguprightplace bottle_mugplace fork_spoonAverage Success Ratezip penbagcoil cablefold towelAverage Success Rate
IDNoMechanisms04/2504/2502/2501/2501/2501/258.7%00/2500/2502/252.7%
MAGIC06/2505/2503/2502/2501/2502/2512.7%00/2500/2503/254.0%
Robot-ABC07/2506/2506/2506/2505/2505/2523.3%01/2500/2505/258.0%
ReKep+13/2512/2510/2511/2508/2511/2543.3%04/2500/2508/2516.0%
VLBiMan++24/2522/2518/2520/2515/2523/2581.3%12/2510/2520/2556.0%
TABLE III: Quantitative comparison results of success rates on six new rearrangement tasks and three new non-rigid targets under the in-distribution (ID) setting (new placements + same objects ) or the out-of-distribution (OOD) setting (new placements + novel instances ) using the humanoid dual-arm robot ( P2 ). The ’—’ means there are no extra new object instances.
Fig. 8: Visualizations of newly added nine tasks executed on the humanoid-style dual-arm platform P2 . They are defined to further confirm the universality of VLBiMan++, including (a) four left-arm or right-arm rearrangement skills, (b) combination of new basic skills for two long-horizon rearrangement tasks, (c) a complex task involving articulated bodies, (d) two dual-arm tasks involving linear or rectangular deformable objects.
Fig. 9: Visualizations of VLBiMan++’s robustness performance in cluttered scenarios on the platform P2 . We selected three tasks for extensive evaluation: ( a ) reorient , ( b ) pouring , and ( c ) inserting . The red arrow indicates the direction of the manually moved or deliberately obscured object (interfering). The cyan arrow and yellow arrow indicate the movement direction of the left and right robotic arms (chasing) respectively.
Dynamic Interference ID setting: new placements + same objectsOOD setting: new placements + novel instances
Dual-Arm Platform Typeplugpeninsertingunscrewpouringhandoverunscrew +pouringAverage Success Rateplugpeninsertingunscrewpouringhandoverunscrew +pouringAverage Success Rate
NoContralateral19/2018/2016/2017/2015/2012/2080.8%15/2017/2014/2014/2014/2009/2069.2%
Humanoid20/2019/2017/2016/2015/2013/2083.3%16/2018/2016/2013/2014/2009/2071.7%
YesContralateral16/2013/2015/2015/2012/2008/2065.8%14/2011/2012/2011/2010/2007/2054.2%
Humanoid16/2014/2015/2014/2013/2009/2067.5%12/2012/2013/2012/2010/2008/2055.8%
TABLE IV: Quantitative results of VLBiMan++’s success rates on six transferred bimanual tasks from the contralateral dual-arm robot ( P1 ) to the humanoid dual-arm robot ( P2 ) under the ID and OOD settings.
Fig. 10: Visualizations of five cross-embodiment transferred primary bimaual tasks executed on the new humanoid-style robot ( P1 →\rightarrow P2 ).
 ID setting: new placements + same objectsOOD setting: new placements + novel instances
Manipulation MethodplugpeninsertingunscrewpouringpressinghandoverAverage Success RateplugpeninsertingunscrewpouringpressinghandoverAverage Success Rate
Mechanisms01/2001/2000/2001/2001/2000/203.3%00/2000/2000/2000/2000/2000/200.0%
MAGIC03/2004/2002/2002/2002/2001/2011.7%01/2002/2000/2001/2001/2000/204.2%
Robot-ABC02/2002/2001/2001/2001/2001/206.7%00/2000/2000/2000/2000/2000/200.0%
ReKep05/2003/2002/2001/2002/2002/2012.5%03/2001/2001/2001/2001/2000/205.8%
ReKep+08/2006/2004/2003/2004/2005/2025.0%05/2002/2003/2002/2003/2001/2013.3%
TABLE V: Quantitative comparison results of success rates on six primary bimanual skills/tasks using the contralateral dual-arm robot ( P1 ). All tasks faced the challenges of dynamic interference and uneven illumination .
Fig. 11: Video snapshots recorded from a third-person perspective for the super long-duration experiment of reorient on platform P2 . It contains keyframes from 20 trials in which the left arm ( cyan circle) or right arm ( yellow circle) was used to upright a bottle lying down in an arbitrary orientation.
Fig. 12: Examples of interesting findings about the closed-loop mechanism. Top row : this case comes from the pre-grasping phase of pouring , where the left arm approaches and grasps the bottle. Bottom row : this case comes from the pre-grasping phase of inserting , where the right arm approaches and grasps the marker from the top direction.
Fig. 13: Taking the inserting task as an example, we can replace the marker pen held by the left arm with other rectangular objects that were completely different ( e.g. , spoon in a1/a2 , brush in a3/a4 , spatula in a5 , syringe in a6 , and toothbrush in a7/a8 ) as in the top row , or change the cup grasped by the right arm to cups of different shapes ( e.g. , mugs with handles in b1/b2/b3/b4 , and ordinary cups without handles in b5/b6/b7/b8 ) as in the bottom row .
Inter- polation Density nnID settingOOD setting
insertingunscrewpouringreorientAverage Success RateinsertingunscrewpouringreorientAverage Success Rate
n=3n=315/2014/2012/2013/2067.5%13/2012/2011/2011/2058.8%
n=4n=418/2015/2015/2016/2080.0%17/2014/2014/2014/2073.8%
n=5n=519/2017/2018/2016/2087.5%18/2016/2017/2015/2082.5%
n=6n=619/2019/2018/2017/2091.3%19/2018/2017/2016/2087.5%
n=7n=719/2018/2018/2018/2092.5%18/2019/2017/2016/2087.5%
TABLE VI: Ablation experiments of the interpolation density nn . To ensure reliable searching of the optimal nn , we did not add any additional dynamic interference in each trail.
VLMs-based groundingstate- aware adaptationIK- refinementcollision avoidanceclosed-loop re-compositionAvg. SR
SAM+DINOv2Ours✓✓✓35.8%
OursAnyGrasp✓✓✓31.7%
OursOurs✗✓✓29.2%
OursOurs✓✗✓34.2%
OursOurs✓✓✗40.8%
OursOurs✓✓✓59.2%
TABLE VII: Ablation studies of VLBiMan++. All trials were completed on six primary tasks using the platform P1 ( ID + interference ).
Fig. 14: Error breakdown performed on VLBiMan++ failures based on the rollout results in Tab. II (covering OOD and interference part).
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
More in this topic

Bimanual Robotic Manipulation

Bimanual robotic manipulation coordinates two robot arms to handle objects, tools, and tasks that require collaboration.

See all 178 papers

Explore similar work

CardsList
Jun 11, 2026

See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation

In bimanual robotic manipulation, task-relevant visual information varies with the task stage and context, while the interaction of the two arms shifts between independent and coordinated modes, making policy learning challenging. However, existing monolithic VLA policies typically process multi-view observations and distinct bimanual interaction patterns without explicitly adapting their visual conditioning or action-generation structure. To address this issue, we propose a bimanual manipulation VLA framework based on Dual-Level Structural Decomposition. The View-Selective Visual Router dynamically adjusts wrist-view contributions to emphasize relevant visual cues, while the Interaction-Aware Action Mixture-of-Experts decomposes action generation into coordinated and arm-wise pathways to adapt to varying bimanual interaction modes. We evaluate the proposed method on six simulated bimanual manipulation tasks in RoboTwin 2.0 and three long-horizon real-world tasks. Our model improves the overall average success rate over a monolithic baseline by 27.7% in simulation and 43.3% in real-world evaluation, while consistently outperforming single-module variants across both settings. These results demonstrate that jointly considering selective visual processing and explicit decomposition of bimanual interaction structures provides an effective inductive bias for robust bimanual manipulation.

Yoon-Ji Choi, Young-Chae Son, Soo-Chul Lim

Dongguk University

Long-Horizon Robotic ManipulationBimanual Robotic ManipulationMixture of ExpertsVLMs for Robotics
AskPDF
Apr 9, 2026

Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning

Bimanual manipulation requires reasoning about where to interact with an object and which arm should perform each action, a joint affordance localization and arm allocation problem that geometry-only planners cannot resolve without semantic understanding of task intent. Existing approaches either treat affordance prediction as coarse part segmentation or rely on geometric heuristics for arm assignment, failing to jointly reason about task-relevant contact regions and arm allocation. We reframe bimanual manipulation as a joint affordance localization and arm allocation problem and propose a hierarchical framework for task-aware bimanual affordance prediction that leverages a Vision-Language Model (VLM) to generalize across object categories and task descriptions without requiring category-specific training. Our approach fuses multi-view RGB-D observations into a consistent 3D scene representation and generates global 6-DoF grasp candidates, which are then spatially and semantically filtered by querying the VLM for task-relevant affordance regions on each object, as well as for arm allocation to the individual objects, thereby ensuring geometric validity while respecting task semantics. We evaluate our method on a dual-arm platform across nine real-world manipulation tasks spanning four categories: parallel manipulation, coordinated stabilization, tool use, and human handover. Our approach achieves consistently higher task success rates than geometric and semantic baselines for task-oriented grasping, demonstrating that explicit semantic reasoning over affordances and arm allocation helps enable reliable bimanual manipulation in unstructured environments.

Fabian Hahne, Vignesh Prasad, Georgia Chalvatzaki +2

Department of Computer Science, Technical University of Darmstadt · Hessian Center for Artificial Intelligence (Hessian.AI), Darmstadt · Robotics Institute Germany (RIG) +2

Bimanual Robotic ManipulationRobotic ManipulationRobotic Grasping
AskPDF
Apr 22, 2026

Bimanual Robot Manipulation via Multi-Agent In-Context Learning

Large Language Models (LLMs) have emerged as powerful reasoning engines for embodied control. In particular, In-Context Learning (ICL) enables off-the-shelf, text-only LLMs to predict robot actions without any task-specific training while preserving their generalization capabilities. Applying ICL to bimanual manipulation remains challenging as the high-dimensional joint action space and tight inter-arm coordination constraints rapidly overwhelm standard context windows. To address this, we introduce BiCICLe (Bimanual Coordinated In-Context Learning), the first framework that enables standard LLMs to perform few-shot bimanual manipulation without fine-tuning. BiCICLe frames bimanual control as a multi-agent leader-follower problem, decoupling the action space into sequential, conditioned single-arm predictions. Evaluated on 13 tasks from the TWIN benchmark, BiCICLe achieves 70.5% average success rate, outperforming the best training-free baseline by 6.1 percentage points and surpassing most supervised methods. We also demonstrate superior real-world performance on 3 tasks without hardware-specific retraining. The project page is available at https://alesspalma.github.io/bicicle

Alessio Palma, Indro Spinelli, Vignesh Prasad +4

Sapienza University of Rome, Italy · TU Darmstadt, Germany · Hessian.AI, Germany

ICL for RoboticsLanguage Model-Based ControlBimanual Robotic ManipulationRobot Learning
AskPDF