TAO-DA: Towards Autonomous Operation--A Dual-Arm Vision-Language-Action Model for Coordinated Manipulation

Yongsheng Zhao; Han Gao; Baoping Cheng; Jingyao Tang; Dian Zhou; Deng Liang; Ji Ge; Xuanzhang Wen

cs.ROSep 27, 2026

TAO-DA: Towards Autonomous Operation--A Dual-Arm Vision-Language-Action Model for Coordinated Manipulation

Authors: Yongsheng Zhao, Han Gao, Baoping Cheng, Jingyao Tang, Dian Zhou, Deng Liang, Ji Ge, Xuanzhang Wen, +2 more

Organizations: China Mobile (Hangzhou) Information Technology Co., Ltd., Hangzhou, China

Abstract

Vision-Language-Action (VLA) models provide a unified framework for grounding high-level semantic information into low-level robot actions, enabling scalable robotic manipulation across diverse tasks. However, existing VLA models lack explicit mechanisms to disentangle the states and intents of the two arms, leading to unintended cross-arm interference that degrades task execution success. To address this issue, we propose a symmetric Dual-Arm Expert (DAE) architecture built upon a shared Vision-Language Model (VLM) backbone with decoupled, arm-specific expert towers. Expert selection is carried out through a two-stage dual-arm intent routing scheme, in which experts are routed either by explicit language instructions in the first stage or by implicit visual semantics in the second stage. Moreover, we introduce a lightweight task progress prediction module that leverages cross-attention between the pre-chunk temporal features and semantic representations of proprioceptive and visual observations to accurately estimate frame-wise task completion progress. This module facilitates task progress synchronization to support coordinated scheduling for collaborative multi-robot tasks. Experimental results demonstrate the effectiveness of our model in dual-arm intent routing and the disentanglement of cross-arm interference, and further provide preliminary evidence of emergent skill generalization from single- to dual-arm tasks (as well as the reverse), together with cross-arm motion-domain skill transfer.

Figures & tables

Fig. 1: Overview of TAO-DA. SigLIP encodes the multi-view observations, and Eagle-2.5-VL fuses visual and language tokens into the context used by DAIR. After routing, inactive end-effector-view tokens are masked at the VLM input when constructing the action-conditioning context. DAE then generates actions through the selected arm-specific pathways, trained with flow matching.

Fig. 2: Architectural variants of DAE. Single-Tower uses a unified state and action space, Dual-Mask introduces weak decoupling through selective token masking in a shared DiT, and Dual-Tower provides strong decoupling through two parameter-independent DiT branches. All variants support single-arm and dual-arm execution.

Category	Task	Mode	Traj.	Duration (h)
Single-arm	Steamer Delivery	Single	1,200	3.0
	Bottle Grasping	Single	880	3.2
	Fruit Grasping	Single	4,000	14.4
	Tea Pouring	Single	1,600	6.0
Dual-arm	Collaborative Tea	Dual	500	2.0
Dual-arm	Desktop Organization	Dual	900	3.3

TABLE I: Real-world teleoperation dataset.

Task	GR00T	$\pi_{0}$	$\pi_{0.5}$	smolVLA	GO-1	TAO-DA
Bottle	85	92.5	95	45	30	95
Steamer-L	90	–	90	–	57.5	94
Steamer-R	97.5	–	75	–	45	95
Green Tea-L	66.7	–	75	–	12.5	75
Black Tea-R	70.8	–	67.5	–	29.2	83
Average	82.0	–	80.5	–	34.8	88.4

TABLE II: Single-arm task success rates (%). The average includes only models evaluated on all five tasks.

Fig. 3: Active-arm trajectory predictions when the inactive arm is perturbed.

Active arm	Model	Clean MSE	Disturbed MSE	Ratio
Right	Single-Tower	$2.69\times 10^{-2}$	$5.23\times 10^{-2}$	1.94
	Dual-Mask	$2.54\times 10^{-2}$	$3.73\times 10^{-2}$	1.46
	Dual-Tower	$2.83\times 10^{-2}$	$2.80\times 10^{-2}$	0.99
Left	Single-Tower	$5.53\times 10^{-2}$	$8.85\times 10^{-2}$	1.60
	Dual-Mask	$5.88\times 10^{-2}$	$6.67\times 10^{-2}$	1.13
	Dual-Tower	$8.54\times 10^{-2}$	$8.54\times 10^{-2}$	1.00

TABLE IV: Offline robustness to inactive-arm disturbances.

Condition	Active arm	Single- Tower	Dual- Mask	Dual- Tower
Nominal	Left	80	90	95
Nominal	Right	70	95	100
Random initial state	Left	40	85	95
Random initial state	Right	60	80	95
Persistent disturbance	Left	25	65	100
Persistent disturbance	Right	20	65	95

TABLE V: Real-robot success rates (%) under the nominal condition, random initial states, and persistent inactive-arm disturbances (20 trials per condition).

Fig. 4: Qualitative comparison on Desktop Organization from identical initial conditions. The objects are located in the robot’s right-arm workspace. Without arm-intent routing, the inactive left arm attempts an empty grasp; with DAIR, it remains stationary while the right arm executes the task.

Fig. 5: Row-normalized confusion matrix for left-only, right-only, and dual-arm intent classification. Each row reports percentages over the corresponding ground-truth class.

Fig. 6: Per-class routing accuracy under instruction-diversity and task-progress ablations. Multi samples semantically equivalent instructions during training, whereas Fixed uses one instruction per task; w/o Progress removes the task-progress module.

Fig. 9: Qualitative single-to-dual-arm composition from separate arm-specific demonstrations. The two learned skills are executed in parallel without target-task bimanual demonstrations or additional fine-tuning.

Fig. 10: Representative tasks in the autonomous restaurant deployment.

Explore similar work

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems
Jun 18, 2026Yandong Wang, Jiaqian Yu, Xiongfeng Peng +8Bimanual Robotic Manipulation Vision-Language-Action Models
Donghua University, Shanghai, China · Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea
DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation
Mar 1, 2026Xiongfeng Peng, Jiaqian Yu, Dingzhe Li +8Robotic Manipulation Vision-Language-Action Models
Advanced Research Lab, Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea · Hanyang University ERICA, South Korea
MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization
Aug 26, 2026Zaibin Zhang, Junlan Xiao, Zhongbo Zhang +11Vision-Language-Action Models Instruction Generalization in VLA Models
Dalian University of Technology · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen +4

Jun 18, 2026J/K move · Enter open · S save

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems

Yandong Wang, Jiaqian Yu, Xiongfeng Peng, Lu Xu+7

Donghua University, Shanghai, China · Samsung R&D Institute China-Beijing (SRCB), China · Samsung AI Center, DS Division, South Korea

Bimanual Robotic Manipulation Vision-Language-Action Models

Open paper AskPDF arXiv

Vision-language-action (VLA) models show strong capabilities in single and dual-arm robotic manipulation. Prior works show coordinated bimanual behaviors can emerge from end-to-end learning, leveraging large vision-language backbones with continuous action prediction. However, as bimanual tasks become tightly coupled and execution constraints become critical, implicit coordination alone is insufficient to ensure reliable, interpretable, and stable behavior. In this work, we propose Co-VLA, a coordination-aware bimanual manipulation framework introducing explicit structural priors into VLA models. We instantiate our method on a state-of-the-art vision-language backbone by replacing its monolithic action head with a Structured Action Expert (SAE) designed for bimanual coordination. Specifically, we introduce explicit structure at the action generation level with a modular coordination-aware loss that shapes shared and residual latents according to task-specific structures. The shared latent encodes task-level coordination intent, while residual latents capture execution adjustments for each arm. At deployment, a Latent-Aware Controller (LAC) interprets the learned representations to modulate synchronization strength, execution asymmetry, smoothness, and safety constraints in real time. LAC operates at the joint-command level and remains compatible with standard control pipelines without requiring force or impedance control. Experiments across simulation and real-world benchmarks show Co-VLA significantly outperforms monolithic baselines, achieving a 27% success rate gain in tight-coordination tasks, more than doubling performance in OOD real-world scenarios (from 13% to 27%), and reducing task completion time by up to 25%.

Abstract

Explore similar work

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems

DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation

MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalization