DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication

Hanchu Zhou; Dechen Gao; Hang Wang; Brendan Lynch; Boqi Zhao; Qiyao Ma; Raman Goyal; Junshan Zhang

cs.ROOct 1, 2026

DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication

Authors: Hanchu Zhou, Dechen Gao, Hang Wang, Brendan Lynch, Boqi Zhao, Qiyao Ma, Raman Goyal, Junshan Zhang

Organizations: University of California, Davis · Microsoft Research · Analog Devices

Abstract

Vision-language models (VLMs) and vision-language-action models (VLAs) have recently driven rapid progress in general-purpose robots, yet most progress has focused on single-robot settings. Extending these capabilities to multi-robot systems remains challenging because robots must coordinate long-horizon behaviors while maintaining reliable, fine-grained execution. We introduce DuoMind, a distributed hierarchical framework for multi-robot coordination through semantic communication. Each robot uses a VLA-based action model for low-level execution and a VLM-based orchestrator for high-level reasoning and inter-agent coordination. At each planning step, the orchestrator at each robot reasons over the task instruction, local observations, and messages received from other robots. It then generates low-level instructions for the action model and semantic messages for peer robots. This architecture exploits the complementary strengths of pretrained models by combining the semantic reasoning capabilities of VLMs with the precise action-generation capabilities of VLAs. To address the scarcity of benchmarks for multi-robot coordination, we further develop RoboPoly, a benchmark comprising long-horizon manipulation tasks that require coordinated, closed-loop execution under distributed control. Experiments on RoboPoly and RoboTwin demonstrate that DuoMind improves multi-robot task performance, while ablation studies confirm the contributions of hierarchical orchestration and semantic communication. More details are available on our project page.

Figures & tables

Figure 1: (a) An illustration of DuoMind , a distributed framework combining an orchestrator and action model for multi-robot coordination. (b) RoboPoly is a benchmark we develop for long-horizon coordination under distributed robot control. (c) DuoMind consistently outperforms the baselines across both benchmarks.

Figure 2: In DuoMind , each robot operates with a VLM as its orchestrator for high-level reasoning and a VLA as its action model for low-level control. At each high-level planning step, the orchestrator at each robot reasons and generates both a low-level instruction for the action model and a structured message for other robots. The action model subsequently follows the low-level instruction and produces fine-grained actions.

Method	Hang Bag	Food Serve	Prepare Snack	Clean Table
DuoMind w/ $\bm{\pi}_{0.5}$	78.00	32.75	18.00	24.75
$\bm{\pi}_{0.5}$ only	52.25	29.75	16.25	16.50
$\bm{\pi}_{0}$ only	58.75	22.25	3.75	0.00

Table 1: Comparison of success rates on RoboPoly tasks.

Method	Hanging Mug	Pick Diverse Bottles	Stack Bowls Two	Put Bottle Dustbin
DuoMind w/ $\bm{\pi}_{0.5}$	24.75	58.50	85.50	34.50
$\bm{\pi}_{0.5}$ only	17.25	48.50	82.50	31.25
$\bm{\pi}_{0}$ only	12.50	42.50	33.75	21.25

Table 2: Comparison of success rates on RoboTwin tasks.

Figure 3: We observe that explicit inter-agent communication enables the robots to better coordinate their behaviors, such as exchanging bread sequentially or placing objects on different plates. By avoiding conflicting or redundant actions in a narrow shared workspace, the robots achieve more efficient and safer coordination.

Panel of Figure 4 — Figure 4: Ablation studies of DuoMind . (a) Success rates when inter-agent communication is disabled. (b) Success rates with different action models.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: The task overview of RoboPoly .

Figure 6: The task overview of selected RoboTwin tasks.

Figure 7: The result of ablation study that disable the inter-agent communication in DuoMind .

Figure 8: Results of the ablation study using different action models in DuoMind .

Task	Method	Train steps	Batch	Peak LR	Execute / horizon	VLM interval
Hang Bag	$\pi_{0.5}$ / $\pi_{0}$ only	10k	256	$2.5\times 10^{-5}$	12/16	–
Hang Bag	DuoMind	10k	256	$2.5\times 10^{-5}$	12/16	4
Food Serve	$\pi_{0.5}$ / $\pi_{0}$ only	10k	256	$2.5\times 10^{-5}$	16/16	–
Food Serve	DuoMind	10k	256	$2.5\times 10^{-5}$	16/16	4
Prepare Snack	$\pi_{0.5}$ / $\pi_{0}$ only	2k	512	$1\times 10^{-4}$	16/16	–
Prepare Snack	DuoMind	2k	512	$1\times 10^{-4}$	16/16	4

Appendix

Table 3: Fine-tuning and evaluation settings on RoboPoly .

Task	Method	Train steps	Batch	Peak LR	Execute / horizon	VLM interval
Hanging Mug	$\pi_{0.5}$ / $\pi_{0}$ only	20k	32	$2.5\times 10^{-5}$	50/50	–
Hanging Mug	DuoMind	20k	32	$2.5\times 10^{-5}$	50/50	4
Pick Diverse Bottles	$\pi_{0.5}$ / $\pi_{0}$ only	20k	32	$2.5\times 10^{-5}$	50/50	–
Pick Diverse Bottles	DuoMind	20k	32	$2.5\times 10^{-5}$	50/50	4
Stack Bowls Two	$\pi_{0.5}$ / $\pi_{0}$ only	20k	32	$2.5\times 10^{-5}$	50/50	–
Stack Bowls Two	DuoMind	20k	32	$2.5\times 10^{-5}$	50/50	2

Appendix

Table 4: Fine-tuning and evaluation settings on RoboTwin.

Figure 9: We evaluate DuoMind on 7 RoboPoly tasks.

Figure 10: We evaluate DuoMind on 8 selected RoboTwin tasks.

Explore similar work

Jun 10, 2026cs.RO

CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy

Multi-robot collaboration allows robots to efficiently take on a wide range of tasks, from moving a couch through a doorway to assembling structures on a construction site. However, achieving such coordination in mobile multi-robot settings remains challenging: centralized methods conditioned on the combined observations of a team scale poorly with team size, and decentralized methods that train one policy per robot often require explicit alignment procedures or information sharing at inference time to overcome partial observability. Our key insight is that the visuomotor priors of pretrained vision-language-action (VLA) models should enable reactive, decentralized collaboration from each robot's local observations alone, without these inference-time assumptions. We propose CHORUS, a framework that adapts a single VLA backbone to control diverse, multi-robot teams. At inference time, each robot runs an independent copy of CHORUS, conditioned only on its own observations and a robot-identifying prompt. In real-world experiments including mobile tape measurement, library book handovers, and laundry basket lifting, CHORUS achieves a 64% point improvement over decentralized, from-scratch models, improves reactivity to teammate behavior by 40% points, and outperforms centralized baselines. Together, these results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration, without per-robot policies or inter-robot communication at inference.

Ria Doshi, Tian Gao, Annie Chen +2

Stanford University

Multi-Robot Systems Collaboration Kinematic Priors Chorus

Ask PDF

Jul 31, 2026cs.RO

D-VLC: Decentralized Vision-Language Collaboration for Heterogeneous Embodied Multi-Robot Systems in Unknown Environments

Multi-robot systems, particularly heterogeneous robot swarms, can improve the efficiency of complex task execution through parallel collaboration and complementary capabilities. However, conventional rule-based methods rely on predefined task models and specialized decision making programs, making it difficult to understand complex semantic instructions and coordinate heterogeneous robots. LLMs introduce strong language understanding and task reasoning capabilities, allowing multi-robot systems to interpret instructions, decompose tasks, and assign roles according to task semantics. VLMs further incorporate visual perception, enabling robots to reason about objects, regions, and spatial relationships in physical environments. Nevertheless, existing LLM/VLM based methods often depend on known maps, centralized and synchronized decision making, limiting their generalization to heterogeneous robots and unseen tasks. We therefore propose a framework that combines decentralized asynchronous reasoning, lightweight information sharing, capability aware collaboration, and a unified action interface, enabling general purpose VLMs to generate robot specific actions executed by learning free experts without task or robot specific training. Experiments across diverse scenarios and multiple VLMs show success rates above 70%, with completion time reduced by up to 55.8% relative to the geometric greedy baseline.

Yuan Zhou, Ruitong Lin, Shen Wang +5

Zhejiang University, Hangzhou, China · Differential Robotics, Hangzhou, China · Beihang University, Beijing, China

Heterogeneous Robot Teams Multi-Robot Systems Recent Vision-Language Models Reasoning Skills Environment

Ask PDF

Sep 21, 2026cs.RO

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

Multi-robot collaboration could enable more efficient and scalable solutions to complex robotic tasks, but collaboration under partial observability remains challenging. Natural-language communication offers a promising approach to coordinating robots under partial observability. However, in decentralized manipulation, jointly learning explicit inter-robot communication and skill-level action selection from multimodal demonstrations remains underexplored for small vision-language models (VLMs) intended for on-device deployment. To address this gap, we introduce RoboTalk, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate. The dataset includes a leader-follower planning protocol, tool calls (perception, manipulation, navigation, and communication), rationale traces, and diversified natural-language communication. Fine-tuning open-source models on our dataset can reach 77% success on novel held-out tasks, a significant improvement over the untuned open source models, which had a success rate of around ~2%.

Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud +3

University of Massachusetts, Amherst, MA, USA. · University of California, Berkeley, CA, USA. · Northeastern University, Boston, MA, USA.

Large-Scale Robot Demonstration Datasets Multi-Robot Systems Coordination

Ask PDF