cs.ROOct 5, 2026

Learning Coordinated Visuomotor Box-Pushing from Solo Demonstrations

Authors: Ryusei Matsumoto, Mei Kinjo, Yoko Sasaki, Keisuke Okumura

Organizations: National Institute of Advanced Industrial Science and Technology (AIST), Japan · Institute of Science Tokyo, Japan · The University of Tokyo, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multi-robot imitation learning, particularly in settings where visuomotor policies are deployed in a communication-free, onboard decentralised style, represents an attractive paradigm. However, its realisation remains insufficiently understood, largely due to the difficulty of collecting collective demonstrations, since a single operator cannot control many robots simultaneously. Meanwhile, unlike coupled collaborative manipulation, many coordinated tasks achieve system-wide efficiency primarily through minimising inter-robot interference. This structure motivates us to study whether data collected by a teleoperated single-robot can be leveraged for large-scale coordinated box-pushing as a testbed. We systematically investigate dataset creation strategies and lightweight policy architectures. In particular, experiments with up to 40 robots highlight the difficulty of acquiring effective coordination solely through passive observation of other operating robots, revealing a concrete bottleneck for multi-robot research.

Figures & tables

Figure 1: We learn coordinated box-pushing policies from solo demonstrations ( left ). A lightweight mixture-of-experts architecture captures diverse skills under onboard resource constraints ( middle ). The resulting policy enables a multi-robot team to clean up scattered objects ( right ). The video is available at https://youtu.be/J9KfPI-hJX4 .
Figure 2: Training dataset breakdowns (# of observation-action pairs) for each policy.
Figure 3: Policy performance across datasets and loss designs. (a) Task throughput, measured as the number of transported objects per unit time. (b) Task progress over time.
Figure 4: Policy performance with data collected via the onboard camera stream. ⋄\diamond markers indicate the reference performance obtained with direct operator observation ( Fig. 3 ).
Figure 5: (a) Effect of dataset scaling to 60K samples, evaluated in terms of task throughput. White diagonal markers indicate the reference performance obtained with 20K-sample training. (b) Dataset ablation results, (c) sensitivity to the robot population in the Static training dataset, and (d) investigation of the MoE architecture using πstatic\pi_{\text{static}} .
Figure 6: Snapshots of the colour classification task (left) and the policy performance (right) .
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Workspace layouts for the box collection task (left) and the colour classification task (right) .
Input Configuration
Sequence length5
Image resolution96×7296\times 72
Backbone and Encoder
BackboneMobileNetV2
Encoder output dimension36
Mixture-of-Experts
Appendix
Table 1: Model and training hyperparameters
Figure 8: (a) Further evaluation of the MoE architecture. (b) Contribution of Static data to policy performance.

Explore similar work

CardsList

  1. CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy

    Jun 10, 2026Ria Doshi, Tian Gao, Annie Chen +2Mobile RoboticsCross-Embodiment Robot Learning

    Stanford University

  2. Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations

    May 12, 2026Lasse Peters, Laura Ferranti, Andrea Bajcsy +1Multi-Agent CoordinationDiffusion Policy

    University of California, Berkeley · Delft University of Technology · Carnegie Mellon University

  3. RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

    Sep 21, 2026Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud +3Multi-Robot SystemsLanguage-Conditioned Robot Manipulation

    University of Massachusetts, Amherst, MA, USA. · University of California, Berkeley, CA, USA. · Northeastern University, Boston, MA, USA.