Organizations: All the authors are with the Division of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology (NAIST), Nara, Japan. · Shandil Dhiresh Bosch is with the Department of Cognitive Robotics, Faculty of Mechanical Engineering, Delft University of Technology, Delft, Netherlands.
Generalizing decentralized multi-robot cooperative grasping and lifting across objects with diverse shapes and physical properties remains a fundamental challenge. Under decentralized execution, two key challenges arise: object-dependent representation learning under partial observability and coordination learning in multi-agent reinforcement learning (MARL) under non-stationarity. A typical approach jointly optimizes object-dependent representations and coordinated policies in an end-to-end manner while randomizing object shapes and physical properties during training. However, this joint optimization tightly couples representation and coordination learning, which can introduce bidirectional interference: inaccurate representations under partial observability can degrade coordination learning, while non-stationarity in MARL can further degrade representation learning, leading to degraded task performance. Inspired by prior work on decoupling representation and control learning in single-robot settings, we propose DeReCo, a MARL framework that decouples representation and coordination learning for object-adaptive decentralized multi-robot cooperative grasping and lifting. DeReCo adopts a two-phase training strategy: (1) coordination learning with privileged object information, and (2) transition to decentralized execution through reconstruction of object-dependent representations from local observations and retraining of coordination policies using the reconstructed representations. Experimental results show that DeReCo outperforms baselines in simulation on three training objects, generalizes to six unseen objects with varying masses and friction coefficients under similar grasping configurations, and achieves superior performance on two unseen objects in real-robot experiments.
Figures & tables
Fig. 1 : Multi-robot cooperative grasping and lifting of diverse objects using two HSRs.
Fig. 2 : Overview of the proposed framework for object-adaptive decentralized multi-robot cooperative grasping and lifting.
Fig. 3 : Objects used in the simulation. All units are meters.
Fig. 4 : Comparison of tracking rewards. DeReCo (Phase 2) shows the learning curve during coordination policy retraining.
Method
Seen Objects
Unseen Objects
Bar
Cylinder
Board
Avg.
Hexagon
Triangle
L bar
Thick bar
Octagon
Semi-ellipse
Avg.
MAPPO w/o PI
0.66
0.54
0.90
0.70
0.57
0.50
0.63
0.70
0.57
0.49
0.58
MAPPO w/o PI + LSTM
0.55
0.26
0.46
0.42
0.33
0.39
0.48
0.21
0.30
0.44
0.36
MAPPO w/o AE
0.81
0.69
0.95
0.82
0.72
0.61
0.74
0.72
0.71
0.60
0.68
MAPPO w/o AE + LSTM
0.57
0.35
0.73
0.55
0.38
0.35
0.50
0.36
0.37
0.42
0.40
MAPPO w PI †
0.92
0.83
0.96
0.91
0.44
0.31
0.53
0.60
0.47
0.33
0.45
TABLE I : Comparison of success rates on seen and unseen objects. The symbol † indicates that privileged information (PI) was used for seen objects during execution.
Fig. 5 : Failure analysis on unseen objects for four representative methods.
Fig. 6 : Objects used in real experiments. All units are meters.
Method
Board
Frame
Success
Error [m]
Success
Error [m]
MAPPO w/o AE
0/5
0.18 ± 0.06
0/5
0.34 ± 0.00
DeReCo
5/5
0.09 ± 0.01
4/5
0.09 ± 0.01
TABLE II : Comparisons of real-robot experiments
Fig. 7 : Comparisons of real-robot demonstrations.
Multi-robot systems must simultaneously optimize competing objectives while maintaining coordinated behavior. Existing multi-agent reinforcement learning approaches often rely on fixed or centralized coordination, which limits adaptability and violates distributed constraints. This work introduces the Coordination-Informed Multi-Objective Reinforcement Learning (CIMORL) framework, integrating a distributed weight prediction mechanism, a privileged expert training strategy, and theoretical guarantees for Pareto-optimal solutions. We present the base CIMORL method alongside two sampling-based variants, CIMORL-TS (Tree Search) and CIMORL-MPPI (MPPI), which leverage privileged global information during training to enable fully decentralized deployment. Experimental validation in cooperative and adversarial scenarios demonstrates a 21.2% hypervolume improvement and superior policy stability compared to state-of-the-art baselines. Real-world experiments with Crazyflie drones further validate the framework's robustness in resource allocation and multi-attacker multi-defend scenarios under partial observability.
Antonio Marino, Esteban Restrepo, Soon-jo Chung +2
University of Cambridge · CNRS, Univ Rennes, Inria, IRISA · Division of Engineering and Applied Science, California Institute of Technology
Decentralized multi-robot coordination under partial observability remains challenging, especially in communication-free settings where agents must act solely from local sensor observations. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.
Jiyue Tao, Shunheng Xin, Tongsheng Shen +2
Robotics and Control Laboratory, School of Advanced Manufacturing and Robotics, and the State Key Laboratory of Turbulence and Complex Systems, Peking University, Beijing, 100871, China · National Innovation Institute of Defense Technology, Beijing 100071, China
We argue that cooperative multi-agent reinforcement learning (MARL) should treat test-time coordination as part of the problem definition rather than a deployment artifact. Our focus is on long-horizon, centrally monitored cooperative systems such as warehouse robotics, automated storage-and-retrieval systems, and cloud-resource scheduling, where teams share a reward, operate under shifting objectives, and already run with global telemetry at deployment. The dominant paradigm in this regime, centralized training with decentralized execution (CTDE), uses global information only during training and provides no coordination channel at execution. This is sufficient when agents are weakly coupled or the environment is stationary, but it fails when the team must realign after conditions change. The limitation is structural rather than algorithmic, and more training cannot add a channel that does not exist at execution time. We call this the recoverability gap. To address it, we name the underexplored design space of intermittent test-time coordination as centralized training with mixed execution (CTME), in which a coordinator observes global state and emits high-level signals while agents act locally. CTME is not a particular learning algorithm, but a cooperative MARL problem setting that makes test-time information structure explicit. It specifies team-level information, high-level signals, and update frequency after training, with CTDE recovered as a limit case. We do not argue that CTDE is obsolete. We argue that removing the test-time channel is a modeling choice, not a necessity, and the wrong default when global telemetry is available and teams must recover from drift.
Dongsu Lee, Sooraj Sathish, Joonkyung Kim +2
The University of Texas at Austin · Texas A&M University · KAIST