Hierarchical Multi-agent Reinforcement Learning for Warehouse Robot Coordination under Communication Loss

Weihao Sun; Gehui Xu; Andreas A. Malikopoulos

eess.SYSep 27, 2026

Hierarchical Multi-agent Reinforcement Learning for Warehouse Robot Coordination under Communication Loss

Authors: Weihao Sun, Gehui Xu, Andreas A. Malikopoulos

Organizations: Systems Engineering Program, Cornell University, Ithaca, NY 14850 USA · Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK · Applied Mathematics, Systems Engineering, Mechanical Engineering, Electrical & Computer Engineering, and School of Civil & Environmental Engineering, Cornell University, Ithaca, NY, USA

Abstract

In this paper, we propose a hierarchical multi-agent reinforcement learning framework for coordinating robot teams in warehouse environments under communication loss. We partition the robot team into groups, with centralized coordination within each group and distributed coordination across groups. Each group uses a recurrent predictor to estimate unavailable interaction information due to communication loss. A higher-level policy then generates a compact coordination reference that conditions the local control policy within each group. A predictive safety filter evaluates and modifies the proposed controls when they violate safety constraints. Simulation results show improved task completion under communication loss, reduced communication growth as the team size increases, and safe operation in the tested scenarios.

Figures & tables

Fig. 1: Overview of the proposed hierarchical coordination framework.

Fig. 2: Training convergence of the proposed method and the comparison baseline under different robot team sizes, with a 0.2 communication loss rate.

Fig. 3: Success rate comparison between the proposed method and IC3Net for teams of $40$ and $60$ robots under communication loss probabilities $p\in\{0.2,0.4,0.6\}$ .

Fig. 4: (a). Communication load per time step for the proposed method and the fully decentralized baseline. (b). Ratio between payloads of the two methods and the analytical result.

$N$	Collision rate	Correction rate	Mean candidates
40	0.00%	3.45%	3.79
60	0.00%	5.48%	16.44
80	0.00%	7.90%	24.15
90	0.00%	6.54%	35.63
100	0.00%	7.34%	24.68

TABLE I: Safety filter performance under

p=0.2

Explore similar work

Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss
Jul 20, 2026Kemal Devrim Kafadar, Eren Özaltun, Mahmud Efnan Şanlı +4Multi-Agent Reinforcement Learning Multi-Agent Coordination
Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye. · Department of Computer Science, University of Stuttgart, Stuttgart, Germany. · Istanbul Technical University Artificial Intelligence and Data Science Application and Research Center, Istanbul, Türkiye. +1
SCALE-COMM: Shared, Contrastively-Aligned Latent Embeddings for MARL Communication
May 26, 2026Mahmoud Abouelyazid, Eman HammadMulti-Agent Reinforcement Learning Mobile Robots
Electrical and Computer Engineering Department Texas A&M University, College Station, TX, USA
HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning
Jun 28, 2026Runze Zhao, Dongruo Zhou, Sumit Kumar Jha +2Multi-Agent Reinforcement Learning Vectors
Luddy School of Informatics, Computing, and Engineering Indiana University Bloomington Bloomington, IN 47408 · Department of Computer & Information Science & Engineering University of Florida Gainesville, FL 32611 · Department of Electrical Engineering and Computer Science United States Military Academy West Point, NY 10996 +1

Jul 20, 2026, cs.MAJ/K move · Enter open · S save

Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss

Kemal Devrim Kafadar, Eren Özaltun, Mahmud Efnan Şanlı, Feyza Orak+3

Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye. · Department of Computer Science, University of Stuttgart, Stuttgart, Germany. · Istanbul Technical University Artificial Intelligence and Data Science Application and Research Center, Istanbul, Türkiye.+1

Multi-Agent Reinforcement Learning Multi-Agent Coordination

Open paper AskPDF arXiv

Robust multi-agent coordination relies heavily on inter-agent communication, which is frequently disrupted by physical and environmental constraints in real-world deployments. To maintain operation during these intermittent communication failures, agents can employ internal prediction models to estimate missing shared state information. However, predictors trained with standard reconstruction objectives treat all transitions equally. In a Reinforcement Learning context, this forces the model to waste capacity learning stochastic exploration noise and the outdated dynamics of suboptimal policies. In this paper, we propose a value-aware extension of Multi-Agent Observation Sharing under Communication Dropout (MARO) to patch communication gaps; we refer to this method as Value-Aware MARO. By dynamically weighting the predictor's loss function using advantage estimates derived from the underlying actor-critic architecture, our objective explicitly couples the predictor's learning process to the policy's evolution. This formulation focuses the model's capacity on the intentional, high-return dynamics actively reinforced by the agents. We evaluate our framework on several tasks within the Multi-Agent Particle Environment under varying communication reliability levels. Experimental results demonstrate that our approach maintains performance under declining communication reliability, particularly below 40%. While our method performs comparably in tasks where the baseline already maintains high coordination, our value-aware weighting effectively prevents the performance collapse observed in the standard predictor during high-attrition scenarios. In these environments, our method achieves an average improvement in mean returns of more than 20% and reduces performance variance by a mean of 64.7% compared to the standard unweighted baseline.

Abstract

Explore similar work

Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss

SCALE-COMM: Shared, Contrastively-Aligned Latent Embeddings for MARL Communication

HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning