Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots

Abdalwhab Bakheet Mohamed Abdalwhab; Giovanni Beltrame; David St-Onge

cs.ROSep 13, 2026

Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots

Authors: Abdalwhab Bakheet Mohamed Abdalwhab, Giovanni Beltrame, David St-Onge

Organizations: INIT Robots Lab, École de technologie supérieure, 1100 Notre-Dame W., Canada · MISTLab, Department of Computer and Software Engineering, Polytechnique Montréal, Canada

Abstract

Reinforcement learning (RL) has shown considerable promise for robotic decision-making, yet deploying multi-agent RL (MARL) on physical multi-robot systems in industrial environments remains challenging. This paper investigates the real-world applicability of decentralized MARL for multi-robot multi-machine tending. We propose Feature-fusion Multi-Agent Proximal Policy Optimization (FMAPPO), which fuses 2D LiDAR measurements with task-specific state information to enable safe decentralized multi-robot task assignment and navigation. A complete simulation-to-reality pipeline was developed using high-fidelity robotic simulation and ROS2 and deployed on physical mobile-manipulator platforms operating under realistic real-world conditions, with the robotic arms disabled during the experiments. We further investigate the sensitivity of the learned policy to command update frequency, an important consideration for real-world deployment. Comparative evaluation in simulation demonstrated that FMAPPO significantly outperformed state-of-the-art baselines with a large effect size, achieving improvements of 106% and 21% in parts delivery and 48% and 11% in parts collection over MAPPO and SMAPPO, respectively. FMAPPO also increased machine utilization by 31 and 10 percentage points, respectively, while reducing collisions by 18% and 15% and increasing the safety score by 14 and 6 percentage points compared with MAPPO and SMAPPO, respectively. Furthermore, real-world experiments demonstrated that the learned decentralized policies can coordinate multiple robots to service multiple machines while maintaining safe operation under real-world sensing and control constraints. Videos of the real-world experiment are available online https://anonymouspapers123.github.io/FMAPPO/.

Figures & tables

Fig. 1: Our Proposed Feature-Fusion Observation Encoder with the LiDAR processing branch.

Fig. 2: Example of a randomly generated layout in Isaac Sim. The environment contains two mobile manipulators (Rangen robots), a drop-off area (blue), and two machines (gray) with green indicators denoting available parts. The red points represent returns from the simulated safety LiDAR sensors mounted on the robots.

Fig. 3: Our Mobile Manipulator (Rangen): AgileX Ranger Mini omnidirectional base equipped with a Kinova Gen 3 arm; this work doesn’t include arm control.

Metric	MAPPO	SMAPPO	FMAPPO
Collected	2.50 (0.15)	3.33 (0.21)	3.71 (0.05)
Delivered	1.62 (0.21)	2.77 (0.34)	3.34 (0.07)
Collisions	2.06 (0.09)	1.99 (0.13)	1.69 (0.15)
Safety Score	0.67 (0.02)	0.75 (0.01)	0.81 (0.02)
Utilization	0.62 (0.04)	0.83 (0.05)	0.93 (0.01)

TABLE I: Performance comparison of FMAPPO vs MAPPO and SMAPPO, when trained for 500K environment steps using 256 parallel environments, with performance metrics computed over the final 200 episodes, reporting the mean (std) over seven different seeds.

Fig. 5: Learning curves for FMAPPO vs MAPPO and SMAPPO

Fig. 6: Frequency sensitivity analysis of the simulation study showing dimensionless metrics (Minimum inner robot separation $d^{*}$ , trajectory smoothness $J_{p}^{*}$ , and path length $S^{*}$ ) for each scenario and tested frequency as medians with interquartile ranges, in addition to the average number of collisions (if any)

Fig. 7: Sensitivity to command update frequency in simulation: overall task performance metrics: Scenario 1 results, reporting average and std.

Fig. 8: Sensitivity to command update frequency in simulation: overall task performance metrics: Scenario 2 results, reporting average and std

Metric	FMAPPO	NoLID	NoVel	NoOrient
Collected	3.71 (0.05)	3.60 (0.07)	3.70 (0.06)	3.73 (0.07)
Delivered	3.34 (0.07)	3.17 (0.10)	3.38 (0.08)	3.37 (0.11)
Collisions	1.69 (0.15)	2.04 (0.15)	1.91 (0.18)	1.75 (0.21)
Safety Score	0.81 (0.02)	0.77 (0.01)	0.79 (0.02)	0.80 (0.02)
Utilization	0.93 (0.01)	0.90 (0.02)	0.92 (0.01)	0.93 (0.02)

TABLE II: Ablation Study: Final Performance Comparison (FMAPPO, NoLID, NoVel, NoOrient) mean (std)

Fig. 9: Real deployment first layout: the two brown boxes on the right represent the machines, and the black box in the back represents the drop area.

Fig. 10: Real deployment second scenario: the two brown boxes on a diagonal represent the machines, and the black box in the bottom right corner represents the drop area.

Fig. 11: Frequency sensitivity analysis of the real deployment experiment showing dimensionless metrics (Minimum inner robot separation $d^{*}$ , action smoothness $J_{a}^{*}$ , and trajectory smoothness $J_{p}^{*}$ ) for each scenario and tested frequency as medians with interquartile ranges

Panel of Figure 14 — Fig. 12: The full trajectories from the actual real-world deployment for scenario 1 and scenario 2 for the case of command update frequency of 40 Hz; showing the robots going for multiple trips of collection and delivery.

Fig. 13: Full FMAPPO Actor Architecture: during training, the policy samples from a normal distribution using the predicted mean and log stds, while in evaluation mode the means are used directly as the action

Fig. 14: Full FMAPPO Critic Architecture with learnable parameters used as the query vector for the multi-head attention

Lidar Processing Branch
Parameter	Value
LiDAR downsampled size	270 (540 for the 2 lidars)
Conv1d layers kernal size	[3,3,3]
Conv1d layers channels num	[8,8,8]
MaxPool1d layers size	[4,4,4]
Lidar embed. dim. (FC size)	8

TABLE III: Model Parameters.

Parameter	Value
LiDAR range	40 meters
Physics step size	1/120 second
Decimation	6
Render Interval	6
Episode Length	15 Seconds
Entites init. position	Random from uniform distribution

TABLE IV: Training Parameters.

Parameter	Value
Collect reward	15.0
Deliver reward	30.0
Collision duration penalty	-1.0
New collisions penalty	-25.0
Uncollected part penalty	-0.1
Dist reward scale	20

TABLE V: Reward Parameters.

Model	Metric	M. Mean	Ours Mean	n	Diff	Shapiro-p	Normal	Test	p	p (Holm)	Sig (Holm)	Effect	E. Size
MAPPO	Collected	2.50	3.71	7	1.22	0.327	Yes	t-test	6.1e-07	6.0e-06	Yes	8.24	large
MAPPO	Delivered	1.62	3.34	7	1.72	0.280	Yes	t-test	6.0e-07	6.0e-06	Yes	8.27	large
MAPPO	Collisions	2.06	1.69	7	-0.36	0.107	Yes	t-test	0.002	0.009	Yes	-2.03	large
MAPPO	S. Score	0.67	0.81	7	0.14	0.407	Yes	t-test	9.3e-06	6.5e-05	Yes	5.18	large
MAPPO	Utilization	0.62	0.93	7	0.30	0.327	Yes	t-test	6.1e-07	6.0e-06	Yes	8.24	large
SMAPPO	Collected	3.33	3.71	7	0.39	0.092	Yes	t-test	0.004	0.014	Yes	1.75	large

TABLE VI: Main comparison experiment in the paper: Statistical Significance Testing FMAPPO vs MAPPO and SMAPPO (normality-selected test; effect size; Holm-Bonferroni-corrected p-values)

Metric	Effect	$F$	$\mathrm{df}_{e}$	$\mathrm{df}_{\mathrm{err}}$	$p$	$\eta^{2}_{p}$	$p$ (Holm)	Sig (Holm)
Min. distance	Frequency	1.35	4	90	0.2602	0.057	0.5204	No
Min. distance	Scenario	824.40	1	90	0.0001	0.902	0.0004	Yes
Min. distance	Freq. $\times$ Scen.	0.97	4	90	0.4363	0.041	0.8726	No
Position smoothness	Frequency	0.08	4	90	0.9997	0.004	0.9997	No
Position smoothness	Scenario	259.71	1	90	0.0001	0.743	0.0004	Yes
Position smoothness	Freq. $\times$ Scen.	0.64	4	90	0.8089	0.028	0.8726	No

TABLE VII: Command frequency update experiment in simulation: Permutation factorial ANOVA of frequency sensitivity (Holm–Bonferroni-corrected

p

-values on primary metrics; partial

\eta^{2}

effect size)

Scenario	Metric	$\mu_{10Hz}$	$\mu_{120Hz}$	n	Diff	Shapiro-p	Normal	Test	p	p (Holm)	Sig	$Sig_{Holm}$	Effect	E. Size
S1	Collected	14.00	14.00	10	0.00	–	–	–	1.000	1.000	No	No	0.00	neglig.
S1	Delivered	12.00	12.00	10	0.00	–	–	–	1.000	1.000	No	No	0.00	neglig.
S1	Collisions	0.00	0.00	10	0.00	–	–	–	1.000	1.000	No	No	0.00	neglig.
S1	Cost	2.35	2.31	10	-0.05	0.542	Yes	t-test	6.2e-07	3.7e-06	Yes	Yes	-3.89	large
S1	Min. dist.	1.94	1.96	10	0.02	0.554	Yes	t-test	0.029	0.147	Yes	No	0.82	large
S1	Mean dist.	2.57	2.58	10	0.01	0.670	Yes	t-test	0.588	1.000	No	No	0.18	neglig.

TABLE VIII: Command frequency update experiment in simulation: Statistical comparison of 120Hz vs 10Hz performance (paired over 10 seeds; normality-selected test; effect size; Holm-Bonferroni-corrected p-values per scenario); t-test: paired t-test, Wilcox: Wilcoxon, cost: Cost per part; Min. dist.: Min inter-robot distance; Mean dist.: Mean inter-robot distance

Model	Metric	M. Mean	Ours Mean	n	Diff	Shapiro-p	Normal	Test	p	p(Holm)	Sig	Sig(Holm)	Effect	E. Size
NoLID	Collected	3.60	3.71	7	0.12	0.788	Yes	t-test	0.002	0.037	Yes	Yes	1.88	large
NoLID	Delivered	3.17	3.34	7	0.17	0.668	Yes	t-test	0.007	0.090	Yes	No	1.49	large
NoLID	Collisions	2.04	1.69	7	-0.35	0.492	Yes	t-test	0.012	0.133	Yes	No	-1.34	large
NoLID	S. Score	0.77	0.81	7	0.04	0.493	Yes	t-test	0.005	0.070	Yes	No	1.61	large
NoLID	Utilization	0.90	0.93	7	0.03	0.788	Yes	t-test	0.002	0.037	Yes	Yes	1.88	large
NoVel	Collected	3.70	3.71	7	0.02	0.307	Yes	t-test	0.304	1.000	No	No	0.43	small

TABLE IX: Ablation study on each component: Statistical Significance Testing vs FMAPPO (normality-selected test; effect size; Holm-Bonferroni-corrected p-values with a significance level of

\alpha

=0.05)

Metric	FMAPPO	NoLvOrient
Collected	3.71 (0.05)	3.51 (0.06)
Delivered	3.34 (0.07)	3.00 (0.12)
Collisions	1.69 (0.15)	2.33 (0.42)
Safety Score	0.81 (0.02)	0.74 (0.04)
Utilization	0.93 (0.01)	0.88 (0.01)

TABLE X: Ablation Study Final Performance Comparison (FMAPPO, NoLvOrient (FMAPPO with no LiDAR, Velocity, or Orientation information included)) mean (std)

Model

Metric

\mu_{model}

\mu_{Ours}

Diff

Shapiro-p

Normal

Test

p (Holm)

Sig

Sig (Holm)

Effect

E. Size

NoLvOrient

Collected

3.51

3.71

0.20

0.381

Yes

t-test

3.8e-05

1.9e-04

Yes

4.07

large

NoLvOrient

Delivered

3.00

3.34

0.34

0.516

Yes

t-test

4.3e-04

0.001

Yes

2.63

large

NoLvOrient

Collisions

2.33

1.69

-0.63

0.068

Yes

t-test

0.002

0.005

Yes

-1.90

large

NoLvOrient

S. Score

0.74

0.81

0.07

0.050

Wilcoxon

0.016

Yes

1.00

large

NoLvOrient

Utilization

0.88

0.93

0.05

0.381

Yes

t-test

3.8e-05

1.9e-04

Yes

4.07

large

TABLE XI: Ablation Study Statistical Significance Testing FMAPPO vs NoLvOrient (FMAPPO with no LiDAR, Velocity, or Orientation information included) (normality-selected test; effect size; Holm-Bonferroni-corrected p-values)

Metric	Effect	$F$	$\mathrm{df}_{e}$	$\mathrm{df}_{\mathrm{err}}$	$p$	$\eta^{2}_{p}$	$p$ (Holm)	Sig (Holm)
Min. distance	Frequency	4.59	2	57	0.0133	0.139	0.0266	Yes
Min. distance	Scenario	5.97	1	57	0.0178	0.095	0.0356	Yes
Min. distance	Freq. $\times$ Scen.	0.54	2	57	0.5811	0.019	1.0	No
Action smoothness	Frequency	2.23	2	57	0.1156	0.073	0.1156	No
Action smoothness	Scenario	0.25	1	57	0.6192	0.004	0.6192	No
Action smoothness	Freq. $\times$ Scen.	2.19	2	57	0.1178	0.071	0.3534	No

TABLE XII: Real Deployment Experiment: Permutation factorial ANOVA of frequency sensitivity (Holm–Bonferroni-corrected

p

-values on primary metrics; partial

\eta^{2}

effect size)

Fig. 15: Real Deployment first layout results: showing (a) task throughput (total collected parts, total delivered parts), (b) efficiency as collection cost, and (c) minimum distance between the two robots.

Fig. 16: Real Deployment second layout results: showing (a) task throughput (total collected parts, total delivered parts), (b) efficiency as collection cost, and (c) minimum distance between the two robots.

Fig. 17: Sample efficiency: FMAPPO vs MAPPO and SMAPPO achieved performance at 10%, 50%, 90% of the training budget, mean (std) over 7 seeds (per-seed trailing 10-episode window) with fixed part processing time of 150 steps

Model

Metric

\mu_{model}

\mu_{ours}

Diff

Shapiro-p

Normal

Test

p_{Holm}

Sig

Sig_{Holm}

Effect

E. Size

MAPPO

Collected

2.42

3.69

1.27

0.394

Yes

t-test

6.2e-07

4.3e-06

Yes

8.22

large

MAPPO

Delivered

1.50

3.28

1.78

0.630

Yes

t-test

4.8e-07

3.8e-06

Yes

8.58

large

MAPPO

S. Score

0.65

0.80

0.15

0.987

Yes

t-test

2.3e-05

1.2e-04

Yes

4.43

large

MAPPO

Utilization

0.60

0.92

0.32

0.394

Yes

t-test

6.2e-07

4.3e-06

Yes

8.22

large

SMAPPO

Collected

3.32

3.69

0.37

0.143

Yes

t-test

0.009

0.028

Yes

1.42

large

SMAPPO

Delivered

2.76

3.28

0.53

0.067

Yes

t-test

0.013

0.028

Yes

1.33

large

TABLE XIII: Sample efficiency: Statistical significance of performance at 50% of training vs FMAPPO (normality-selected test; effect size; Holm-Bonferroni-corrected p-values) with fixed part processing time of 150 steps

Metric	FMAPPO	MAPPO	Imp. vs MAPPO	SMAPPO	Imp. vs SMAPPO
Collected	5.71 (0.18)	3.30 (0.23)	73.03%	4.23 (0.32)	34.99%
Delivered	4.98 (0.19)	2.26 (0.26)	120.35%	3.26 (0.38)	52.76%
Collisions	2.02 (0.14)	2.39 (0.17)	$-15.48\%$	2.26 (0.19)	$-10.62\%$
Safety Score	0.84 (0.01)	0.70 (0.02)	0.14	0.77 (0.02)	0.07

TABLE XIV: Random Part Processing Time: Final Performance Comparison (FMAPPO, MAPPO, SMAPPO) mean (std)

Fig. 18: Random Part Processing Time Learning curves overview over 10 seeds

Model

Metric

\mu_{model}

\mu_{ours}

Diff

Shapiro-p

Normal

Test

p (Holm)

Sig

Sig (Holm)

Effect

E. Size

MAPPO

Collected

3.30

5.71

2.41

0.257

Yes

t-test

5.6e-09

3.4e-08

Yes

6.68

large

MAPPO

Delivered

2.26

4.98

2.72

0.251

Yes

t-test

4.2e-09

3.4e-08

Yes

6.89

large

MAPPO

Collisions

2.39

2.02

-0.37

0.228

Yes

t-test

4.4e-04

8.8e-04

Yes

-1.70

large

MAPPO

S. Score

0.70

0.84

0.14

0.377

Yes

t-test

4.2e-09

3.4e-08

Yes

6.90

large

SMAPPO

Collected

4.23

5.71

1.48

0.520

Yes

t-test

1.2e-06

3.8e-06

Yes

3.62

large

SMAPPO

Delivered

3.26

4.98

1.72

0.390

Yes

t-test

9.6e-07

3.8e-06

Yes

3.70

large

TABLE XV: Random parts processing time: Statistical Significance Testing vs FMAPPO (normality-selected test; effect size; Holm-Bonferroni-corrected p-values)

Fig. 19: Sample efficiency: achieved performance at 10%, 50%, 90% of the training budget, mean (std) over 10 seeds (per-seed trailing 10-episode window) with random part processing time between 60 and 900 environment steps

Model

Metric

\mu_{model}

\mu_{ours}

Diff

Shapiro-p

Normal

Test

p (Holm)

Sig

Sig (Holm)

Effect

E. Size

MAPPO

Collected

3.18

5.60

2.42

0.748

Yes

t-test

2.2e-09

1.1e-08

Yes

7.43

large

MAPPO

Delivered

2.11

4.87

2.76

0.787

Yes

t-test

1.8e-09

1.1e-08

Yes

7.61

large

MAPPO

S. Score

0.69

0.83

0.14

0.868

Yes

t-test

1.0e-08

4.1e-08

Yes

6.24

large

SMAPPO

Collected

4.23

5.60

1.37

0.928

Yes

t-test

1.8e-06

3.6e-06

Yes

3.44

large

SMAPPO

Delivered

3.28

4.87

1.59

0.989

Yes

t-test

8.2e-07

2.5e-06

Yes

3.77

large

SMAPPO

S. Score

0.77

0.83

0.06

0.662

Yes

t-test

7.1e-06

Yes

2.91

large

TABLE XVI: Random parts processing time: Statistical significance of performance at 50% of training vs FMAPPO (normality-selected test; effect size; Holm-Bonferroni-corrected p-values)

Panel of Figure 36 — Fig. 20: Position ( $\sigma$ 2 cm), velocity ( $\sigma$ 2 cm/s), LiDAR ( $\sigma$ 1 cm), action ( $\sigma$ 1 cm/s), and initial positions ( $\sigma$ 10 cm), in the left Scenario 1, and in the right Scenario 2

Panel of Figure 37 — Fig. 21: Position ( $\sigma$ 20 cm), velocity ( $\sigma$ 20 cm/s), LiDAR ( $\sigma$ 1 cm), action ( $\sigma$ 1 cm/s), and initial positions ( $\sigma$ 20 cm), in the left Scenario 1, and in the right Scenario 2

Panel of Figure 38 — Fig. 22: Position ( $\sigma$ 10 cm), velocity ( $\sigma$ 20 cm/s), LiDAR ( $\sigma$ 1 cm), action ( $\sigma$ 1 cm/s), and initial positions ( $\sigma$ 20 cm), in the left Scenario 1 and in the right Scenario 2

Decentralized MARL

Decentralized multi-agent reinforcement learning studies how autonomous agents learn coordinated behavior from local information, with or without communication.

See all 122 papers

Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shielding
Sep 7, 2026Yacine El Yamani, Hanna Krasowski, Elena VanneauxMulti-Robot Coordination Robot Safety
ENSTA, IP Paris, Palaiseau, France · UC Berkeley, USA
CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy
Jun 10, 2026Ria Doshi, Tian Gao, Annie Chen +2Mobile Robotics Cross-Embodiment Robot Learning
Stanford University
Cooperative-Competitive Team Play of Real-World Craft Robots
Feb 24, 2026Rui Zhao, Xihui Li, Yizheng Zhang +6Cooperative MARL Multi-Robot Coordination
The authors are with Tencent Robotics X Laboratory, Shenzhen, Guangdong, China. · The author is with Tsinghua University, Shenzhen, Guangdong, China.

Sep 7, 2026J/K move · Enter open · S save

Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shielding

Yacine El Yamani, Hanna Krasowski, Elena Vanneaux

ENSTA, IP Paris, Palaiseau, France · UC Berkeley, USA

Multi-Robot Coordination Robot Safety Decentralized MARL Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Environments are increasingly populated by multiple robots performing independent tasks with limited prior knowledge of each other. Deploying such multi-agent systems presents significant challenges. Specifically, shifts in deployment states compared to training data can lead to poor policy performance and compromised safety. While safety shields exist to mitigate these risks, they are typically reactive, which degrades performance near unseen obstacles,and centralized, limiting their scalability. To address this, we propose a decentralized framework that integrates predictive shielding with model-based finite horizon Q-learning. This approach allows agents to safely adapt their pre-trained policies during deployment. Furthermore, to mitigate livelocks in symmetric scenarios, we introduce a communication- free protocol for conflict resolution

Abstract

Decentralized MARL

Explore similar work

Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shielding

CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy

Cooperative-Competitive Team Play of Real-World Craft Robots