Distributed Secure Learning Control for Large-scale Multirobots under Stealthy Actuator Attacks

Xinglong Zhang; Qingwen Ma; Cong Li; Hui Yin; Changxin Zhang; Yueying Wang; Wei Pan; Xin Xu

cs.ROSep 7, 2026

Distributed Secure Learning Control for Large-scale Multirobots under Stealthy Actuator Attacks

Authors: Xinglong Zhang, Qingwen Ma, Cong Li, Hui Yin, Changxin Zhang, Yueying Wang, Wei Pan, Xin Xu

Organizations: College of Intelligence Science and Technology, National University of Defense Technology, Changsha, 410073, China · College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China · School of Mechatronic Engineeringand Automation, Shanghai University, Shanghai 200444, China · Department of Computer Science, The University of Manchester, 2628 CD Manchester, U.K.

Abstract

Distributed learning control for multirobot systems (MRS) offers significant flexibility in presence of uncertainties but lacks provable performance guarantees. A promising direction involves integrating reinforcement learning (RL) into distributed model predictive control (DMPC), leveraging the strengths of RL in nonlinear policy design and the receding-horizon replanning capabilities of DMPC. However, ensuring secure control within such a learning framework under malicious cyber attacks, particularly stealthy ones, remains a critical challenge, because the distributed policies generation depends on information exchange among neighbors, where compromised agents can rapidly influence the behavior of others through the communication network. This article proposes a distributed secure learning control (DSLC) framework for large-scale MRS under malicious, stealthy actuator attacks. Our framework offers two key features: (i) a unified approach that enables secure learning control across various coordination scenarios and (ii) a game-theoretic distributed learning-based predictive control strategy that learns how to balance the attacker and defender through a differential-game based DMPC framework. Specifically, DSLC employs a distributed attacker-actor-critic architecture to learn the optimal defense and attack policies online within each prediction interval. Unlike numerical optimization-based controllers that calculate open-loop control sequences, our method simultaneously generates adversarial attack policies and corresponding defense policies in analytical closed-loop form. The defense policies could be directly generalized to MRS with varying scales and diverse actuator attack probabilities. The effectiveness and scalability of DSLC are validated through comprehensive simulations and real-world experiments in multiple wheeled robots via various control tasks.

Figures & tables

Fig. 1: The scheme diagram of the distributed secure learning control (DSLC) framework. The attackers that deteriorate the control performance and the defenders that secure the closed-loop system are modeled as multi-players in a differential-game based DMPC framework. The resulting problem is solved by a computationally efficient attacker-actor-critic learning algorithm, which generates the closed-loop control policies for the attacker and defender. $\vec{\beta}_{t}=(\beta_{t,1},\cdots,\beta_{t,M})$ .

Attacks	Methods	Two Robots			Four Robots			Six Robots
Attacks	Methods	IAE	ITAE	$J_{c}$	IAE	ITAE	$J_{c}$	IAE	ITAE	$J_{c}$
$\beta=0.1$	DSLC	0.57	0.47	3.35	0.82	0.8	4.13	0.93	1.22	7.09
	DRHRL [ 8 ]	7.04	46.14	44.45	11.49	101.41	39.52	25.65	255.22	249.43
	DRMPC [ 6 ]	19.56	206.85	23.68	38.51	546.78	127.14	89.83	1014.86	470.44
$\beta=0.2$	DSLC	0.65	0.5	3.36	0.73	1.42	3.84	0.78	1.15	6.01
	DRHRL [ 8 ]	17.63	172.64	283.94	28.45	214.12	187.36	24.34	241.9	277.96

TABLE I: Criterion for the Accessment of the Distributed Control Performance under Different Attacks.

Fig. 2: Communication graphs of multirobots, where Robot 0 is the virtual leader.

Fig. 3: The curves of the cost to go

J_{p}=\int_{\tau=0}^{t_{\rm sim}}r(\tau)d\tau

for formation control under different attack probabilities, where

t_{\rm sim}

is the terminal time of the simulation period.

Fig. 4: The trajectories of six-robot formation with different random attack probabilities under DRHRL [ 8 ] , DRMPC [ 6 ] , and DSLC.

Fig. 5: The curves of $\bar{r}_{i}=\|x_{\scriptscriptstyle\mathcal{N}_{i}}\|_{Q_{i}}^{2}+\|u_{s,i}\|_{R_{s,i}}^{2}$ for general formation of 100 and 1000 robots under different random probabilistic attacks using the policies learned from two robots.

Fig. 6: Schematic diagram of the MRS experimental platform under additive deception actuator attacks. (a) The scene of the experimental platform; (b) The information flow for MRS under attacks.

Fig. 7: In Scenario I, trajectories and snapshots of 4 robots without attacks ( $\beta=0$ ) under DSLC.

Fig. 8: In Scenario II, trajectories and snapshots of 4 robots without attacks ( $\beta=0$ ) under DSLC.

Fig. 9: Scenario I: trajectories and snapshots of 6 robots without attacks ( $\beta=0$ ) under DSLC.

Fig. 10: Scenario II: trajectories and snapshots of 6 robots without attacks ( $\beta=0$ ) under DSLC.

Fig. 11: Scenario I: trajectories and snapshots of 4 robots with an attack probability of $\beta=0.5$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 12: Scenario II: trajectories and snapshots of 4 robots with an attack probability of $\beta=0.5$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 13: Scenario I: trajectories and snapshots of 6 robots with an attack probability of $\beta=0.5$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 14: Scenario II: trajectories and snapshots of 6 robots with an attack probability of $\beta=0.5$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 15: Scenario I: trajectories and snapshots of 4 robots with an attack probability of $\beta=1$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 16: Scenario II: trajectories and snapshots of 4 robots with an attack probability of $\beta=1$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 17: Scenario I: trajectories snapshots of 6 robots with an attack probability of $\beta=1$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Fig. 18: Scenario II: trajectories and snapshots of 6 robots with an attack probability of $\beta=1$ . (a) DSLC; (b) DRHRL. The robots under DSLC formed and maintained a stable rectangular formation, whereas the formation under DRHRL was disrupted by the attack.

Attack probabilities	Approaches	4 Robots		6 Robots
Attack probabilities	Approaches	Scenario (i)	Scenario (ii)	Scenario (i)	Scenario (ii)
$\beta=0.5$	DSLC	1.6	7.36	1.83	15.17
$\beta=0.5$	DRHRL [ 8 ]	1.97	8.46	5.09	20.47
$\beta=1$	DSLC	1.85	7.22	2.19	14.92
$\beta=1$	DRHRL [ 8 ]	3.22	8.27	7.3	16.12

TABLE II: Control Performance under Attacks with Different Probabilities for General Formation in the Experiments

Fig. 19: Trajectories and snapshots for affine formation with an attack probability of $\beta=0.5$ . (a) DSLC. (b) DRHRL. The robots under DSLC formed a stable rectangular formation and dynamically expanded or contracted it to avoid collisions, whereas the robots under DRHRL failed to avoid the obstacles due to the attack.

Fig. 20: Trajectories and snapshots for affine formation with an attack probability of $\beta=1$ . (a) DSLC. (b) DRHRL. The robots under DSLC formed a stable rectangular formation and dynamically expanded or contracted it to avoid collisions, whereas the robots under DRHRL failed to avoid the obstacles due to the attack.

Fig. 21: Trajectories and snapshots for containment with an attack probability of $\beta=0.5$ . (a) DSLC. (b) DRHRL. The robots under DSLC formed a stable containment shape and successfully reconfigured to accommodate a newly joined robot. In contrast, the follower robot under DRHRL failed to effectively enter the convex hull and maintain safe distances, resulting in collisions.

Fig. 22: Trajectories and snapshots for containment with an attack probability of $\beta=0.5$ . (a) DSLC. (b) DRHRL. The robots under DSLC formed a stable containment shape and successfully reconfigured to accommodate a newly joined robot. In contrast, the follower robot under DRHRL failed to effectively enter the convex hull and maintain safe distances, resulting in collisions.

Tasks	Approaches	$\beta=0.5$	$\beta=1$
Affine Formation	DSLC	4.41	4.06
Affine Formation	DRHRL [ 8 ]	6.54	6.31
Containment	DSLC	21.44	15.84
Containment	DRHRL [ 8 ]	31.66	40

TABLE III: Performance Assessment of Affine Formation and Containment under Different Attacks in the Experiments

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 23: The trajectories of six-robot formation with different random attack probabilities across robots: (a),

\vec{\beta}=(0.4,0.8,0.6,0.6,0.4,0.4)

; (b),

\vec{\beta}=(0.3,0.2,0.3,0.5,0.2,0.6)

; (c),

\vec{\beta}=(0.3,0.5,0.4,1,0.9,0.3)

; (d),

\vec{\beta}=(0.9,0.3,0.5,0.1,0.1,0.9)

, where

\vec{\beta}=(\beta_{1},\,\cdots,\,\beta_{6}).

Fig. 24: The trajectories of two-robot formation with different attack probabilities under DRHRL [ 8 ] , DRMPC [ 6 ] , and DSLC.

Fig. 25: The trajectories of four-robot formation with different attack probabilities under DRHRL [ 8 ] , DRMPC [ 6 ] , and DSLC.

Fig. 26: The trajectories of a six-robot formation illustrating the deployment of a defense policy trained under a specific attack probability to scenarios with different attack probabilities.

Robot Number	$\beta$	Average Computation Time (ms)
Robot Number	$\beta$	DSLC	DRMPC
2	0.1	$1.38$	$25.6$
	0.4	$1.53$	$16.3$
	0.7	$1.28$	$37.4$
4	0.2	$2.28$	$56.2$
	0.5	$2.00$	$54.2$

Appendix

TABLE IV: Average Computational Time Comparison between DSLC and DRMPC

Task	Robot Number	$\beta$	Completion Time (s)
Task	Robot Number	$\beta$	DSLC	DRHRL
Containment	5	0.5	25.02	36.11
Containment	5	1	25.02	34.28
Affine Formation	4	0.5	10.01	10.01
Affine Formation	4	1	10.02	10.02
General Formation	4 (Scenario I)	0.5	11.11	11.18

Appendix

TABLE V: Task Completion Time Comparison

Fig. 27: The trajectories of a six-robot formation under DSLC in the presence of a destabilizing attack policy, i.e.,

u_{a,i}=Kx_{i},\,\forall i\in\mathbb{N}_{1}^{M}

, where

K=[0.5,1,0.1,0.5;0.2,0.7,0.3,1]

is a randomly generated matrix.

Multi-Robot Systems

Multi-robot systems coordinate teams of robots to plan, perceive, communicate, and act together in shared tasks and environments.

See all 206 papers

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance
Apr 6, 2026Qintong Xie, Weishu Zhan, Peter ChinMulti-Robot Formation Control Model Predictive Control
Thayer School of Engineering, Dartmouth College, NH 03784. · Department of Computer Science, The University of Manchester, Manchester M13 9PL, United Kingdom.
Resilient Output Containment under Undisclosed Leader Dynamics and Actuator Attacks
Jun 25, 2026Mohammadreza Nematollahi, Khashayar Khorasani, Nader MeskinCyber-Physical Systems Multi-Agent Control
Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada. · Department of Electrical Engineering, Qatar University, Doha, Qatar.
Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks
Sep 28, 2026Theodore Rogalski, Shirantha WelikalaMulti-Agent Control Decentralized MARL
Electrical & Computer Engineering Department Stevens Institute of Technology Hoboken, USA

Apr 6, 2026J/K move · Enter open · S save

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance

Qintong Xie, Weishu Zhan, Peter Chin

Thayer School of Engineering, Dartmouth College, NH 03784. · Department of Computer Science, The University of Manchester, Manchester M13 9PL, United Kingdom.

Multi-Robot Formation Control Model Predictive Control Multi-Robot Motion Planning Control Barrier Functions

Open paper AskPDF arXiv

Multi-robot systems (MRS) are essential for large-scale applications such as disaster response, material transport, and warehouse logistics, yet ensuring robust, safety-aware formation control in cluttered and dynamic environments remains a major challenge. Existing model predictive control (MPC) approaches suffer from limitations in scalability and provable safety, while control barrier functions (CBFs), though principled for safety enforcement, are difficult to handcraft for large-scale nonlinear systems. This paper presents FORMULA, a safe distributed, learning-enhanced predictive control framework that integrates MPC with Control Lyapunov Functions (CLFs) for stability and neural network-based CBFs for decentralized safety, eliminating manual safety constraint design. This scheme maintains formation integrity during obstacle avoidance, resolves deadlocks in dense configurations, and reduces online computational load. Simulation results demonstrate that FORMULA enables scalable, safety-aware, formation-preserving navigation for multi-robot teams in complex environments.

Abstract

Multi-Robot Systems

Explore similar work

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance

Resilient Output Containment under Undisclosed Leader Dynamics and Actuator Attacks

Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks