Organizations: Chair of Information-oriented Control, Technical University of Munich, 80333 Munich, Germany · Chair of Robotics and Systems Intelligence, Technical University of Munich, 80992 Munich, Germany · Department of Electrical and Computer Engineering, University of California at Riverside, Riverside, CA 92521 USA
Ensuring the safe operation of multi-agent systems (MASs) under uncertain environments is crucial for cooperative robotic, where external disturbances and inaccurate dynamic models can significantly compromise performance and reliability. To address this challenge, calibrated machine learning models, particularly Gaussian process (GP) regression, are extensively employed due to their interpretable performance quantification. As the interconnected communication of MASs facilitates cooperative learning, agents are able to enhance learning performance by exchanging local GP inferences with their neighbors and aggregating the received information via distributed GP strategies. However, variations in computational power and prediction tasks among agents inevitably lead to heterogeneous computational delays and differences in query points, which are often overlooked in existing aggregation methods. To overcome these limitations, this work proposes an asynchronous cooperative learning strategy that explicitly accounts for prediction accuracy, query point variations and delay effects. Additionally, a distributed control law based on an adjoint MAS is developed to ensure the desired control performance. Simulations on unmanned surface vehicles validate the effectiveness of the proposed approach, demonstrating substantial improvements in both learning and control performance compared to the state-of-the-art approaches.
Figures & tables
Fig. 1 : Block diagram of the proposed distributed control framework with asynchronous cooperative online learning, where the blue lines indicate the signal to be transmitted to neighbors and the red lines shows the information received from neighbors. The dashed dot line infers the initialization step implemented only once at time t=0 .
Fig. 2 : Setting of the multi-USV system. (a) Generalized coordinate. (b) Communication topology G .
Fig. 3 : Aggregated prediction error ∥d(x(t))−d^(t)∥ and the prediction error bound ∥η~(t)∥ in Theorem 1 .
Fig. 4 : Maximal consensus error ∥qi(t)−qj(t)∥ and velocity error ∥q˙i(t)∥ for all i,j∈V with i=j .
Fig. 5 : Aggregated prediction error ∥d(x(t))−d^(t)∥ w.r.t time t from 7 cooperative learning-based control methods. The solid line represents the mean value among 100 Monte Carlo tests, and the shadowed area denotes the variances.
Fig. 6 : Maximal consensus error ∥qi(t)−qj(t)∥ between any two agents i,j∈V with i=j and maximal velocity error ∥q˙i(t)∥ for any agent i∈V over time t using different cooperative learning-based control strategies. The solid line represents the mean value among 100 Monte Carlo tests, and the shadowed area denotes the variances.
Fig. 7 : Maximal aggregated prediction error ∥d(x(t))−d^(t)∥ in time domain t∈[10,30] for different cooperative learning-based control strategy, which is summarized from 100 times Monte Carlo tests.
Fig. 8 : Maximal consensus error ∥qi(t)−qj(t)∥ and velocity error ∥q˙i(t)∥ for all i,j∈V with i=j in time domain t∈[10,30] for different cooperative learning-based control strategy, which is from 100 times Monte Carlo tests.
Multi-agent control coordinates interacting autonomous agents to achieve shared goals while respecting safety, communication, and resource constraints.
We propose DistGP: a multi-robot learning method for collaborative learning of a global function using only local experience and computation. We utilise a sparse Gaussian process (GP) model with a factorisation that mirrors the multi-robot structure of the task, and admits distributed training via Gaussian belief propagation (GBP). Our loopy model outperforms Tree-Structured GPs \cite{bui2014tree} and can be trained online and in settings with dynamic connectivity. We show that such distributed, asynchronous training can reach the same performance as a centralised, batch-trained model, albeit with slower convergence. Last, we compare to DiNNO \cite{yu2022dinno}, a distributed neural network (NN) optimiser, and find DistGP achieves superior accuracy, is more robust to sparse communication and is better able to learn continually.
Seth Nabarro, Mark van der Wilk, Andrew J. Davison
Dyson Robotics Lab, Imperial College London · Department of Computer Science, University of Oxford
In this work, we propose a framework that combines multi-agent reinforcement learning (MARL) with model-based control to achieve safe, dynamically feasible actions in cooperative multi-agent tasks. Multi-agent reinforcement learning provides the advantage of learning cooperative policies for multi-agent teams from discrete non-differentiable rewards in a long planning horizon. Model-predictive control is robust and offers safe, dynamically feasible actions in a fast replanning framework for short horizons. We propose an algorithm that extends actor-critic model predictive control for MARL which we refer to as multi-agent actor-critic model predictive control (MA-AC-MPC). We demonstrate the capabilities of this algorithm by applying it to a multi-agent pursuit-evasion scenario. Specifically, we compare the evader team's strategy using the MA-AC-MPC model and a multi-layer perceptron model (MA-AC-MLP). The pursuer team uses augmented proportional navigation as it is accepted as an advanced adversarial control law. We also provide an example with a heterogeneous environment where a drone and omni-wheeled rover cooperate to achieve repeatable and successful landing with 100% success rate in hardware for MA-AC-MPC compared to 60% for MA-AC-MLP. We demonstrate the robustness of the proposed MA-AC-MPC algorithm in hardware for both environments.
Christian Llanes, Spencer W. Jensen, Samuel Coogan
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA. · Sandia National Laboratories, Albuquerque, NM 87123, USA.
Distributed trajectory estimation arises in many applications across robotics, but existing implementations typically do not consider asynchrony in agents' communications and computations. Therefore, we propose an asynchronous block coordinate descent algorithm for distributed trajectory estimation. We consider a team of agents that observes a team of robots and estimates the robots' states over a sliding window. The agents solve an approximation of the maximum a posteriori estimation problem, which we derive. We show this approximation introduces negligible errors and eliminates up to 96.9% of communications among agents. Next, we prove that agents' iterates converge exponentially fast to the optimal estimate of the robots' states. Simulations show that this approach has up to 64% less error than a comparable state-of-the-art algorithm. Experiments on mobile robots show this approach is robust to delays whose lengths span three orders of magnitude.
Adam Pooley, Matthew Hale
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA