Edge-Assisted Multi-Robot Visual-Inertial SLAM with Efficient Communication
Authors: Xin Liu, Shuhuan Wen, Jing Zhao, Tony Z. Qiu, Hong Zhang
Organizations: the Department of Key Lab of Industrial Computer Control Engineering of Hebei Province, Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, 066004, China. · Intelligent Transport System Research Center, Wuhan University of Technology, Wuhan, China · the Department of Civil and Environmental Engineering at the University of Alberta, Canada. · the Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China.
The integration of cloud computing and edge computing is an effective way to achieve global consistent and real-time multi-robot Simultaneous Localization and Mapping (SLAM). Cloud computing effectively solves the problem of limited computing, communication and storage capacity of terminal equipment. However, limited bandwidth and extremely long communication links between terminal devices and the cloud result in serious performance degradation of multi-robot SLAM systems. To reduce the computational cost of feature tracking and improve the real-time performance of the robot, a lightweight SLAM method of optical flow tracking based on pyramid IMU prediction is proposed. On this basis, a centralized multi-robot SLAM system based on a robot-edge-cloud layered architecture is proposed to realize real-time collaborative SLAM. It avoids the problems of limited on-board computing resources and low execution efficiency of single robot. In this framework, only the feature points and keyframe descriptors are transmitted and lossless encoding and compression are carried out to realize real-time remote information transmission with limited bandwidth resources. This design reduces the actual bandwidth occupied in the process of data transmission, and does not cause the loss of SLAM accuracy caused by data compression. Through experimental verification on the EuRoC dataset, compared with the current most advanced local feature compression method, our method can achieve lower data volume feature transmission, and compared with the current advanced centralized multi-robot SLAM scheme, it can achieve the same or better positioning accuracy under low computational load.
Figures & tables
Fig. 1: Multi-robot Collaborative SLAM communication architecture.
Fig. 2: System overview. Each robot is equipped with a Visual-Inertial sensor suite. After analyzing the sensor information, the compressed data is transmitted to a fixed edge server. The edge server can conduct location and map construction in local scenes by running the VIO in real-time and independently. The communication interface is used to exchange data (keyframe local binary features and mappoints) between the edge server and the cloud. The cloud can run computationally expensive, global, and non-real-time tasks: redundant keyframe culling, loop closure, and global map fusion.
Fig. 3: IMU pre-integration diagram. Integrating all IMU information between the kth frame and the (k+1)th frame, the position, velocity and rotation (PVQ) of the (k+1)th frame can be obtained as the initial value of the visual estimation.
Fig. 4: The choice of encoding mode. The decision of keyframes depends on the number of points tracked by the IMU-assisted LK optical flow and the time interval from the previous keyframe.
Fig. 6: Visual-Inertial joint optimization factor graph in the cloud.
Fig. 7: The tracking results of traditional LK optical flow tracking and IMU-assisted optical flow tracking on EuRoC. Raw image data (a), traditional LK optical flow tracking results without IMU participation (b), IMU-assisted LK optical flow tracking results (c).
Fig. 8: The number of points tracked and the track time spent. The blue bar represents the number of points tracked by LK optical flow method, and the orange bar represents the number tracked by the proposed. The purple broken line represents the time spent in LK tracking and the green broken line represents the time spent in the proposed tracking.
Methods
Intra-Coding in [ 58 ]
The Proposed
KF
Non-KF
Memory
217 bits
217 bits
27 bits
TABLE I: Comparison of the memory required for a feature point
Fig. 9: The proportion of each part of each feature in different methods under lossless compression.
Sequences
MH_01-MH_03
MH_01-MH_05
V1_01-V1_03
V2_01-V2_03
Method
CCM-SLAM [ 49 ]
0.077
-
-
-
VINS-mono [ 33 ]
-
0.210
-
-
ORB-SLAM3 [ 15 ]
0.037
0.065
0.040
0.048
CVIDS [ 51 ]
-
0.074
-
-
COVINS AWS [ 11 ]
0.025
0.039
0.049
-
TABLE II: RMSE of ATE in units of on the EuRoC dataset. Our method is to calculate the average after five runs. Note that the bold means the best, and ”-” means that the result of the corresponding method cannot be found in the literature.
Methods
ATE/m
Number of Keyframes
Number of Mappoints
The Proposed (w/o MBP)
0.051
1550
57169
The Proposed (w/ MBP)
0.046
925
41923
TABLE III: Comparison of results using different redundant keyframe removal methods.
Fig. 10: The results of collaborative localization and mapping in the V2 sequences of the proposed method.
Fig. 11: The trajectories of collaborative localization and mapping in the V2 sequences of the proposed method.
Fig. 12: The experimental scene and the equipments.
Fig. 13: The results of collaborative localization and mapping in the real scene of the proposed method.
Fig. 14: Comparison of measured trajectory and groundtruth in real scene.
Fig. 15: Trajectory overlaid with 2D grid map of the real scene for visual comparison.
Figure 19
Shuhuan Wen was received the PhD degree in control theory and control engineering from the Yanshan University, Qinhuangdao, China in 2005. She is currently a Professor of automatic control in the Department of Electric Engineering, Yanshan University. She has coauthored one book, about 40 papers. Her research interests include SLAM, computer vision, robotics, 3D object recognization and reconstruction. Dr. Wen was a Visiting Scholar of the Ottawa University, Carleton University and Simon Fraser University in Canada in 2011-2013. She is also a visiting professor at University of Alberta in Canada in 2021-2022. She is an AE of IROS in 2021 to 2022.
Table 20
Figure 21Figure 22
Hong Zhang was received the B.Sc degree from Northeastern University, Boston, USA, in 1982, and the PH.D.degree from Purdue University. West Lafayette, IN, USA, in 1986, both in electrical engineering. He conducted post-doctoral research at the University of Pennsylvania from 1986 to 1987 before joining the Department of Computing Science, University of Alberta, Canada. He is an NSERC Industrial Research Chair. His current research interests include robotics computer vision, and image processing.
Collaborative Simultaneous Localization and Mapping (C-SLAM) is a fundamental capability for multi-robot teams as it enables downstream tasks like planning and navigation. However, existing C-SLAM back-end algorithms that are required to solve this problem struggle to address the practical realities of real-world deployments (i.e. communication limitations, outlier measurements, and online operation). In this paper we propose Robust Incremental Manifold Edge-based Separable ADMM (riMESA) -- a robust, incremental, and distributed C-SLAM back-end that is resilient to outliers, reliable in the face of limited communication, and can compute accurate state estimates for a multi-robot team in real-time. Through the development of riMESA, we, more broadly, make an argument for the use of Consensus Alternating Direction Method of Multipliers as a theoretical foundation for distributed optimization tasks in robotics like C-SLAM due to its flexibility, accuracy, and fast convergence. We conclude this work with an in-depth evaluation of riMESA on a variety of C-SLAM problem scenarios and communication network conditions using both synthetic and real-world C-SLAM data. These experiments demonstrate that riMESA is able to generalize across conditions, produce accurate state estimates, operate in real-time, and outperform the accuracy of prior works by a factor >7x on real-world datasets.
Daniel McGann, Michael Kaess
The authors are with the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
We introduce a data-efficient distributed Simultaneous Localization and Mapping (SLAM) framework for multi-robot operation in GPS-denied and communication-constrained environments. The framework is designed for teams of robots equipped with LiDAR, cameras, and inertial sensors, where robots may operate from independently initialized local coordinate frames and share only limited spatial overlap. Our approach constructs a scene graph by using fused RGB-LiDAR point clouds to generate both a semantically segmented point cloud layer, and a layer of discrete bounded objects, to accompany estimated robot trajectories. Scene graph matching is performed collaboratively through exchanging and matching object data with neighboring robots. To maximize communication efficiency, we utilize a multi-step data exchange and optimization process. We demonstrate the effectiveness and communication efficiency of the proposed approach using real-world datasets containing limited trajectory overlap, viewpoint variation, repeated structures, and degraded visual conditions.
Yewei Huang, Tixiao Shan, Abhinav Rajvanshi +4
Dartmouth College, 100 N Main St, Hanover, NH 03755, USA · SRI International, 201 Washington Rd, Princeton, NJ 08540, USA · Stevens Institute of Technology, 1 Castle Point Terrace, Hoboken, NJ 07030, USA
Multi-robot simultaneous localization and mapping (SLAM) is a fundamental task in multi-robot operations. Robots must have a common understanding of their location and that of their team members to complete coordinated actions. However, multi-robot SLAM between Uncrewed Surface Vessels (USVs) and Autonomous Underwater Vehicles (AUVs) has primarily been achieved through acoustic pinging between robots to retrieve range measurements; a measurement technique requires that robots to be in similar locations simultaneously, have an uninterrupted path for signal propagation, and may necessitate synchronized clocks. This is especially challenging in complex, cluttered maritime environments, where structures may impede signals. However, these same structures may be observable above and below the water's surface, presenting an opportunity for inter-robot SLAM loop closure between USV and AUV data streams. This work builds upon recent research on inter-robot SLAM loop closure between USV and AUV data, extending it to propose a centralized multi-robot SLAM system. Each robot performs its state estimation, and we detect loop closures between each AUV and the USV data. These inter-robot loop closures are used to merge each robot's state estimate into a centralized graph, yielding estimates for the whole time history of the USV and all AUVs in the system. Validation is performed using real-world perceptual data in three different environments. Results show improved errors for AUVs in the multi-robot SLAM system compared to single-robot SLAM over the same trajectories. To our knowledge, this is the first instance of a multi-robot SLAM system with AUVs and USVs built on loop closures rather than acoustic distance measurements.
John McConnell, Armon Shariati, Paul Szenher +1
United States Naval Academy, Annapolis, MD, USA · Shield-AI, San Diego CA, USA · Stevens Institute of Technology, Hoboken, NJ, USA