RMGS-SLAM: Real-time Multi-sensor Gaussian Splatting SLAM
Authors: Dongen Li, Yi Liu, Junqi Liu, Zewen Sun, Zefan Huang, Shuo Sun, Jiahui Liu, Chengran Yuan, +3 more
Organizations: Dongen Li, Yi Liu, Zewen Sun, Zefan Huang, Chengran Yuan, Francis E.H. Tay, and Marcelo H. Ang Jr. are with the Advanced Robotics Centre, National University of Singapore. · Junqi Liu and Hongliang Guo are with the College of Computer Science, Sichuan University. · Shuo Sun is with Singapore-MIT Alliance for Research and Technology. · Jiahui Liu is with the School of Aeronautics and Astronautics, Shanghai Jiao Tong University.
Achieving real-time Simultaneous Localization and Mapping (SLAM) based on 3D Gaussian splatting (3DGS) in large-scale real-world environments remains challenging, as existing methods still struggle to jointly achieve low-latency pose estimation, continuous 3D Gaussian reconstruction, and long-term global consistency. In this paper, we present a tightly coupled LiDAR-Inertial-Visual 3DGS-based SLAM framework for real-time pose estimation and photorealistic mapping in large-scale real-world scenes. The system executes state estimation and 3D Gaussian primitive initialization in parallel with global Gaussian optimization, enabling continuous dense mapping. To improve Gaussian initialization quality and accelerate optimization convergence, we introduce a cascaded strategy that combines feed-forward predictions with geometric priors derived from voxel-based principal component analysis. To enhance global consistency, we perform loop closure directly on the optimized global Gaussian map by estimating loop constraints through Gaussian-based Generalized Iterative Closest Point registration, followed by pose-graph optimization. We also collect challenging large-scale looped outdoor sequences with hardware-synchronized LiDAR-camera-IMU and ground-truth trajectories for realistic evaluation. Extensive experiments on both public datasets and our dataset demonstrate that the proposed method achieves a state of the art among real-time efficiency, localization accuracy, and rendering quality across diverse real-world scenes.
Figures & tables
Fig. 1: Overview of the proposed system. The framework follows a four-module design. (i) A LIV front-end developed upon a Sequential IESKF system [ 4 ] first estimates ego-motion and produces time-synchronized poses, RGB images, projected depth images, and voxel-PCA geometric priors. (ii) These outputs are then used to initialize 3D Gaussian primitives through a cascaded strategy that combines feed-forward predictions with geometry-aware priors. (iii) The global Gaussian map is then optimized asynchronously under photometric, structural, and geometric constraints. (iv) To improve global consistency, loop closure is performed directly on the Gaussian representation via Gaussian GICP, and the resulting loop constraints are jointly optimized with odometry in a pose graph.
Fig. 2: Cascaded Gaussian Primitive Initialization. (i) Each point is projected into the previous and current loopframe images, and the closer valid view is used. (ii) On the predicted per-pixel Gaussian attribute maps, only the closest point is retained at each pixel and its attributes are bilinearly interpolated. (iii) Model-initialized points are transformed to the world frame and the rest are initialized from voxel-PCA priors or an isotropic heuristic.
Fig. 3: Qualitative comparison of rendering results. The blue boxed region is enlarged and displayed in the corner for detailed visual inspection.
Method
Metric
Lecture Center01
HKU Campus
Retail Street
CBD Building02
Driving1
Driving2
HKairport 03
HKisland 03
Gaussian-LIC2 [ 15 ]
PSNR ↑
31.21
27.06
24.64
23.16
22.09
23.72
14.84
17.65
SSIM ↑
0.91
0.76
0.77
0.77
0.76
0.81
0.36
0.44
LPIPS ↓
0.17
0.21
0.76
0.23
0.28
0.22
0.59
0.54
Real-time factor ↓
1.14
1.09
1.05
1.17
1.08
2.14
∞
∞
GS-LIVM [ 16 ]
PSNR ↑
21.48
12.06
17.34
13.57
16.17
14.86
×
2.96
SSIM ↑
0.75
0.42
0.51
0.51
0.55
0.56
×
0.16
TABLE I: Rendering quality and runtime efficiency comparison on selected datasets.
Method
Driving1
Driving2
HKairport03
HKisland03
FAST-LIVO2 [ 4 ]
3.55
2.86
1.41
2.74
Gaussian-LIC2 [ 15 ]
3.66
2.81
×
×
GS-LIVM [ 16 ]
4.86
2.95
×
65.26
MonoGS [ 9 ]
6.75
13.08
106.42
140.87
SplaTAM [ 10 ]
63.32
108.12
117.12
156.63
RMGS-SLAM (ours)
0.41
1.55
1.39
2.72
TABLE II: Pose accuracy comparison on different datasets.
Fig. 4: Trajectory comparison for the Driving1 sequence. The upper-left subfigure shows the overall 3D trajectory, and the others show local zoomed-in views at approximately 8.3× magnification.
Fig. 5: Runtime analysis across small (1, HKU Campus) and large (2, Driving1) squences. The subfigure (a) presents compute time and memory consumption of each module sampled at 0.5 s intervals. The subfigure (b) summarizes the corresponding time-weighted overall runtime percentage of each module.
Variant
HKU Campus(small)
Driving1(large)
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
w/o voxel-PCA & FFM
27.53
0.82
0.15
19.57
0.73
0.30
w/o FFM
29.77
0.85
0.17
21.81
0.76
0.29
w/o voxel-PCA
27.98
0.81
0.16
20.76
0.75
0.27
[2pt/2pt] w/ FFM (DepthSplat)
28.91
0.83
0.15
20.77
0.75
0.26
[2pt/2pt] w/o loop closure
30.88
0.88
0.11
19.82
0.74
0.30
TABLE III: Ablation study of different component choices.
School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, P. R. China, and also with Zhejiang Key Laboratory of Industrial Big Data and Robot Intelligent Systems, Hangzhou Innovation Institute, Beihang University, Hangzhou 310051, P. R. China. · School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, P. R. China.