GS3LAM: Gaussian Semantic Splatting SLAM

Linfei Li; Lin Zhang; Zhong Wang; Ying Shen

cs.CVMar 29, 2026

GS3LAM: Gaussian Semantic Splatting SLAM

Authors: Linfei Li, Lin Zhang, Zhong Wang, Ying Shen

Organizations: School of Software Engineering, Tongji University Shanghai, China · Department of Automation, Shanghai Jiaotong University Shanghai, China

Abstract

Recently, the multi-modal fusion of RGB, depth, and semantics has shown great potential in dense Simultaneous Localization and Mapping (SLAM). However, a prerequisite for generating consistent semantic maps is the availability of dense, efficient, and scalable scene representations. Existing semantic SLAM systems based on explicit representations are often limited by resolution and an inability to predict unknown areas. Conversely, implicit representations typically rely on time-consuming ray tracing, failing to meet real-time requirements. Fortunately, 3D Gaussian Splatting (3DGS) has emerged as a promising representation that combines the efficiency of point-based methods with the continuity of geometric structures. To this end, we propose GS3LAM, a Gaussian Semantic Splatting SLAM framework that processes multimodal data to render consistent, dense semantic maps in real-time. GS3LAM models the scene as a Semantic Gaussian Field (SG-Field) and jointly optimizes camera poses and the field via multimodal error constraints. Furthermore, a Depth-adaptive Scale Regularization (DSR) scheme is introduced to resolve misalignments between scale-invariant Gaussians and geometric surfaces. To mitigate catastrophic forgetting, we propose a Random Sampling-based Keyframe Mapping (RSKM) strategy, which demonstrates superior performance over common local covisibility optimization methods. Extensive experiments on benchmark datasets show that GS3LAM achieves increased tracking robustness, superior rendering quality, and enhanced semantic precision compared to state-of-the-art methods. Source code is available at https://github.com/lif314/GS3LAM.

Figures & tables

Figure 1. Our proposed GS 3 LAM utilizes the 3D semantic Gaussian representation and the differentiable splatting rasterization pipeline, and jointly optimizes camera poses and field for appearance, geometry and semantics, achieving robust tracking, real-time high-quality rendering, and precise 3D semantic reconstruction.

Panel of Figure 2 — Figure 2. Illustration of optimization bias on Replica “Office 3”.

Figure 3. The framework overview of GS 3 LAM. GS 3 LAM models the scene as a Semantic Gaussian Field (SG-Field). For geometric-semantic consistent keyframe mapping, an adaptive 3D Gaussian expansion technique and a Random Sampling-based Keyframe Mapping (RSKM) strategy are employed. GS 3 LAM optimizes camera poses and SG-Field using appearance, geometry, and semantics, along with a Depth-adaptive Scale Regularization (DSR) scheme.

Figure 4. The forgetting problem in SG-Field. During the incremental optimization process, Gaussians $\mathcal{G}_{A}$ in camera $A$ are initially optimized. However, when optimizing the Gaussians $\mathcal{G}_{B}$ in camera $B$ , the co-visible Gaussians $\mathcal{G}_{C}=\mathcal{G}_{A}\cap\mathcal{G}_{B}$ tend to be excessively fitted to the latest frame of camera $B$ , resulting in a decrease in the reconstruction quality of the previous frame captured by camera $A$ .

Figure 5. Qualitative comparison with SOTA methods on virtual Replica ( Straub et al., 2019 ) and real-world ScanNet ( Dai et al., 2017 ) datasets.

Method	Metric	0000	0059	0106	0169	0181	0207	Avg.
NICE-SLAM ( Zhu et al., 2022 )	PSNR $\uparrow$	18.71	16.55	17.29	18.75	15.56	18.38	17.54
	$\operatorname{SSIM}\uparrow$	0.641	0.605	0.646	0.629	0.562	0.646	0.621
	LPIPS $\downarrow$	0.561	0.534	0.510	0.534	0.602	0.552	0.548
Vox-Fusion ( Yang et al., 2022 )	PSNR $\uparrow$	19.06	16.38	18.46	18.69	16.75	19.66	18.17
	SSIM $\uparrow$	0.662	0.615	0.753	0.650	0.666	0.696	0.673
	LPIPS $\downarrow$	0.515	0.528	0.439	0.513	0.532	0.500	0.504

Table 1. Rendering performance on ScanNet ( Dai et al., 2017 ) .

Method	Metrics	R0	R1	R2	O0	O1	O2	O3	O4	Avg.
NICE- SLAM ( Zhu et al., 2022 )	PSNR $\uparrow$	22.12	22.47	24.52	29.07	30.34	19.66	22.23	24.94	24.42
	$\operatorname{SSIM}\uparrow$	0.689	0.757	0.874	0.874	0.886	0.797	0.801	0.856	0.809
	LPIPS $\downarrow$	0.330	0.271	0.208	0.229	0.181	0.235	0.209	0.198	0.233
Vox- Fusion ( Yang et al., 2022 )	PSNR $\uparrow$	22.39	22.36	23.92	27.79	29.83	20.33	23.47	25.21	24.41
	SSIM $\uparrow$	0.683	0.751	0.798	0.857	0.876	0.794	0.803	0.847	0.801
	LPIPS $\downarrow$	0.303	0.269	0.234	0.241	0.184	0.243	0.213	0.199	0.236

Table 2. Rendering performance on Replica ( Straub et al., 2019 ) .

Method	R0	R1	R2	O0	O1	O2	O3	O4	Avg.
NICE-SLAM ( Zhu et al., 2022 )	0.97	1.31	1.07	0.88	1.00	1.06	1.10	1.13	1.07
Vox-Fusion ( Yang et al., 2022 )	1.37	4.70	1.47	8.48	2.04	2.58	1.11	2.94	3.09
ESLAM ( Johari et al., 2023 )	0.71	0.70	0.52	0.57	0.55	0.58	0.72	0.63	0.63
Co-SLAM ( Wang et al., 2023 )	0.65	1.13	1.43	0.55	0.50	0.46	1.40	0.77	0.86
Point-SLAM ( Sandström et al., 2023 )	0.61	0.41	0.37	0.38	0.48	0.54	0.69	0.72	0.53
GS-SLAM ( Yan et al., 2024 )	0.48	0.53	0.33	0.52	0.41	0.59	0.46	0.70	0.50

Table 3. Tracking performance on Replica ( Straub et al., 2019 ) (ATE RMSE

\downarrow

[cm]).

Method	Room 0	Room 1	Room 2	Office 0	Avg.
NIDS SLAM ( Haghighi et al., 2023 )	82.45	84.08	76.99	85.94	82.37
DNS SLAM ( Li et al., 2023 )	88.32	84.90	81.20	84.66	84.77
SNI-SLAM ( Zhu et al., 2024b )	88.42	87.43	86.16	87.63	87.41
GS 3 LAM (Ours)	96.83	96.68	96.40	96.61	96.63

Table 4. Semantic reconstruction accuracy on Replica ( Straub et al., 2019 ) (mIoU

\uparrow

[%]).

Method	Metrics
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Depth [cm] $\downarrow$	ATE [cm] $\downarrow$	mIoU [%] $\uparrow$
w/o $M_{obs}$	19.63	0.720	0.493	13.31	43.12	30.23
w/o $M_{unobs}$	39.10	0.986	0.068	1.08	0.28	90.44
w/o DSR	40.04	0.990	0.059	0.85	0.25	93.96
w/o RSKM	37.48	0.983	0.081	1.23	0.29	89.13
Ours	41.21	0.993	0.046	0.41	0.21	97.35

Table 5. The ablation study on Replica “Office 1”.

Figure 6. The ablation study of DSR on Replica “Office 1”.

Figure 7. The ablation study of RSKM on Replica “Office 3”.

Figure 8. Illustration of optimization bias.

Method	Mapping /Iteration(ms)	Mapping /Frame(s)	Tracking /Iteration(ms)	Tracking /Frame(s)	Rendering (FPS)
NICE-SLAM ( Zhu et al., 2022 )	89	1.15	27	1.06	2.64
Vox-Fusion ( Yang et al., 2022 )	98	1.47	64	1.92	1.63
Point-SLAM ( Sandström et al., 2023 )	57	3.52	27	1.11	2.96
SplaTAM ( Keetha et al., 2024 )	83	4.94	70	2.82	59.91
GS 3 LAM (Ours)	55	4.29	89	3.01	106.56

Table 6. Runtime analysis on Replica “Office 0”.

Method	R0	R1	R2	O0	O1	O2	O3	O4	Avg.
SplaTAM ( Keetha et al., 2024 )	71.30	65.90	52.75	59.91	57.77	82.18	63.43	84.49	67.22
GS 3 LAM (Ours)	121.21	93.46	75.55	97.32	95.33	135.69	101.27	153.09	109.12

Table 7. Rendering speed (FPS

\uparrow

) on Replica ( Straub et al., 2019 ) .

Methods	Metrics	Room0	Room1	Room2	Office0	Office1	Office2	Office3	Office4	Avg.
SNI-SLAM ( Zhu et al., 2024b ) CVPR 24	PSNR [dB] $\uparrow$	25.91	28.17	29.15	33.86	30.34	29.10	29.02	29.87	29.43
	SSIM $\uparrow$	0.885	0.910	0.938	0.965	0.927	0.950	0.950	0.952	0.935
	LPIPS $\downarrow$	0.307	0.292	0.245	0.182	0.225	0.238	0.192	0.198	0.235
	mIoU [ $\%$ ] $\uparrow$	88.42	87.43	86.16	87.63	78.63	86.49	74.01	80.22	83.62
	ATE RMSE [cm] $\downarrow$	0.50	0.55	0.45	0.35	0.41	0.33	0.62	0.50	0.46
SGS-SLAM ( Li et al., 2024 ) arXiv 24	PSNR [dB] $\uparrow$	32.50	34.25	35.10	38.54	39.20	32.90	32.05	32.75	34.66

Table 8. The comparative analysis of our GS 3 LAM on Replica ( Straub et al., 2019 ) with concurrent semantic SLAM-related studies. Under competitive tracking accuracy, GS 3 LAM achieves state-of-the-art rendering quality and semantic reconstruction.

Method	0000	0059	0106	0169	0181	0207	Avg.
NICE-SLAM ( Zhu et al., 2022 )	12.00	14.00	7.90	10.90	13.40	6.20	10.70
Vox-Fusion ( Yang et al., 2022 )	68.84	24.18	8.41	27.28	23.30	9.41	26.90
Point-SLAM ( Sandström et al., 2023 )	10.24	7.81	8.65	22.16	14.77	9.54	12.19
SplaTAM ( Keetha et al., 2024 )	12.83	10.10	17.72	12.08	11.10	7.46	11.88
GS 3 LAM (Ours)	11.34	10.78	17.00	11.35	10.57	6.39	11.24

Table 9. Tracking performance on ScanNet ( Dai et al., 2017 ) (ATE RMSE

\downarrow

[cm]).

Panel of Figure 9 — Figure 9. Optimization bias on Replica ( Straub et al., 2019 ) . Our proposed RSKM strategy not only improves rendering quality (higher mean PSNR $\mu_{PSNR}$ ) but also enhances the global consistency of the map (lower PSNR variance $\sigma_{PSNR}$ ). The LCKM strategy employed in SplaTAM ( Keetha et al., 2024 ) exhibits lower PSNR in regions with high covisibility and frequent optimization iterations, thereby hindering model convergence in these areas. Conversely, in regions with fewer covisible frames, the reduced optimization iterations lead to under-optimized model, resulting in decreased PSNR.

Figure 10. The visualization of the semantic Gaussian fields constructed by our GS 3 LAM on the Replica ( Straub et al., 2019 ) and ScanNet ( Dai et al., 2017 ) datasets. GS 3 LAM demonstrates robust tracking capabilities and achieves real-time high-quality rendering at 109 FPS, along with precise 3D semantic reconstruction.

Figure 11. Semantic Gaussian field decoupling by our GS 3 LAM. GS 3 LAM is capable of real-time construction of 3D semantic maps that exhibit geometric, appearance, and semantic consistency, thereby enabling potential downstream real-time tasks.

Figure 12. More rendering results on Replica ( Straub et al., 2019 ) .

Figure 13. More rendering results on ScanNet ( Dai et al., 2017 ) .

Figure 14. Semantic rendering on Replica ( Straub et al., 2019 ) and ScanNet ( Dai et al., 2017 ) . It is noteworthy that on the ScanNet dataset, which contains real data with inaccurately annotated semantic labels, our GS 3 LAM exhibits the capability to achieve more precise segmentation results at object boundaries.

Explore similar work

Apr 14, 2026

RMGS-SLAM: Real-time Multi-sensor Gaussian Splatting SLAM

Achieving real-time Simultaneous Localization and Mapping (SLAM) based on 3D Gaussian splatting (3DGS) in large-scale real-world environments remains challenging, as existing methods still struggle to jointly achieve low-latency pose estimation, continuous 3D Gaussian reconstruction, and long-term global consistency. In this paper, we present a tightly coupled LiDAR-Inertial-Visual 3DGS-based SLAM framework for real-time pose estimation and photorealistic mapping in large-scale real-world scenes. The system executes state estimation and 3D Gaussian primitive initialization in parallel with global Gaussian optimization, enabling continuous dense mapping. To improve Gaussian initialization quality and accelerate optimization convergence, we introduce a cascaded strategy that combines feed-forward predictions with geometric priors derived from voxel-based principal component analysis. To enhance global consistency, we perform loop closure directly on the optimized global Gaussian map by estimating loop constraints through Gaussian-based Generalized Iterative Closest Point registration, followed by pose-graph optimization. We also collect challenging large-scale looped outdoor sequences with hardware-synchronized LiDAR-camera-IMU and ground-truth trajectories for realistic evaluation. Extensive experiments on both public datasets and our dataset demonstrate that the proposed method achieves a state of the art among real-time efficiency, localization accuracy, and rendering quality across diverse real-world scenes.

Dongen Li, Yi Liu, Junqi Liu +8

Dongen Li, Yi Liu, Zewen Sun, Zefan Huang, Chengran Yuan, Francis E.H. Tay, and Marcelo H. Ang Jr. are with the Advanced Robotics Centre, National University of Singapore. · Junqi Liu and Hongliang Guo are with the College of Computer Science, Sichuan University. · Shuo Sun is with Singapore-MIT Alliance for Research and Technology. +1

Simultaneous Localization and Mapping 3DGS SLAM 3DGS Acceleration

Ask PDF

Sep 7, 2026

LightSplat: Real-Time High-Fidelity 3D Gaussian SLAM with Loop Closure

SLAM systems based on 3D Gaussian Splatting (3DGS) have recently demonstrated promising reconstruction accuracy for dense 3D scene representations. However, current 3DGS systems struggle to meet the strict demands of real-world deployments due to severe limitations in operational performance and map adaptability. To this end, we propose LightSplat, a hybrid-representation RGB-D SLAM framework. It synergizes local sparse features for robust and fast tracking with a dual-thread backend that progressively constructs dense Gaussian submaps. Crucially, we enable online loop closure through feature-accelerated 3DGS registration, refining overall map consistency through pose graph optimization. Ultimately, LightSplat achieves the online reconstruction of high-fidelity Gaussian map. Extensive experiments on multiple datasets and real-world robotic platform demonstrate that our method achieves near state-of-the-art reconstruction quality and the capability to accommodate practical camera motions, maintaining an average framerate of 8 FPS. Overall, LightSplat provides an efficient and robust foundation for deploying high-fidelity 3DGS in real-world environments.

Junze Bao, Ye Gao, Yiming Huang +5

School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, P. R. China, and also with Zhejiang Key Laboratory of Industrial Big Data and Robot Intelligent Systems, Hangzhou Innovation Institute, Beihang University, Hangzhou 310051, P. R. China. · School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, P. R. China.

Simultaneous Localization and Mapping 3D Gaussian Splatting Streaming 3D Reconstruction 3DGS SLAM

Ask PDF

Jul 23, 2026

GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition

Existing Gaussian-splatting-based monocular Simultaneous Localization and Mapping (SLAM) systems are either tailored to short sequences, are not real-time, or suffer from prohibitive GPU memory requirements, limiting their applicability in realistic, long-horizon scenarios. To address this, we present GLAM-SLAM, a real-time, decoupled Gaussian-splatting SLAM system designed for large-scale outdoor scenes. We ensure lightweight tracking using a robust, feature-based SLAM frontend, while for mapping, we adopt a structured, sparse anchor grid representation that ensures scalable operation and maintains scene coherence across long-term sequences. To satisfy the dense initialization requirements of 3D Gaussian Splatting (3DGS), we introduce a geometry-based flow-densification anchoring strategy using epipolar constraints. Furthermore, by treating mapping as a multi-scene problem, we propose a scene-partitioning strategy that introduces a strong spatial inductive bias via MLP initializations to generate localized Gaussians. We evaluate our system on the challenging, long-sequence KITTI Odometry, Oxford RobotCar, and M'alaga datasets. Extensive ablations and comparisons demonstrate a 15% improvement in reconstruction quality over the second-best performer, while maintaining real-time performance and the ability to scale to longer sequences. Code is publicly available for the benefit of the community.

Panagiotis Mermigkas, Argyris Manetas, Petros Maragos

HERON - Hellenic Robotics Center of Excellence, Athena Research Center, Greece. · Institute of Robotics, Athena Research Center, Marousi, Greece. · Electrical and Computer Engineering, National Technical University of Athens, 15780 Zografou, Athens, Greece.

Simultaneous Localization and Mapping 3D Gaussian Splatting 3DGS SLAM 3D Reconstruction Gaussian Splatting

Ask PDF