M^3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM
Authors: Kerui Ren, Guanghao Li, Changjian Jiang, Yingxiang Xu, Tao Lu, Linning Xu, Junting Dong, Jiangmiao Pang, Mulin Yu, Bo Dai
Organizations: Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University · Shanghai Innovation Institute · Zhejiang University · Beijing Institute of Technology · The Chinese University of Hong Kong · The University of Hong Kong
Streaming reconstruction from uncalibrated monocular video remains challenging, as it requires both high-precision pose estimation and computationally efficient online refinement in dynamic environments. While coupling 3D foundation models with SLAM frameworks is a promising paradigm, a critical bottleneck persists: most multi-view foundation models estimate poses in a feed-forward manner, yielding pixel-level correspondences that lack the requisite precision for rigorous geometric optimization. To address this, we present M^3, which augments the Multi-view foundation model with a dedicated Matching head to facilitate fine-grained dense correspondences and integrates it into a robust Monocular Gaussian Splatting SLAM. M^3 further enhances tracking stability by incorporating dynamic area suppression and cross-inference intrinsic alignment. Extensive experiments on diverse indoor and outdoor benchmarks demonstrate state-of-the-art accuracy in both pose estimation and scene reconstruction. Notably, M^3 reduces ATE RMSE by 64.3% compared to VGGT-SLAM 2.0 and outperforms ARTDECO by 2.11 dB in PSNR on the ScanNet++ dataset.
Figures & tables
Figure 1: We demonstrate the execution of our M 3 pipeline on a challenging manor sequence. Our approach delivers robust, high-precision pose estimation while achieving high-fidelity scene reconstruction from monocular video sequences. Please refer to our project page for more demos: https://city-super.github.io/M3/
Figure 2: The M 3 Pipeline. The framework consists of joint tracking and global optimization for pose estimation and a mapper for scene reconstruction. For monocular sequences, Pi3X processes retrieved historical keyframes and new frames in one inference to facilitate factor graph construction and keyframe selection. Following the Neural Gaussian and LOD architecture of ARTDECO [ 19 ] , Gaussians are initialized via Laplacian norm and optimized jointly with camera poses.
Dataset
DROID-SLAM
MASt3R-SLAM
VGGT-SLAM
VGGT-SLAM 2.0
ARTDECO
Ours
Pi3X
ScanNet++
0.623
0.346
1.081
0.182
0.137
0.065
-
ScanNetV2
0.217
0.106
0.852
0.073
0.141
0.051
-
Waymo
7.535
16.610
13.166
1.295
2.410
0.773
0.827
KITTI
8.212
18.073
10.985
2.521
2.938
0.890
1.441
Table 1: Tracking comparisons in terms of ATE RMSE (m). We compare against state-of-the-art SLAM frameworks on both indoor and outdoor datasets. The best and second-best results are highlighted in bold and underline , respectively.
Indoor-dataset
ScanNet++
ScanNetV2
VR-NeRF
Efficiency
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
#GS
Time
MonoGS
18.56
0.732
0.582
21.88
0.827
0.508
14.69
0.565
0.728
25.2k
10.2 min
S3PO-GS
22.60
0.812
0.457
24.05
0.850
0.439
22.10
0.766
0.528
29.9k
17.1 min
HI-SLAM2
23.89
0.811
0.342
23.75
0.820
0.295
28.25
0.868
0.277
152.1k
6.9 min
OnTheFly-NVS
22.00
0.808
0.336
23.38
0.861
0.348
28.23
0.888
0.257
210.5k
1.4 min
ARTDECO
26.71
0.864
0.246
26.03
0.902
0.278
29.15
0.883
0.231
936.7k
5.1 min
Table 2: Rendering comparisons across indoor and outdoor datasets. We report visual quality metrics, the number of Gaussians, and average training time. The best and second-best results are highlighted in bold and underline , respectively.
Figure 3: Qualitative comparisons of rendering against on-the-fly reconstruction baselines across diverse datasets. M 3 preserves high-fidelity rendering details in challenging environments, particularly in regions highlighted by white rectangles.
Method
Ours
Depth-Anything-3
AnySplat
Dataset
PSNR ↑
SSIM ↑
LPIPS ↓
GPU ↓
PSNR ↑
SSIM ↑
LPIPS ↓
GPU ↓
PSNR ↑
SSIM ↑
LPIPS ↓
GPU ↓
ScanNet++
27.789
0.873
0.211
22.4G
17.973
0.666
0.441
105.5G
22.337
0.757
0.236
94.6G
Waymo
28.346
0.870
0.273
23.2G
21.760
0.701
0.338
82.0G
21.974
0.729
0.341
73.1G
Table 3: Comparisons against feed-forward Gaussian Splatting methods. We evaluate reconstruction quality and efficiency on fixed-length sequences. Our optimization-based approach consistently outperforms SOTA feed-forward models.
Figure 4: Qualitative comparisons of rendering against feed-forward Gaussian Splatting methods. M 3 consistently preserves fine-grained visual details.
Figure 5: Effect of the Motion Map in dynamic environments. By detecting dynamic regions, the Motion Map enables moving objects to be excluded from the static reconstruction, thereby improving structural consistency in dynamic scenes.
Metrics
Full
w/o descriptor
w/o intr. align
w/o global opt.
pi3x→pi3
split front&back-end
L =16
ATE ↓
0.065
0.094
0.236
0.130
0.068
0.098
0.077
PSNR ↑
28.82
27.73
26.14
28.49
28.77
28.41
28.57
SSIM ↑
0.892
0.874
0.851
0.886
0.892
0.886
0.888
LPIPS ↓
0.190
0.221
0.258
0.201
0.191
0.203
0.199
Table 4: Ablation studies on the ScanNet++ dataset evaluating the effect of key design choices and hyperparameters.
Method
Ours (Pi3X)
Depth-Anything-3
VGGT
Dataset
ATE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
PSNR ↑
SSIM ↑
LPIPS ↓
ScanNet++
0.065
28.82
0.892
0.190
0.079
28.09
0.882
0.211
0.106
27.33
0.869
0.234
Table 5: Comparison of Multi-view Foundation Models within the M 3 Framework. We evaluate the performance of different backbones on the ScanNet++ dataset [ 48 ] . Pi3X [ 41 ] and Depth-Anything-3 [ 22 ] represent the state-of-the-art in geometric priors, with Pi3X achieving the highest accuracy due to its superior cross-inference consistency.
Figure 6: Qualitative comparisons of camera trajectories against on-the-fly reconstruction baselines across diverse datasets. Our visualized trajectories align most closely with the ground truth (GT), exhibiting no significant divergence or inconsistency.
Figure 7: Qualitative comparisons of rendering against on-the-fly reconstruction baselines in outdoor scenes. Our rendered outputs exhibit superior clarity and are devoid of visual artifacts.
Figure 8: Qualitative comparisons of rendering against on-the-fly reconstruction baselines in indoor scenes. Our rendered outputs exhibit superior clarity and are devoid of visual artifacts.
Gaussian splatting has recently gained traction as a compelling map representation for SLAM systems, enabling dense and photo-realistic scene modeling. However, its application to monocular SLAM remains challenging due to the lack of reliable geometric cues from monocular input. Without geometric supervision, mapping or tracking could fall in local-minima, resulting in structural degeneracies and inaccuracies. To address this challenge, we propose GaussianFlow SLAM, a monocular 3DGS-SLAM that leverages optical flow as a geometry-aware cue to guide the optimization of both the scene structure and camera poses. By encouraging the projected motion of Gaussians, termed GaussianFlow, to align with the optical flow, our method introduces consistent structural cues to regularize both map reconstruction and pose estimation. Furthermore, we introduce normalized error-based densification and pruning modules to refine inactive and unstable Gaussians, thereby contributing to improved map quality and pose accuracy. Experiments conducted on public datasets demonstrate that our method achieves superior rendering quality and tracking accuracy compared with state-of-the-art algorithms. The source code is available at: https://github.com/url-kaist/gaussianflow-slam.
Dong-Uk Seo, Jinwoo Jeon, Eungchang Mason Lee +1
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon 34141, Republic of Korea
Recent advances in geometric foundation models have emerged as a promising alternative for addressing the challenge of dense reconstruction in monocular visual simultaneous localization and mapping (SLAM). Although geometric foundation models enable SLAM to leverage variable input views, the previous methods remain confined to two-view pairs or fixed-length inputs without sufficient deliberation of geometric context for view selection. To tackle this problem, we propose AIM-SLAM, a dense monocular SLAM framework that exploits an adaptive and informative multi-view keyframe prioritization with dense pointmap predictions from visual geometry grounded transformer (VGGT). Specifically, we introduce the selective information- and geometric-aware multi-view adaptation (SIGMA) module, which employs voxel overlap and information gain to retrieve a candidate set of keyframes and adaptively determine its size. Furthermore, we formulate a joint multi-view Sim(3) optimization that enforces consistent alignment across selected views, substantially improving pose estimation accuracy. The effectiveness of AIM-SLAM is demonstrated on real-world datasets, where it achieves state-of-the-art pose estimation performance and accurate dense reconstruction results. Our system supports ROS integration, with code is available at https://aimslam.github.io/.
Jinwoo Jeon, Dong-Uk Seo, Eungchang Mason Lee +1
The School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Republic of Korea · KAIST InnoCORE LLM, KAIST, Daejeon, 34141, Republic of Korea
Monocular 3D Gaussian Splatting SLAM suffers from critical limitations in time efficiency, geometric accuracy, and multi-view consistency. These issues stem from the time-consuming Train-from-Scratch optimization and the lack of inter-frame scale consistency from single-frame geometry priors. We contend that a feed-forward paradigm, leveraging multi-frame context to predict Gaussian attributes directly, is crucial for addressing these challenges. We present Flash-Mono, a system composed of three core modules: a feed-forward prediction frontend, a 2D Gaussian Splatting mapping backend, and an efficient hidden-state-based loop closure module. We trained a recurrent feed-forward frontend model that progressively aggregates multi-frame visual features into a hidden state via cross attention and jointly predicts camera poses and per-pixel Gaussian properties. By directly predicting Gaussian attributes, our method bypasses the burdensome per-frame optimization required in optimization-based GS-SLAM, achieving a 10x speedup while ensuring high-quality rendering. The power of our recurrent architecture extends beyond efficient prediction. The hidden states act as compact submap descriptors, facilitating efficient loop closure and global Sim(3) optimization to mitigate the long-standing challenge of drift. For enhanced geometric fidelity, we replace conventional 3D Gaussian ellipsoids with 2D Gaussian surfels. Extensive experiments demonstrate that Flash-Mono achieves state-of-the-art performance in both tracking and mapping quality, highlighting its potential for embodied perception and real-time reconstruction applications. Project page: https://victkk.github.io/flash-mono.
Zicheng Zhang, Ke Wu, Xiangting Meng +3
Fudan University · ShanghaiTech University · Shanghai Jiao Tong University