SGAD-SLAM: Splatting Gaussians at Adjusted Depth for Better Radiance Fields in RGBD SLAM
Organizations: Machine Perception Lab, Wayne State University, Detroit, USA
Abstract
3D Gaussian Splatting (3DGS) has made remarkable progress in RGBD SLAM. Current methods usually use 3D Gaussians or view-tied 3D Gaussians to represent radiance fields in tracking and mapping. However, these Gaussians are either too flexible or too limited in movements, resulting in slow convergence or limited rendering quality. To resolve this issue, we adopt pixel-aligned Gaussians but allow each Gaussian to adjust its position along its ray to maximize the rendering quality, even if Gaussians are simplified to improve system scalability. To speed up the tracking, we model the depth distribution around each pixel as a Gaussian distribution, and then use these distributions to align each frame to the 3D scene quickly. We report our evaluations on widely used benchmarks, justify our designs, and show advantages over the latest methods in view rendering, camera tracking, runtime, and storage complexity. Please see our project page for code and videos at https://machineperceptionlab.github.io/SGAD-SLAM-Project .
Figures & tables
| Method | Rm0 | Rm1 | Rm2 | Off0 | Off1 | Off2 | Off3 | Off4 | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | |||||||||
| NICE-SLAM [ 103 ] | 1.69 | 2.04 | 1.55 | 0.99 | 0.90 | 1.39 | 3.97 | 3.08 | 1.95 |
| DF-Prior [ 26 ] | 1.39 | 1.55 | 2.60 | 1.09 | 1.23 | 1.61 | 3.61 | 1.42 | 1.81 |
| Vox-Fusion [ 82 ] | 0.27 | 1.33 | 0.47 | 0.70 | 1.11 | 0.46 | 0.26 | 0.58 | 0.65 |
| ESLAM [ 33 ] | 0.71 | 0.70 | 0.52 | 0.57 | 0.55 | 0.58 | 0.72 | 0.63 | 0.63 |
| Point-SLAM [ 59 ] | 0.61 | 0.41 | 0.37 | 0.38 | 0.48 | 0.54 | 0.72 | 0.63 | 0.52 |
| Method | fr1/desk | fr2/xyz | fr3/office | Avg. |
|---|---|---|---|---|
| Neural Implicit Fields | ||||
| NICE-SLAM [ 103 ] | 4.3 | 31.7 | 3.9 | 13.3 |
| Vox-Fusion [ 82 ] | 3.5 | 1.5 | 26.0 | 10.3 |
| Point-SLAM [ 59 ] | 4.3 | 1.3 | 3.5 | 3.0 |
| Loopy-SLAM [ 41 ] | 3.8 | 1.6 | 3.4 | 2.9 |
| 3D Gaussian Splatting | ||||
| Method | SplaTAM [ 34 ] | Gaussian- SLAM [ 87 ] | VTGS- SLAM [ 27 ] | Loop Splat [ 102 ] | Ours |
|---|---|---|---|---|---|
| Training views | 26.71 | 30.38 | 32.22 | 30.20 | 36.73 |
| Novel views | 17.82 | 21.27 | 21.46 | 21.30 | 22.32 |
| Tracking /Frame(s) | Mapping /Frame(s) | Total /Frame(s) | NUM1 | NUM2 | |
| NICE-SLAM [ 103 ] | 1.06 | 1.15 | 2.21 | - | - |
| Point-SLAM [ 59 ] | 1.11 | 3.52 | 4.63 | - | - |
| SplaTAM [ 34 ] | 2.70 | 4.89 | 7.59 | 5832 | 5832 |
| Gaussian-SLAM [ 87 ] | 0.83 | 0.93 | 1.76 | 32592 | 1983 |
| GS-ICP SLAM [ 20 ] | 0.03 | 1.02 | 1.05 | 1544 | 1544 |
| Ours | 0.01 | 0.89 | 0.90 | 326400 | 816 |
| Mode1 | Mode2 | Mode3 | Ours | |
|---|---|---|---|---|
| Num of Gaussians | 4450 | 205 | 205 | 205 |
| Num of attributes | 14 | 14 | 5 | 6 |
| PSNR | 20.90 | 37.67 | 37.16 | 38.60 |
| SSIM | 0.771 | 0.995 | 0.995 | 0.997 |
| LPIPS | 0.216 | 0.017 | 0.019 | 0.012 |
| 10 | 20 | 30 | 40 | Ours | |
|---|---|---|---|---|---|
| PSNR | 39.86 | 39.86 | 39.86 | 39.86 | 39.86 |
| SSIM | 0.997 | 0.997 | 0.997 | 0.997 | 0.997 |
| LPIPS | 0.013 | 0.013 | 0.013 | 0.013 | 0.012 |
| ATE RMSE | 2.1 | 2.1 | 2.2 | 2.2 | 2.0 |
| Point2point w/o scale(GICP) | Point2point w/ Ellipse scale | Point2surf w/o scale | Point2surf w/ Plane scale | Ours | |
|---|---|---|---|---|---|
| ATE RMSE | 106.1 | 17.0 | 104.6 | 27.0 | 2.0 |
| Method | Metric | Rm0 | Rm1 | Rm2 | Off0 | Off1 | Off2 | Off3 | Off4 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | ||||||||||
| NICE-SLAM [ 103 ] | PSNR | 22.12 | 22.47 | 24.52 | 29.07 | 30.34 | 19.66 | 22.23 | 24.94 | 24.42 |
| SSIM | 0.689 | 0.757 | 0.814 | 0.874 | 0.886 | 0.797 | 0.801 | 0.856 | 0.809 | |
| LPIPS | 0.330 | 0.271 | 0.208 | 0.229 | 0.181 | 0.235 | 0.209 | 0.198 | 0.233 | |
| Vox-Fusion [ 82 ] | PSNR | 22.39 | 22.36 | 23.92 | 27.79 | 29.83 | 20.33 | 23.47 | 25.21 | 24.41 |
| SSIM | 0.683 | 0.751 | 0.798 | 0.857 | 0.876 | 0.794 | 0.803 | 0.847 | 0.801 | |
| Method | Metric | Rm0 | Rm1 | Rm2 | Off0 | Off1 | Off2 | Off3 | Off4 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | ||||||||||
| NICE-SLAM [ 103 ] | Depth L1 [cm] | 1.81 | 1.44 | 2.04 | 1.39 | 1.76 | 8.33 | 4.99 | 2.01 | 2.97 |
| F1 | 45.0 | 44.8 | 43.6 | 50.0 | 51.9 | 39.2 | 39.9 | 36.5 | 43.9 | |
| Vox-Fusion [ 82 ] | Depth L1 [cm] | 1.09 | 1.90 | 2.21 | 2.32 | 3.40 | 4.19 | 2.96 | 1.61 | 2.46 |
| F1 | 69.9 | 34.4 | 59.7 | 46.5 | 40.8 | 51.0 | 64.6 | 50.7 | 52.2 | |
| ESLAM [ 33 ] | Depth L1 [cm] | 0.97 | 1.07 | 1.28 | 0.86 | 1.26 | 1.71 | 1.43 | 1.06 | 1.18 |
| Method | Metric | fr1/desk | fr2/xyz | fr3/office | Avg. |
|---|---|---|---|---|---|
| Neural Implicit Fields | |||||
| NICE-SLAM [ 103 ] | PSNR | 13.83 | 17.87 | 12.89 | 14.86 |
| SSIM | 0.569 | 0.718 | 0.554 | 0.614 | |
| LPIPS | 0.482 | 0.344 | 0.498 | 0.441 | |
| Vox-Fusion [ 82 ] | PSNR | 15.79 | 16.32 | 17.27 | 16.46 |
| SSIM | 0.647 | 0.706 | 0.677 | 0.677 | |
| Method | Metric | 0000 | 0059 | 0106 | 0169 | 0181 | 0207 | Avg. |
|---|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | ||||||||
| NICE-SLAM [ 103 ] | PSNR | 18.71 | 16.55 | 17.29 | 18.75 | 15.56 | 18.38 | 17.54 |
| SSIM | 0.641 | 0.605 | 0.646 | 0.629 | 0.562 | 0.646 | 0.621 | |
| LPIPS | 0.561 | 0.534 | 0.510 | 0.534 | 0.602 | 0.552 | 0.548 | |
| Vox-Fusion [ 82 ] | PSNR | 19.06 | 16.38 | 18.46 | 18.69 | 16.75 | 19.66 | 18.17 |
| SSIM | 0.662 | 0.615 | 0.753 | 0.650 | 0.666 | 0.696 | 0.673 | |
| Method | Rm0 | Rm1 | Rm2 | Off0 | Off1 | Off2 | Off3 | Off4 | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | |||||||||
| NICE-SLAM [ 103 ] | 1.69 | 2.04 | 1.55 | 0.99 | 0.90 | 1.39 | 3.97 | 3.08 | 1.95 |
| DF-Prior [ 26 ] | 1.39 | 1.55 | 2.60 | 1.09 | 1.23 | 1.61 | 3.61 | 1.42 | 1.81 |
| Vox-Fusion [ 82 ] | 0.27 | 1.33 | 0.47 | 0.70 | 1.11 | 0.46 | 0.26 | 0.58 | 0.65 |
| ESLAM [ 33 ] | 0.71 | 0.70 | 0.52 | 0.57 | 0.55 | 0.58 | 0.72 | 0.63 | 0.63 |
| Point-SLAM [ 59 ] | 0.61 | 0.41 | 0.37 | 0.38 | 0.48 | 0.54 | 0.72 | 0.63 | 0.52 |
| Method | fr1/desk | fr2/xyz | fr3/office | Avg. |
|---|---|---|---|---|
| Neural Implicit Fields | ||||
| NICE-SLAM [ 103 ] | 4.3 | 31.7 | 3.9 | 13.3 |
| Vox-Fusion [ 82 ] | 3.5 | 1.5 | 26.0 | 10.3 |
| Point-SLAM [ 59 ] | 4.3 | 1.3 | 3.5 | 3.0 |
| Loopy-SLAM [ 41 ] | 3.8 | 1.6 | 3.4 | 2.9 |
| 3D Gaussian Splatting | ||||
| Method | fr1/desk | fr2/xyz | fr3/office | Avg. |
|---|---|---|---|---|
| VTGS-SLAM [ 27 ] | 382.4 | 3462.5 | 400.5 | 1415.1 |
| Ours | 2.2 | 1.7 | 2.0 | 2.0 |
| Method | SplaTAM [ 34 ] | LoopSplat [ 102 ] | VTGS-SLAM [ 27 ] | GS-ICP SLAM [ 20 ] | Ours(w/o init.) | Ours(w/ init.) |
|---|---|---|---|---|---|---|
| ATE RMSE | 11.3 | 7.3 | 9.7 | 115.8 | 122.4 | 4.7 |
| Method | 0000 | 0059 | 0106 | 0169 | 0181 | 0207 | Avg. |
|---|---|---|---|---|---|---|---|
| Neural Implicit Fields | |||||||
| NICE-SLAM [ 103 ] | 12.0 | 14.0 | 7.9 | 10.9 | 13.4 | 6.2 | 10.7 |
| Vox-Fusion [ 82 ] | 68.8 | 24.2 | 8.4 | 27.3 | 23.3 | 9.4 | 26.9 |
| Point-SLAM [ 59 ] | 10.2 | 7.8 | 8.7 | 22.2 | 14.8 | 9.5 | 12.2 |
| Loopy-SLAM [ 41 ] | 4.2 | 7.5 | 8.3 | 7.5 | 10.6 | 7.9 | 7.7 |
| 3D Gaussian Splatting | |||||||
| Method | a | b | c | d | e | Avg. |
|---|---|---|---|---|---|---|
| Neural Implicit Fields | ||||||
| Point-SLAM [ 59 ] | 246.16 | 632.99 | 830.79 | 271.42 | 574.86 | 511.24 |
| ESLAM [ 33 ] | 25.15 | 2.15 | 27.02 | 20.89 | 35.47 | 22.14 |
| Loopy-SLAM [ 41 ] | - | - | 25.16 | 234.25 | 81.48 | 113.63 |
| 3D Gaussian Splatting | ||||||
| SplaTAM [ 34 ] | 1.50 | 0.57 | 0.31 | 443.10 | 1.58 | 89.41 |
| Method | a | b | c | d | e | Avg. |
|---|---|---|---|---|---|---|
| 3D Gaussian Splatting | ||||||
| SplaTAM [ 34 ] | 28.02 | 27.93 | 29.48 | 19.65 | 28.48 | 26.71 |
| Gaussian-SLAM [ 87 ] | 30.06 | 30.02 | 31.15 | 28.75 | 31.94 | 30.38 |
| VTGS-SLAM [ 27 ] | 32.84 | 31.02 | 32.44 | 31.43 | 33.38 | 32.22 |
| LoopSplat [ 102 ] | 30.15 | 30.08 | 30.04 | 28.94 | 31.78 | 30.20 |
| Ours | 35.95 | 34.84 | 35.81 | 35.71 | 41.32 | 36.73 |
| Method | a | b | c | d | e | Avg. |
|---|---|---|---|---|---|---|
| 3D Gaussian Splatting | ||||||
| SplaTAM [ 34 ] | 23.95 | 22.66 | 13.95 | 8.47 | 20.06 | 17.82 |
| Gaussian-SLAM [ 87 ] | 26.66 | 24.42 | 15.01 | 18.35 | 21.91 | 21.27 |
| VTGS-SLAM [ 27 ] | 25.55 | 24.25 | 16.94 | 18.59 | 21.95 | 21.46 |
| LoopSplat [ 102 ] | 25.60 | 23.65 | 15.87 | 18.86 | 22.51 | 21.30 |
| Ours | 26.81 | 26.79 | 15.38 | 20.89 | 21.71 | 22.32 |
3DGS SLAM
3D Gaussian Splatting simultaneous localization and mapping builds renderable 3D Gaussian maps while estimating camera motion online.
See all 63 papers