PLS-GS: Probabilistic Lidar Scaffolds for Compact, Surface-Aligned Gaussian Splatting
Organizations: QUT Centre for Robotics · Emesent
Abstract
While 3D Gaussian Splatting (3DGS) is typically optimized from images only, many robotics and mapping applications provide both camera and high-definition LiDAR data, combining rich visual appearance with accurate geometric structure. We introduce PLS-GS, a probabilistic LiDAR scaffold for Gaussian Splatting that exploits this complementary sensing setting to construct a compact 3DGS representation aligned to the measured surface. PLS-GS fits a Hierarchical Gaussian Mixture Model jointly over position and color, yielding a continuous and adaptive scaffold that eliminates the need for discrete scene partitioning such as voxel grids or octrees. The fitted mixture then guides the renderable 3DGS Gaussians: each rendering Gaussian is associated with a GMM component and constrained in the local surface frame induced by that component's covariance. A Mahalanobis support loss bounds the deviation of the rendering Gaussians from the corresponding LiDAR-derived surface. Across five large outdoor scenes, PLS-GS places primitives markedly closer to the LiDAR measurements than both image- and LiDAR-initialized baselines, while employing substantially fewer primitives, yielding a considerably more compact model and faster rendering. At matched capacity, PLS-GS places primitives closer to the LiDAR measurements, with comparable rendering quality. Code is available at https://github.com/k-harikumar/PLS-GS.
Figures & tables
| Anchors | Rendering | Geometry (at ) | Efficiency | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Scene | Method | Init | Fin | PSNR | SSIM | LPIPS | P | R | F1 | Ch. | FPS | Train | Size |
| Campus [1pt] | Scaffold-GS (SfM) | 436K | 1370K | 27.71 | 0.863 | 0.175 | 0.195 | 0.102 | 0.134 | 1.190 | 123.9 | 22:33 | 432.9 |
| Scaffold-GS (LiDAR) | 436K | 1301K | 27.20 | 0.854 | 0.177 | 0.222 | 0.156 | 0.183 | 0.994 | 124.6 | 21:05 | 411.1 | |
| PLS-GS | 404K | 0 434K | 25.68 | 0.803 | 0.253 | 0.460 | 0.118 | 0.188 | 0.788 | 219.8 | 16:34 | 163.3 | |
| Village [1pt] | Scaffold-GS (SfM) | 310K | 1052K | 27.77 | 0.875 | 0.143 | 0.388 | 0.273 | 0.320 | 0.673 | 130.5 | 20:39 | 332.3 |
| Scaffold-GS (LiDAR) | 311K | 1009K | 26.95 | 0.864 | 0.149 | 0.386 | 0.313 | 0.346 | 0.664 | 130.7 | 19:02 | 318.8 | |
| Rendering | Geometry (at ) | Efficiency | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Scene | Method | Anchors | PSNR | SSIM | LPIPS | P | R | F1 | Ch. | FPS | Train | Size |
| Campus [1pt] | Scaffold-GS (static) | 434K | 25.89 | 0.814 | 0.234 | 0.251 | 0.082 | 0.124 | 1.324 | 311.6 | 10:36 | 137.2 |
| PLS-GS | 434K | 25.68 | 0.803 | 0.253 | 0.460 | 0.118 | 0.188 | 0.788 | 219.8 | 16:34 | 26.1 + 137.2 | |
| Village [1pt] | Scaffold-GS (static) | 432K | 26.33 | 0.838 | 0.190 | 0.362 | 0.221 | 0.274 | 0.906 | 248.8 | 11:55 | 136.4 |
| PLS-GS | 432K | 26.14 | 0.838 | 0.193 | 0.489 | 0.239 | 0.321 | 0.559 | 176.7 | 16:16 | 25.9 + 136.4 | |
| College [1pt] | Scaffold-GS (static) | 517K | 25.11 | 0.813 | 0.228 | 0.237 | 0.067 | 0.105 | 1.538 | 318.2 | 11:34 | 163.4 |
| Anchors | Rendering | Geometry (at ) | Efficiency | ||||||||||
| Configuration | Growth | Init | Final | PSNR | SSIM | LPIPS | P | R | F1 | Ch. | FPS | Train | Size |
| Scaffold-GS: photometric growth | |||||||||||||
| SfM | off | 855K | 0 855K | 25.98 | 0.837 | 0.200 | 0.195 | 0.086 | 0.119 | 1.243 | 202.9 | 22:06 | 270.2 |
| SfM | on | 855K | 1938K | 26.65 | 0.863 | 0.169 | 0.265 | 0.132 | 0.176 | 1.009 | 102.0 | 28:46 | 612.5 |
| LiDAR | off | 855K | 0 855K | 24.98 | 0.817 | 0.215 | 0.240 | 0.110 | 0.151 | 1.204 | 210.7 | 17:19 | 270.2 |
| LiDAR | on | 855K | 1746K | 25.87 | 0.853 | 0.172 | 0.292 | 0.178 | 0.221 | 0.918 | 109.7 | 25:45 | 551.7 |
| HGMM fit | Training | Total | ||||||
|---|---|---|---|---|---|---|---|---|
| Scene | Method | LiDAR pts | Time | Peak (GB) | Time | Peak (GB) | Time | Peak (GB) |
| Campus | Scaffold-GS (SfM) | 79.8M | – | – | 22:33 | 10.09 | 22:33 | 10.09 |
| Scaffold-GS (LiDAR) | – | – | 21:05 | 10.42 | 21:05 | 10.42 | ||
| PLS-GS | 33:20 | 10.05 | 16:34 | 0 7.61 | 49:54 | 10.05 | ||
| Village | Scaffold-GS (SfM) | 26.8M | – | – | 20:39 | 0 7.86 | 20:39 | 0 7.86 |
| Scaffold-GS (LiDAR) | – | – | 19:02 | 0 7.21 | 19:02 | 0 7.21 | ||
| Rendering | Geometry (at ) | Capacity | ||||||
|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | LPIPS | Precision | Recall | F1 | Ch. | Size | |
| 24.93 | 0.790 | 0.261 | 0.211 | 0.044 | 0.073 | 1.931 | 191.5 | |
| 25.01 | 0.801 | 0.243 | 0.291 | 0.073 | 0.117 | 1.092 | 192.8 | |
| (final) | 25.11 | 0.812 | 0.230 | 0.363 | 0.094 | 0.150 | 0.880 | 194.4 |
| 24.72 | 0.797 | 0.255 | 0.487 | 0.111 | 0.181 | 0.805 | 194.7 | |
| 23.90 | 0.740 | 0.326 | 0.532 | 0.064 | 0.114 | 0.938 | 188.6 | |
| Rendering | Geometry (at ) | Capacity | ||||||
|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | LPIPS | P | R | F1 | Ch. | Size | |
| 5 | 24.97 | 0.800 | 0.254 | 0.368 | 0.064 | 0.110 | 0.960 | 164.5 |
| 10 (adopted) | 25.11 | 0.812 | 0.230 | 0.363 | 0.094 | 0.150 | 0.880 | 194.4 |
| 15 | 25.37 | 0.824 | 0.210 | 0.387 | 0.157 | 0.222 | 0.817 | 228.0 |
| 20 | 25.33 | 0.827 | 0.202 | 0.397 | 0.196 | 0.261 | 0.777 | 258.6 |
3D Gaussian Splatting
3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.
See all 1,080 papers