Recent progress in feed-forward 3D Gaussian Splatting (3DGS) has notably improved rendering quality. However, the spatially uniform and highly redundant 3DGS map generated by previous feed-forward 3DGS methods limits their integration into downstream reconstruction tasks. We propose SparseSplat, the first feed-forward 3DGS model that adaptively adjusts Gaussian density according to scene structure and information richness of local regions, yielding highly compact 3DGS maps. To achieve this, we propose entropy-based probabilistic sampling, generating large, sparse Gaussians in textureless areas and assigning small, dense Gaussians to regions with rich information. Additionally, we designed a specialized point cloud network that efficiently encodes local context and decodes it into 3DGS attributes, addressing the receptive field mismatch between the general 3DGS optimization pipeline and feed-forward models. Extensive experimental results demonstrate that SparseSplat can achieve state-of-the-art rendering quality with only 22% of the Gaussians and maintain reasonable rendering quality with only 1.5% of the Gaussians. Project page: https://victkk.github.io/SparseSplat-page/.
Figures & tables
Figure 2: Overall Pipeline of SparseSplat. Our method begins with using a frozen backbone [ 39 ] to generate feature maps and depth maps from multi-view posed images. Next, in the Adaptive Primitive Sampling stage, the entropy maps are calculated and transformed into probability maps to perform sampling, resulting in sparse 2D pixels. These pixels are then back-projected into 3D space using the predicted depth to form 3D Sparse Anchor Points. Finally, for each anchor point, we gather its local point cloud neighborhood via KNN. This local neighborhood is fed into a lightweight prediction head to predict its complete Gaussian attributes ( α,s,q,c ). All predicted Gaussian primitives are then merged to generate the final scene.
Figure 3: The Locality of Classic 3DGS Optimization. In this example, three 3D Gaussian primitives are splatted onto the 2D image plane. Primitive gc covers two pixels: one covered exclusively by gc , and another accumulating contributions from all three primitives. During backpropagation, gradients propagate to gc through both pixels. Notably, ga and gb modulate the gradient flow at the shared pixel by affecting the rendering process. The gradient flow to gc is detailed in ( 3 ).
Method
Category
PSNR ↑
SSIM ↑
LPIPS ↓
GS Cnt ↓
Time (s) ↓
MVSplat
pixel-aligned
22.95
0.774
0.192
688k
0.260
DepthSplat
pixel-aligned
24.17
0.816
0.152
688k
0.128
GGN
postprocess
20.23
0.570
0.268
162k
0.320
Long-LRM
postprocess
20.92
0.627
0.265
200k
0.115
AnySplat
Voxelization
17.45
0.471
0.320
608k
0.378
17.34
0.463
0.330
528k
0.384
Table 1: Quantitative comparison on the DL3DV dataset. ↑ indicates higher is better, ↓ indicates lower is better. ”GS Cnt” denotes the average number of Gaussian primitives across scenes.
Figure 4: Rendering quality comparisons on DL3DV. Our model matches the SOTA rendering quality of DepthSplat with only 150k Gaussians (vs. 688k). Under sparse settings (40k and 10k), our method maintains structural integrity and shows minor progressive blurring.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
MVSplat
19.13
0.628
0.423
DepthSplat
26.47
0.836
0.175
Ours-150k
26.64
0.846
0.180
Table 2: Cross-dataset generalization results on the Replica dataset.
Sampling Strategy
PSNR ↑
SSIM ↑
LPIPS ↓
Entropy (Ours)
22.36
0.718
0.262
Random
21.50
0.696
0.288
Laplacian
21.95
0.705
0.267
Table 3: Ablation on the primitive sampling strategy.
K
PSNR ↑
SSIM ↑
LPIPS ↓
0
21.53
0.683
0.291
5
22.23
0.713
0.267
10
22.34
0.717
0.264
20
22.36
0.718
0.262
30
22.38
0.718
0.261
Table 4: Effect of K in KNN.
Head
PSNR ↑
SSIM ↑
LPIPS ↓
Graph-Conv
20.78
0.664
0.329
MLP
21.47
0.683
0.299
Max Pooling
Fail
Fail
Fail
Geo-aware Attn
22.36
0.718
0.263
Table 5: Effect of Prediction Head.
Figure 6: Rendering Efficiency vs. Gaussian Count. X-axis log-scaled. We evaluate the rendering frame rate (FPS) across scenes with varying degrees of sparsity. While pixel-aligned baselines (typified by ∼ 688k Gaussians) operate at 71.9 FPS, our sparse-by-design approach significantly accelerates rendering. Our 150k model achieves ∼ 3 × speedup (208.6 FPS), and extremely sparse settings (10k–40k) unlock rates suitable for high-frequency robotics control loops ( > 600 FPS).
Dataset
Method
Sensor
Gaussian Primitives (M)
Replica (office0)
GS-ICP-SLAM
RGB-D
1.57 M
Photo-SLAM
RGB-D
0.08 M
Photo-SLAM
Monocular
0.08 M
TUM-RGBD (fr1/desk)
GS-ICP-SLAM
RGB-D
1.56 M
SplaTAM
RGB-D
0.96 M
Gaussian-SLAM
RGB-D
0.70 M
Table 6: Quantitative comparison of Gaussian primitive counts across different methods and datasets. Data sourced from CaRtGS.
Model
Backbone
Sampling
KNN
Attention
Total
Ours-10k
89.68
1.66
3.21
5.95
100.50
Ours-40k
89.68
1.82
14.87
15.04
121.41
Ours-100k
89.68
2.16
52.02
65.18
209.04
Ours-150k
89.68
2.08
219.64
103.31
414.71
Table 7: Inference Time Breakdown (ms). We detail the computational cost of each component in SparseSplat across different Gaussian counts. Note that the total time reported here is slightly higher than in the main text due to the synchronization overhead introduced by time profiling.