Point-Based 3D Reconstruction from Sparse Views under Known Illumination
Authors: Magnus Kaufmann Gjerde, Joakim Bruslund Haurum, Jeppe Revall Frisvad, Markus Worchel, J. Andreas Bærentzen, Thomas B. Moeslund
Organizations: Visual Analysis & Perception Lab, Aalborg University · Centre for Software Technology, University of Southern Denmark · Pioneer Centre for Artificial Intelligence · Department of Applied Mathematics and Computer Science, Technical University of Denmark · Technische Universität Berlin
Sparse view 3D reconstruction is commonly addressed with neural implicit surfaces or dense point-based representations such as Gaussian splatting. Surface-aware splatting methods improve extracted geometry through oriented primitives and regularization, while RadiosityGS incorporates differentiable light transport through a radiosity inspired finite-element surfel formulation. We propose a differentiable point rendering method based on opacity-bearing beta surfels. An opacity explicit adjoint light transport formulation provides gradients for surfel geometry and appearance parameters, allowing physically based light transport to constrain reconstruction. Across five synthetic objects reconstructed from ten posed views, our method achieves the lowest mean symmetric Chamfer distance among the evaluated baselines and reduces mean Chamfer distance by 28.5% relative to the strongest point-based baseline while using only 267 surfels on average, approximately ~161 fewer primitives. Directional Chamfer results further show improved accuracy and competitive completion relative to related point-based methods. These results show that, in the controlled direct illumination setting, compact beta surfels combined with transport-based optimization can recover surfaces without relying on the tens to hundreds of thousands of primitives used by the evaluated baselines.
Figures & tables
Figure 1: Overview of our optimization pipeline. Starting from a sparse beta-surfel initialization, we alternate primal rendering, residual evaluation, adjoint transport, and parameter updates. The adjoint source is the image-space residual between the rendered and reference images.
Figure 2: Overview of the synthetic sparse-view object reconstruction dataset. Each scene is reconstructed from ten posed input views rendered with known static point-light illumination and Lambertian materials. For visual clarity, we show one representative training image per scene.
Method
Teapot
LEGO
Horse
Dragon
Plant
Mean ↓
Avg pts ↓
NeuS2 [ 22 ]
95.850
112.344
90.242
101.955
93.212
98.721
–
NeuS [ 21 ]
1.396
1.424
2.022
1.397
2.180
1.684
–
GeoSVR [ 10 ] †
3.667
1.446
2.054
3.255
10.296
4.144
–
2DGS 7K [ 5 ]
2.471
1.429
1.702
2.251
2.891
2.149
43.0k
2DGS 30K [ 5 ]
53.570
30.563
39.075
53.149
15.917
38.455
51.0k
SuGaR [ 4 ]
5.837
1.406
1.993
2.613
3.468
3.064
421.4k
Table 1: Main geometry reconstruction results for sparse-view object reconstruction. Entries report symmetric Chamfer distance (CD), scaled by 102 for readability; lower is better. The mean is computed across all five scenes. The final column reports the average number of optimized primitives for point-based methods. Within the point-based methods, the best , second-best , and third-best results are highlighted.
Figure 3: Qualitative reconstruction comparison across synthetic object scenes. We compare the ground-truth geometry with our reconstruction and baseline methods. Our method preserves the main object structure while using fewer optimized primitives.
Method
Accuracy ↓
Completion ↓
2DGS 7K [ 5 ]
0.020
0.023
2DGS 30K [ 5 ]
0.743
0.026
SuGaR [ 4 ]
0.037
0.024
GOF [ 24 ]
0.295
0.014
RadiosityGS [ 6 ]
0.512
0.026
Ours
0.017
0.014
Table 2: Unscaled mean directional Chamfer distance for the point-based methods across five scenes. Accuracy penalizes spurious geometry, while completion penalizes missing regions.
Setting
Ld
Ln
Lα
Learn β
CD ↓
Acc. ↓
Comp. ↓
Full
✓
✓
✓
✓
0.0132
0.0138
0.0125
w/o Ld
✓
✓
✓
0.0162
0.0171
0.0152
w/o Ln
✓
✓
✓
0.0185
0.0193
0.0178
w/o Lα
✓
✓
✓
0.0241
0.0187
0.0295
Fixed β
✓
✓
✓
0.0278
0.0245
0.0312
Table 3: Ablation study on the Horse scene using ten input views. Ld : depth distortion loss; Ln : normal consistency loss; Lα : weak opacity prior. CD denotes unscaled symmetric Chamfer distance; lower is better.
We present 3D Surface Splatting (3DSS), the first differentiable surface splatting renderer for physically-based inverse rendering from multi-view images. Our central insight is that the surface separation problem at the heart of surface splatting admits a direct formulation in terms of the reconstruction kernels themselves. From this foundation we derive a coverage-based compositing model whose per-layer opacity arises directly from the accumulated Elliptical Weighted Average reconstruction weight, yielding anti-aliased silhouettes and informative visibility gradients at sparsely covered edges. Combined with forward microfacet shading under co-optimized HDR environment lighting and density-aware adaptive refinement, 3DSS jointly recovers shape, spatially-varying BRDF materials, and illumination. Because the optimized representation is a set of oriented surface samples, it bridges natively to mesh-based workflows via surface reconstruction from oriented point cloud methods. We evaluate 3DSS against mesh-based, implicit, and Gaussian-splatting baselines across geometry reconstruction, novel-view synthesis, and novel-illumination relighting.
3D Gaussian Splatting (3DGS) has demonstrated impressive performance in 3D scene reconstruction. Beyond novel view synthesis, it shows great potential for multi-view surface reconstruction. Existing methods employ optimization-based reconstruction pipelines that achieve precise and complete surface extractions. However, these approaches typically require dense input views and high time consumption for per-scene optimization. To address these limitations, we propose SurfelSplat, a feed-forward framework that generates efficient and generalizable pixel-aligned Gaussian surfel representations from sparse-view images. We observe that conventional feed-forward structures struggle to recover accurate geometric attributes of Gaussian surfels because the spatial frequency of pixel-aligned primitives exceeds Nyquist sampling rates. Therefore, we propose a cross-view feature aggregation module based on the Nyquist sampling theorem. Specifically, we first adapt the geometric forms of Gaussian surfels with spatial sampling rate-guided low-pass filters. We then project the filtered surfels across all input views to obtain cross-view feature correlations. By processing these correlations through a specially designed feature fusion network, we can finally regress Gaussian surfels with precise geometry. Extensive experiments on DTU reconstruction benchmarks demonstrate that our model achieves comparable results with state-of-the-art methods, and predict Gaussian surfels within 1 second, offering a 100x speedup without costly per-scene training.
Surface reconstruction under sparse-view settings remains challenging due to limited geometric cues. Volume rendering methods based on signed distance functions often produce over-smoothed surfaces, while 3D Gaussian Splatting (3DGS), though time-efficient, suffers from incomplete geometry due to the lack of reliable depth supervision and the limitation of being optimized only from given input views. In this paper, we present Sparse-GS2Mesh, a stereo-aware framework for surface reconstruction from sparse views. While 3DGS and stereo matching have been leveraged for surface reconstruction under dense view settings, we extend them to operate effectively under sparse view conditions by first initializing 3DGS using epipolar depth priors to mitigate the 3DGS overfitting problem, followed by our three key components: (I) adaptive baseline selection, (II) fine-tuning with a stereo matching network, and (III) 2D/3D co-regularized fine-tuning. Given a warmed-up 3DGS initialized with epipolar depth, the adaptive baseline selection automatically determines a baseline to synthesize for each sparse view. We then fine-tune 3DGS by backpropagating depth-refining gradients from the stereo matching network, effectively specializing the 3DGS for stereo matching. The 2D/3D co-regularization further helps obtain stable reconstruction, addressing weak geometric cues in close stereo views. Sparse-GS2Mesh achieves a 15% improvement over state-of-the-art methods in little-overlap settings and comparable results in large-overlap settings. Project page: https://yhnoh8623.github.io/projects/s-gs2mesh/.