Authors: Ningjing Fan, Yiqun Wang, Dong-Ming Yan, Peter Wonka
Organizations: Chongqing University, China · MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS, China · King Abdullah University of Science and Technology (KAUST), Saudi Arabia
The reflective appearance, especially strong and typically near-field specular reflections, poses a fundamental challenge for accurate surface reconstruction and novel view synthesis. Existing Gaussian splatting methods either fail to model near-field specular reflections or rely on explicit ray tracing at substantial computational cost. We present Ref-DGS, a reflective dual Gaussian splatting framework that addresses this trade-off by decoupling surface reconstruction from specular reflection within an efficient rasterization-based pipeline. Ref-DGS introduces a dual Gaussian scene representation consisting of geometry Gaussians and complementary local reflection Gaussians that capture near-field specular interactions without explicit ray tracing, along with a global environment reflection field for modeling far-field specular reflections. To predict specular radiance, we further propose a lightweight, physically-aware specular adaptive mixing shader that fuses global and local specular features. Experiments demonstrate that Ref-DGS achieves state-of-the-art performance on reflective scenes while training substantially faster than ray-based Gaussian methods.
Figures & tables
Figure 1. Left: Surface reconstruction results on the RefReal dataset for two challenging scenes, gardenspheres (top) and toycar (bottom), both with strong near-field specular reflections. The bottom row reports the average training time. Right: Results on the GlossyReal dataset for the vase scene, showing surface reconstruction and rendering quality. Our method produces more accurate geometry and higher-quality reflections, as highlighted in the close-up regions.
Figure 2. An overview of our Ref-DGS framework.
Figure 3. Left: Input image. Middle: Normal map of the local reflection Gaussians Glocal . Right: Illustration of specular reflection, where the surface normal n is perpendicular to the surface, and the image distance ( id ) equals the object distance ( od ).
ball
car
coffee
helmet
teapot
toaster
mean
time ↓
Ref-NeRF
1.55
14.93
12.24
29.48
9.23
42.87
18.38
–
ENVIDR
0.74
7.10
9.23
1.66
2.47
6.45
4.61
–
UniSDF
0.45
6.88
8.00
1.72
2.80
6.45
8.71
–
PGSR
66.93
4.62
2.91
6.01
1.01
15.31
16.13
27.5m
MILo
61.30
4.47
2.36
4.87
0.62
16.49
15.02
18.9m
GaussianShader
7.03
14.05
14.93
9.33
7.17
13.08
10.93
1.10h
Table 1. Quantitative surface reconstruction comparison on the ShinySynthetic dataset (normal MAE and training time). Best results are highlighted in red and second-best in yellow.
angel
bell
cat
horse
luyu
potion
tbell
teapot
mean
CD ↓
PGSR
0.77
3.08
3.39
0.90
1.16
3.99
4.59
4.57
2.81
MILo
0.77
16.52
2.84
1.02
44.64
3.39
5.08
4.40
9.83
GaussianShader
0.85
1.10
2.56
0.73
1.07
4.74
5.74
3.40
2.53
Ref-Gaussian
0.45
0.70
1.68
0.64
0.88
0.81
0.59
1.01
0.95
Ref-GS
0.41
0.74
1.73
0.47
0.89
1.05
0.52
0.88
0.84
Table 2. Quantitative surface reconstruction comparison on the GlossySynthetic dataset (CD ×102 and normal MAE). Best results are highlighted in red and second-best in yellow.
Datasets
ShinySynthetic
GlossySynthetic
RefReal
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Ref-NeRF
33.32
0.956
0.110
25.65
0.905
0.112
23.62
0.646
0.239
ENVIDR
32.88
0.969
0.072
29.06
0.947
0.060
23.80
0.606
0.332
PGSR
26.82
0.921
0.119
25.59
0.913
0.091
24.43
0.696
0.220
MILo
27.28
0.921
0.124
25.69
0.914
0.096
23.87
0.643
0.291
GaussianShader
30.42
0.951
0.082
27.11
0.922
0.082
23.46
0.647
0.257
Table 3. Quantitative NVS comparison. Best results are highlighted in red and second-best in yellow.
Figure 4. Qualitative surface reconstruction results (meshes and normals) on the ShinySynthetic (coffee, toaster) and GlossySynthetic (cat) datasets.
Figure 5. Qualitative rendering quality and surface reconstruction on the GlossyReal (bear, vase) dataset.
Figure 6. Qualitative NVS results on the ShinySynthetic (coffee, toaster) and GlossySynthetic (cat, tbell, teapot) datasets.
Figure 7. Ablation study on surface reconstruction.
Figure 8. Ablation study on surface normal estimation. Angular error maps (red: high, blue: low) show that the full model significantly reduces geometric artifacts compared to ablated variants. Chamfer distances (CD) are reported at the bottom.
Figure 9. Ablation study on novel view synthesis.
Figure 10. Ablation study on the diffuse leakage issue, visualizing the predicted diffuse color.
EnvGS
MaterialRefGS
Ref-GS
Ours
FPS ↑
26.22
30.07
67.63
76.34
Table 4. Rendering speed (FPS) on the ShinySynthetic dataset.
PSNR ↑
SSIM ↑
LPIPS ↓
MAE ↓
CD ↓
MAE ↓
w/o Flocal w/o Glocal
34.36
0.971
0.057
1.53
0.82
1.99
w/o Glocal
34.70
0.973
0.057
1.46
0.73
1.93
w/o Adap Mix
34.98
0.974
0.054
1.46
0.64
1.92
w/o Phy-Aware
35.02
0.975
0.053
1.45
0.62
1.90
Full model
35.21
0.975
0.053
1.43
0.62
1.88
Table 5. Ablation study on the ShinySynthetic (PSNR, SSIM, LPIPS, and normal MAE) and GlossySynthetic (CD ×102 and normal MAE) datasets.
While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.
Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa
Department of Computer Science and Communications Engineering, Waseda University, Japan
Gaussian Splatting has significantly improved the quality of novel view synthesis with explicit Gaussian representation. However, we observed that existing 3D Gaussian Splatting methods (3DGS) often suffer from surface collapse issues on reflective regions, and thus produce inferior geometry and low-quality specular. In this work, we propose a physically-based deferred rendering framework, named Reflection-aware Gaussian Splatting (RGS), that can accurately model specular regions and improve novel view synthesis performance. Specifically, we found that a powerful 3D foundation model can provide a strong 3D geometric prior to foster correct geometric modeling. Based on this, we propose a cross-view shape consistency regularization to regularize the geometry surface with the large model prior and cross-view constraints. In this manner, our RGS can produce smoother geometric surfaces on reflective regions while reducing geometric hollows. To further improve rendering results on reflective regions, we present a reflection-aware densification strategy that is designed to capture specular variations across various views. With this strategy, our RGS is able to render novel views of objects in higher quality. Extensive experiments demonstrate our method consistently renders high-quality reflective objects, achieving state-of-the-art performance.
Xiaobiao Du, Yida Wang, Cheng Bi +2
University of Technology Sydney · Li Auto Inc. · Adelaide University
In recent years, 3D Gaussian splatting (3DGS) has achieved remarkable progress in novel view synthesis. However, accurately reconstructing glossy surfaces under complex illumination remains challenging, particularly in scenes with strong specular reflections and multi-surface interreflections. To address this issue, we propose SSR-GS, a specular reflection modeling framework for glossy surface reconstruction. Specifically, we introduce a prefiltered Mip-Cubemap to model direct specular reflections efficiently, and propose an IndiASG module to capture indirect specular reflections. Furthermore, we design Visual Geometry Priors (VGP) that couple a reflection-aware visual prior via a reflection score (RS) to downweight the photometric loss contribution of reflection-dominated regions, with geometry priors derived from VGGT, including progressively decayed depth supervision and transformed normal constraints. Extensive experiments on both synthetic and real-world datasets demonstrate that SSR-GS achieves state-of-the-art performance in glossy surface reconstruction.