Due to the real-time rendering performance, 3D Gaussian Splatting (3DGS) has emerged as the leading method for radiance field reconstruction. However, its reliance on spherical harmonics for color encoding inherently limits its ability to separate diffuse and specular components, making it challenging to accurately represent complex reflections. To address this, we propose a novel enhanced Gaussian kernel that explicitly models specular effects through view-dependent opacity. Meanwhile, we introduce an error-driven compensation strategy to improve rendering quality in existing 3DGS scenes. Our method begins with 2D Gaussian initialization and then adaptively inserts and optimizes enhanced Gaussian kernels, ultimately producing an augmented radiance field. Experiments demonstrate that our method not only surpasses state-of-the-art NeRF methods in rendering performance but also achieves greater parameter efficiency. Project page at: https://xiaoxinyyx.github.io/augs.
Figures & tables
Figure 1: We propose an augmented radiance field, which leverages Gaussian kernels with view-dependent opacity to accurately model specular highlights in the scene (left). It can be seamlessly plugged into existing Gaussian Splatting-based methods as a post enhancement, and notably, even using second-order spherical harmonics ( sh=2 ) is sufficient to capture complex illumination (right).
Figure 2: Our post-enhancement method for Gaussian Splatting. We begin with image-space refinement using 2D Gaussians to reconstruct regions exhibiting significant errors (left). Leveraging geometric information from depth maps, we then project these 2D Gaussians into world space (middle). The newly added Gaussians feature optimizable view-dependent opacity and are jointly optimized with existing Gaussians to recover challenging view-dependent color (right).
Figure 3: Inspired by classical Phong shading, we model view-dependent opacity with a cosine-weighted function whose shape is controlled by two parameters: β , which governs the lobe’s sharpness, and T , which determines its angular extent. Along with the central orientation of the lobe, each new kernel introduces 5 learnable parameters to a standard Gaussian primitive.
Method
Mip-NeRF 360
Tanks&Temples
Deep Blending
NeRF Synthetic
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Mip-NeRF 360
27.69
0.792
0.237
22.22
0.758
0.257
29.40
0.900
0.245
33.25
0.962
0.039
Zip-NeRF
28.54
0.828
0.189
-
-
-
-
-
-
33.10
0.971
0.031
[5pt/2pt] 3DGS
27.21
0.815
0.214
23.14
0.841
0.183
29.41
0.903
0.243
33.31
0.969
0.037
VoD-3DGS[L]
27.79
0.818
0.213
23.91
0.860
0.160
29.84
0.908
0.240
33.69
0.971
0.030
Spec-Gaussian
28.18
0.835
0.176
23.77
0.855
0.166
29.53
0.905
0.241
34.19
0.971
0.028
Table 1: Qualitative comparison across Mip-NeRF 360, Tanks&Temples, Deep Blending, and NeRF Synthetic. All scores for the baseline methods are directly taken from their papers, when available. Note that we use the VoD-3DGS variant with higher memory consumption and the anchor-free Spec-Gaussian, as recommended by the authors; for DBS, we adopted the data stopping at 30k iterations for fairness. Our method consistently outperforms state-of-the-art explicit and implicit approaches under the MCMC framework, whether utilizing second- or third-order spherical harmonics.
Figure 4: Qualitative comparison across real captured scenes. Benefiting from the newly designed Gaussian kernel with view-dependent transparency and the rendering loss-driven initialization, our approach surpasses all state-of-the-art explicit and implicit methods on real-world datasets.
Table 6Table 7
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Mip-NeRF 360
NeRF Synthetic
2D Training and Projection
742 s
178 s
Joint Optimization
168 s
32 s
Appendix
Table 6: The time consumption for the post-enhancement stage on the Mip-NeRF 360 and NeRF Synthetic datasets is reported as follows. It should be noted that using multiple GPUs can significantly accelerate this process, as the 2D-stage training operates independently on each image.
Figure 5: Comparison between Spherical Beta functions in DBS Liu et al. (2025) and opacity lobe. The three Gaussians in the first row model color using three differently oriented spherical beta functions, with maximum function values of 1, 2, and 4, all having an opacity of 0.5. The kernels in the second row use three differently oriented opacity lobes, with the same maximum amplitude and identical color intensity. Our method is more stable and flexible in reconstructing outgoing radiance, as it is less affected by the ordering of Gaussian kernels and has greater numerical stability.
Figure 6: Qualitative comparison on our synthetic shiny dataset rendered using environment maps with complex lighting conditions .
Method
Mip-NeRF 360
NeRF Synthetic
Total(Post-proc.) time
Memory
FPS
Total(Post-proc.) time
Memory
FPS
Spec-Gaussian
44.8 min
882 MB
43
12.0 min
75 MB
136
Ours (3DGS,sh=3)
27.9(15.0) min
691 MB
110
6.0(3.4) min
68 MB
446
Ours (MCMC,sh=2)
31.9(14.9)min
467 MB
96
7.0(3.2)min
44 MB
434
Ours (MCMC,sh=3)
25.3(15.2)min
722 MB
90
7.9(3.5) min
68 MB
414
Appendix
Table 7: Analysis of training time, memory usage, and FPS. ”Total time” represents the combined duration of the original scene reconstruction and post-processing enhancement.
metrics
bicycle
flowers
garden
stump
treehill
room
counter
kitchen
bonsai
mean
Zip- NeRF
PSNR ↑
25.80
22.40
28.20
27.55
23.89
32.65
29.38
32.50
34.46
28.54
SSIM ↑
0.769
0.642
0.860
0.800
0.681
0.925
0.902
0.928
0.949
0.828
LPIPS ↓
0.208
0.273
0.118
0.193
0.242
0.196
0.185
0.116
0.173
0.189
3DGS- MCMC
PSNR ↑
26.15
22.12
28.16
27.80
23.31
32.48
29.51
32.27
32.88
28.29
SSIM ↑
0.81
0.64
0.89
0.82
0.66
0.94
0.92
0.94
0.95
0.84
LPIPS ↓
0.18
0.31
0.10
0.19
0.29
0.25
0.22
0.14
0.22
0.21
Appendix
Table 8: Quantitative results of rendering quality per scene on Mip-NeRF 360 dataset.
Tanks and Temples
Deep Blending
metrics
truck
train
mean
drjohnson
playroom
mean
Zip- NeRF
PSNR ↑
-
-
-
-
-
-
SSIM ↑
-
-
-
-
-
-
LPIPS ↓
-
-
-
-
-
-
3DGS- MCMC
PSNR ↑
26.11
22.47
24.29
29.00
30.33
29.67
SSIM ↑
0.89
0.83
0.86
0.89
0.90
0.89
Appendix
Table 9: Quantitative results of rendering quality per scene on Tanks and Temples and Deep Blending dataset.
metrics
chair
drums
ficus
hotdog
lego
materials
mic
ship
mean
Zip- NeRF
PSNR ↑
34.84
25.84
33.90
37.14
34.84
31.66
35.15
31.38
33.10
SSIM ↑
0.983
0.944
0.985
0.984
0.980
0.969
0.991
0.929
0.971
LPIPS ↓
0.017
0.050
0.015
0.020
0.019
0.032
0.007
0.091
0.031
3DGS- MCMC
PSNR ↑
36.51
26.29
35.07
37.82
36.01
30.59
37.29
30.82
33.80
SSIM ↑
0.99
0.95
0.99
0.99
0.98
0.96
0.99
0.91
0.97
LPIPS ↓
0.02
0.04
0.01
0.02
0.02
0.04
0.01
0.12
0.04
Appendix
Table 10: Quantitative results of rendering quality per scene on NeRF synthetic dataset.
Method
Metrics
bicycle
flowers
garden
stump
treehill
room
counter
kitchen
bonsai
Mean
Ours (3DGS, sh=3)
Δ PSNR ↑
0.110
-0.067
0.307
0.089
0.168
0.720
1.230
0.806
1.765
0.570
Δ SSIM ↑(10−3)
3.26
5.17
2.84
4.77
3.19
2.77
5.49
2.11
5.25
3.87
Δ LPIPS ↓(10−3)
-5.09
-28.6
-3.00
-9.09
-14.9
-5.04
-8.08
-3.16
-7.23
-9.35
Ours (MCMC, sh=2)
Δ PSNR ↑
0.227
0.007
0.397
0.212
0.085
0.967
1.622
0.976
2.225
0.746
Δ SSIM ↑(10−3)
4.44
3.39
4.83
3.25
2.96
4.52
8.66
3.14
6.39
4.62
Δ LPIPS ↓(10−3)
-5.58
-12.5
-5.24
-4.62
-7.04
-6.97
-11.7
-4.94
-7.93
-7.38
Appendix
Table 11: Quantitative analysis of rendering quality improvements after enhancement per scene on Mip-NeRF 360 dataset.
Method
Metrics
chair
drums
ficus
hotdog
lego
materials
mic
ship
Mean
Ours (3DGS, sh=3)
Δ PSNR ↑
0.130
0.239
0.132
0.080
0.160
0.137
0.146
0.113
0.142
Δ SSIM ↑(10−4)
1.82
4.45
1.97
2.37
3.57
4.38
1.50
6.20
3.28
Δ LPIPS ↓(10−4)
-0.724
-5.99
-1.73
-5.69
-3.56
-3.32
-1.25
-6.17
-3.55
Ours (MCMC, sh=2)
Δ PSNR ↑
0.265
0.288
0.419
0.231
0.364
0.400
0.464
0.235
0.333
Δ SSIM ↑(10−4)
3.42
4.74
6.54
5.85
6.45
12.5
4.36
3.86
5.97
Δ LPIPS ↓(10−4)
-4.92
-13.0
-5.97
-7.44
-6.47
-11.5
-3.69
-13.9
-8.36
Appendix
Table 12: Quantitative analysis of rendering quality improvements after enhancement per scene on NeRF synthetic dataset.
Method
Metrics
Tanks and Temples
Deep Blending
truck
train
Mean
drjohnson
playroom
Mean
Ours (3DGS, sh=3)
Δ PSNR ↑
0.348
0.356
0.352
0.169
0.059
0.114
Δ SSIM ↑(10−3)
2.90
3.04
2.97
1.29
0.537
0.915
Δ LPIPS ↓(10−3)
-7.44
-3.34
-5.39
-5.96
-6.31
-6.14
Ours (MCMC, sh=2)
Δ PSNR ↑
0.460
0.522
0.491
0.256
0.126
0.191
Δ SSIM ↑(10−3)
4.07
6.18
5.12
2.17
1.28
1.72
Appendix
Table 13: Quantitative analysis of rendering quality improvements after enhancement per scene.
Method
Metrics
bicycle
flowers
garden
stump
treehill
room
counter
kitchen
bonsai
Mean
3DGS
Primitives (106)
4.8
3.0
4.1
4.5
3.3
1.4
1.2
1.5
1.2
2.8
Memory (MB)
1081
675
928
1017
732
316
263
328
275
624
DBS
Primitives (106)
6.0
3.0
5.0
4.5
3.5
1.5
1.5
1.5
1.5
3.1
Memory (MB)
618
309
515
464
361
155
155
155
155
320
Ours (3DGS, sh=3)
Primitives (106)
5.3
3.3
4.5
5.0
3.6
1.5
1.3
1.6
1.3
3.0
Memory (MB)
1198
748
1029
1128
811
350
291
363
305
691
Appendix
Table 14: Quantitative results of memory footprint per scene on Mip-NeRF 360 dataset.
bicycle
flowers
garden
stump
treehill
room
counter
kitchen
bonsai
Mean
Method
Primitives ( 106 )
5.9
3.3
5.2
4.8
3.7
1.5
1.2
1.8
1.3
3.2
3DGS-MCMC
Memory (MB)
1328
743
1170
1069
833
338
270
405
293
716
Ours (MCMC, sh=2)
865
484
763
697
543
220
176
264
191
467
Ours (MCMC, sh=3)
1338
748
1179
1077
839
340
272
408
295
722
Appendix
Table 15: Quantitative results of memory footprint per scene on Mip-NeRF 360 dataset.
Figure 7: Projection results of 2D Gaussians. We present the optimization results of the 2D primitives in image space along with the rendered outputs after back-projection into world space. The 4th column images display the L1 distance between the renderings before and after enhancement. The additional Guassians can also help restore the background reconstruction for outdoor scenes.
Figure 8: Qualitative analysis across real-captured scenes.
Figure 9: Diffuse specular decomposition. We convert both the diffuse color rendering of the original Gaussians and the enhanced rendering into linear color space to decouple the diffuse and specular components.
Figure 10: Qualitative comparison with DBS on diffuse-specular decomposition. The top-right and bottom-right sub-figures display the diffuse and specular components, respectively.
Neural radiance field (NeRF) and 3D Gaussian splatting (3DGS) are two mainstream approaches for novel view synthesis. They often show complementary performance, i.e., 3DGS demonstrating faster rendering speed and NeRF demonstrating higher rendering quality. Motivated by this, we propose leveraging NeRF-rendered images for 3DGS. Specifically, we target street scenes and utilize a pre-trained street-specific NeRF method to produce training images for a target 3DGS method. In our 3DGS training, NeRF-rendered images are used to remove transient objects in street-level input views and to generate bird's-eye views as additional views, inheriting the higher-quality rendering of NeRF into 3DGS. We further incorporate a diffusion-based image enhancement to improve the image quality of the additional views. Experimental results on one synthetic and two real datasets demonstrate that our proposed method improves street-scene rendering while preserving the speed of 3DGS and the quality of NeRF.
We revisit the role of appearance modeling in 3D Gaussian Splatting (3DGS) and show that limited expressiveness in view-dependent reflectance is a key driver of representation redundancy. In standard 3DGS, low-order spherical harmonics (SH) are used, restricting the splats' ability to model high-frequency directional effects, which is typically compensated by increasing the number of splats. We propose NRF-GS: Neural Residual Fields for Gaussian Splatting, a hybrid representation that replaces per-splat SH-bases with a shared neural residual field. Each Gaussian encodes a compact set of appearance features and a lambertian base color, while a lightweight global scene-level MLP predicts view-dependent residuals conditioned on viewing direction, distance, and per-splat features. This formulation enhances directional reflectance modeling by combining diffuse per-splat reflectance representations with a shared global function for high-frequency details, enabling both higher expressiveness and parameter sharing across splats. Our key insight is that by accurately capturing high-frequency directional reflectance, especially in specular regions, the GS-representation becomes more expressive, reducing the need for geometrically redundant splats. As a result, NRF-GS achieves comparable or better rendering quality while reducing the number of Gaussians by up to 50%, and produces visibly improved specular and high-frequency details.
While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.
Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa
Department of Computer Science and Communications Engineering, Waseda University, Japan