Augmented Radiance Field: A General Framework for Enhanced Gaussian Splatting

Yixin Yang; Bojian Wu; Yang Zhou; Hui Huang

cs.CVFeb 23, 2026

Augmented Radiance Field: A General Framework for Enhanced Gaussian Splatting

Authors: Yixin Yang, Bojian Wu, Yang Zhou, Hui Huang

Organizations: VCC, CSSE, Shenzhen University · Tencent Games

Abstract

Due to the real-time rendering performance, 3D Gaussian Splatting (3DGS) has emerged as the leading method for radiance field reconstruction. However, its reliance on spherical harmonics for color encoding inherently limits its ability to separate diffuse and specular components, making it challenging to accurately represent complex reflections. To address this, we propose a novel enhanced Gaussian kernel that explicitly models specular effects through view-dependent opacity. Meanwhile, we introduce an error-driven compensation strategy to improve rendering quality in existing 3DGS scenes. Our method begins with 2D Gaussian initialization and then adaptively inserts and optimizes enhanced Gaussian kernels, ultimately producing an augmented radiance field. Experiments demonstrate that our method not only surpasses state-of-the-art NeRF methods in rendering performance but also achieves greater parameter efficiency. Project page at: https://xiaoxinyyx.github.io/augs.

Figures & tables

Figure 1: We propose an augmented radiance field, which leverages Gaussian kernels with view-dependent opacity to accurately model specular highlights in the scene (left). It can be seamlessly plugged into existing Gaussian Splatting-based methods as a post enhancement, and notably, even using second-order spherical harmonics ( sh=2 ) is sufficient to capture complex illumination (right).

Figure 2: Our post-enhancement method for Gaussian Splatting. We begin with image-space refinement using 2D Gaussians to reconstruct regions exhibiting significant errors (left). Leveraging geometric information from depth maps, we then project these 2D Gaussians into world space (middle). The newly added Gaussians feature optimizable view-dependent opacity and are jointly optimized with existing Gaussians to recover challenging view-dependent color (right).

Figure 3: Inspired by classical Phong shading, we model view-dependent opacity with a cosine-weighted function whose shape is controlled by two parameters:

\beta

, which governs the lobe’s sharpness, and

T

, which determines its angular extent. Along with the central orientation of the lobe, each new kernel introduces 5 learnable parameters to a standard Gaussian primitive.

Method	Mip-NeRF 360			Tanks&Temples			Deep Blending			NeRF Synthetic
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Mip-NeRF 360	27.69	0.792	0.237	22.22	0.758	0.257	29.40	0.900	0.245	33.25	0.962	0.039
Zip-NeRF	28.54	0.828	0.189	-	-	-	-	-	-	33.10	0.971	0.031
[5pt/2pt] 3DGS	27.21	0.815	0.214	23.14	0.841	0.183	29.41	0.903	0.243	33.31	0.969	0.037
VoD-3DGS[L]	27.79	0.818	0.213	23.91	0.860	0.160	29.84	0.908	0.240	33.69	0.971	0.030
Spec-Gaussian	28.18	0.835	0.176	23.77	0.855	0.166	29.53	0.905	0.241	34.19	0.971	0.028

Table 1: Qualitative comparison across Mip-NeRF 360, Tanks&Temples, Deep Blending, and NeRF Synthetic. All scores for the baseline methods are directly taken from their papers, when available. Note that we use the VoD-3DGS variant with higher memory consumption and the anchor-free Spec-Gaussian, as recommended by the authors; for DBS, we adopted the data stopping at 30k iterations for fairness. Our method consistently outperforms state-of-the-art explicit and implicit approaches under the MCMC framework, whether utilizing second- or third-order spherical harmonics.

Figure 4: Qualitative comparison across real captured scenes. Benefiting from the newly designed Gaussian kernel with view-dependent transparency and the rendering loss-driven initialization, our approach surpasses all state-of-the-art explicit and implicit methods on real-world datasets.

Table 6

Table 7

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Mip-NeRF 360	NeRF Synthetic
2D Training and Projection	742 s	178 s
Joint Optimization	168 s	32 s

Appendix

Table 6: The time consumption for the post-enhancement stage on the Mip-NeRF 360 and NeRF Synthetic datasets is reported as follows. It should be noted that using multiple GPUs can significantly accelerate this process, as the 2D-stage training operates independently on each image.

Figure 5: Comparison between Spherical Beta functions in DBS Liu et al. (2025) and opacity lobe. The three Gaussians in the first row model color using three differently oriented spherical beta functions, with maximum function values of 1, 2, and 4, all having an opacity of 0.5. The kernels in the second row use three differently oriented opacity lobes, with the same maximum amplitude and identical color intensity. Our method is more stable and flexible in reconstructing outgoing radiance, as it is less affected by the ordering of Gaussian kernels and has greater numerical stability.

Figure 6: Qualitative comparison on our synthetic shiny dataset rendered using environment maps with complex lighting conditions .

Method	Mip-NeRF 360			NeRF Synthetic
Method	Total(Post-proc.) time	Memory	FPS	Total(Post-proc.) time	Memory	FPS
Spec-Gaussian	44.8 min	882 MB	43	12.0 min	75 MB	136
Ours (3DGS,sh=3)	27.9(15.0) min	691 MB	110	6.0(3.4) min	68 MB	446
Ours (MCMC,sh=2)	31.9(14.9)min	467 MB	96	7.0(3.2)min	44 MB	434
Ours (MCMC,sh=3)	25.3(15.2)min	722 MB	90	7.9(3.5) min	68 MB	414

Appendix

Table 7: Analysis of training time, memory usage, and FPS. ”Total time” represents the combined duration of the original scene reconstruction and post-processing enhancement.

	metrics	bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	mean
Zip- NeRF	PSNR $\uparrow$	25.80	22.40	28.20	27.55	23.89	32.65	29.38	32.50	34.46	28.54
	SSIM $\uparrow$	0.769	0.642	0.860	0.800	0.681	0.925	0.902	0.928	0.949	0.828
	LPIPS $\downarrow$	0.208	0.273	0.118	0.193	0.242	0.196	0.185	0.116	0.173	0.189
3DGS- MCMC	PSNR $\uparrow$	26.15	22.12	28.16	27.80	23.31	32.48	29.51	32.27	32.88	28.29
	SSIM $\uparrow$	0.81	0.64	0.89	0.82	0.66	0.94	0.92	0.94	0.95	0.84
	LPIPS $\downarrow$	0.18	0.31	0.10	0.19	0.29	0.25	0.22	0.14	0.22	0.21

Appendix

Table 8: Quantitative results of rendering quality per scene on Mip-NeRF 360 dataset.

		Tanks and Temples			Deep Blending
	metrics	truck	train	mean	drjohnson	playroom	mean
Zip- NeRF	PSNR $\uparrow$	-	-	-	-	-	-
	SSIM $\uparrow$	-	-	-	-	-	-
	LPIPS $\downarrow$	-	-	-	-	-	-
3DGS- MCMC	PSNR $\uparrow$	26.11	22.47	24.29	29.00	30.33	29.67
	SSIM $\uparrow$	0.89	0.83	0.86	0.89	0.90	0.89

Appendix

Table 9: Quantitative results of rendering quality per scene on Tanks and Temples and Deep Blending dataset.

	metrics	chair	drums	ficus	hotdog	lego	materials	mic	ship	mean
Zip- NeRF	PSNR $\uparrow$	34.84	25.84	33.90	37.14	34.84	31.66	35.15	31.38	33.10
	SSIM $\uparrow$	0.983	0.944	0.985	0.984	0.980	0.969	0.991	0.929	0.971
	LPIPS $\downarrow$	0.017	0.050	0.015	0.020	0.019	0.032	0.007	0.091	0.031
3DGS- MCMC	PSNR $\uparrow$	36.51	26.29	35.07	37.82	36.01	30.59	37.29	30.82	33.80
	SSIM $\uparrow$	0.99	0.95	0.99	0.99	0.98	0.96	0.99	0.91	0.97
	LPIPS $\downarrow$	0.02	0.04	0.01	0.02	0.02	0.04	0.01	0.12	0.04

Appendix

Table 10: Quantitative results of rendering quality per scene on NeRF synthetic dataset.

Method	Metrics	bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Mean
Ours (3DGS, sh=3)	$\Delta$ PSNR $\uparrow$	0.110	-0.067	0.307	0.089	0.168	0.720	1.230	0.806	1.765	0.570
	$\Delta$ SSIM $\uparrow(10^{-3})$	3.26	5.17	2.84	4.77	3.19	2.77	5.49	2.11	5.25	3.87
	$\Delta$ LPIPS $\downarrow(10^{-3})$	-5.09	-28.6	-3.00	-9.09	-14.9	-5.04	-8.08	-3.16	-7.23	-9.35
Ours (MCMC, sh=2)	$\Delta$ PSNR $\uparrow$	0.227	0.007	0.397	0.212	0.085	0.967	1.622	0.976	2.225	0.746
	$\Delta$ SSIM $\uparrow(10^{-3})$	4.44	3.39	4.83	3.25	2.96	4.52	8.66	3.14	6.39	4.62
	$\Delta$ LPIPS $\downarrow(10^{-3})$	-5.58	-12.5	-5.24	-4.62	-7.04	-6.97	-11.7	-4.94	-7.93	-7.38

Appendix

Table 11: Quantitative analysis of rendering quality improvements after enhancement per scene on Mip-NeRF 360 dataset.

Method	Metrics	chair	drums	ficus	hotdog	lego	materials	mic	ship	Mean
Ours (3DGS, sh=3)	$\Delta$ PSNR $\uparrow$	0.130	0.239	0.132	0.080	0.160	0.137	0.146	0.113	0.142
	$\Delta$ SSIM $\uparrow(10^{-4})$	1.82	4.45	1.97	2.37	3.57	4.38	1.50	6.20	3.28
	$\Delta$ LPIPS $\downarrow(10^{-4})$	-0.724	-5.99	-1.73	-5.69	-3.56	-3.32	-1.25	-6.17	-3.55
Ours (MCMC, sh=2)	$\Delta$ PSNR $\uparrow$	0.265	0.288	0.419	0.231	0.364	0.400	0.464	0.235	0.333
	$\Delta$ SSIM $\uparrow(10^{-4})$	3.42	4.74	6.54	5.85	6.45	12.5	4.36	3.86	5.97
	$\Delta$ LPIPS $\downarrow(10^{-4})$	-4.92	-13.0	-5.97	-7.44	-6.47	-11.5	-3.69	-13.9	-8.36

Appendix

Table 12: Quantitative analysis of rendering quality improvements after enhancement per scene on NeRF synthetic dataset.

Method	Metrics	Tanks and Temples			Deep Blending
Method	Metrics	truck	train	Mean	drjohnson	playroom	Mean
Ours (3DGS, sh=3)	$\Delta$ PSNR $\uparrow$	0.348	0.356	0.352	0.169	0.059	0.114
	$\Delta$ SSIM $\uparrow(10^{-3})$	2.90	3.04	2.97	1.29	0.537	0.915
	$\Delta$ LPIPS $\downarrow(10^{-3})$	-7.44	-3.34	-5.39	-5.96	-6.31	-6.14
Ours (MCMC, sh=2)	$\Delta$ PSNR $\uparrow$	0.460	0.522	0.491	0.256	0.126	0.191
	$\Delta$ SSIM $\uparrow(10^{-3})$	4.07	6.18	5.12	2.17	1.28	1.72

Appendix

Table 13: Quantitative analysis of rendering quality improvements after enhancement per scene.

Method	Metrics	bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Mean
3DGS	Primitives $(10^{6})$	4.8	3.0	4.1	4.5	3.3	1.4	1.2	1.5	1.2	2.8
3DGS	Memory (MB)	1081	675	928	1017	732	316	263	328	275	624
DBS	Primitives $(10^{6})$	6.0	3.0	5.0	4.5	3.5	1.5	1.5	1.5	1.5	3.1
DBS	Memory (MB)	618	309	515	464	361	155	155	155	155	320
Ours (3DGS, sh=3)	Primitives $(10^{6})$	5.3	3.3	4.5	5.0	3.6	1.5	1.3	1.6	1.3	3.0
Ours (3DGS, sh=3)	Memory (MB)	1198	748	1029	1128	811	350	291	363	305	691

Appendix

Table 14: Quantitative results of memory footprint per scene on Mip-NeRF 360 dataset.

		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Mean
Method	Primitives ( $10^{6}$ )	5.9	3.3	5.2	4.8	3.7	1.5	1.2	1.8	1.3	3.2
3DGS-MCMC	Memory (MB)	1328	743	1170	1069	833	338	270	405	293	716
Ours (MCMC, sh=2)		865	484	763	697	543	220	176	264	191	467
Ours (MCMC, sh=3)		1338	748	1179	1077	839	340	272	408	295	722

Appendix

Table 15: Quantitative results of memory footprint per scene on Mip-NeRF 360 dataset.

Figure 7: Projection results of 2D Gaussians. We present the optimization results of the 2D primitives in image space along with the rendered outputs after back-projection into world space. The 4th column images display the L1 distance between the renderings before and after enhancement. The additional Guassians can also help restore the background reconstruction for outdoor scenes.

Figure 8: Qualitative analysis across real-captured scenes.

Figure 9: Diffuse specular decomposition. We convert both the diffuse color rendering of the original Gaussians and the enhanced rendering into linear color space to decouple the diffuse and specular components.

Figure 10: Qualitative comparison with DBS on diffuse-specular decomposition. The top-right and bottom-right sub-figures display the diffuse and specular components, respectively.

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,019 papers

Leveraging NeRF-Rendered Images for 3D Gaussian Splatting
Jun 8, 2026Mizuki Morikawa, Yuta Shimizu, Chunyu Li +23D Gaussian Splatting Novel View Synthesis
Institute of Science Tokyo, Tokyo, Japan
NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting
Sep 29, 2026Pratik Singh Bisht, Andreas Kolb3DGS Compression 3D Gaussian Splatting
Computer Graphics Group University of Siegen
Gaussian Splatting with Reflections Guided by What is Seen
Oct 9, 2026Yiming Liang, Tianyu Xiao, Hiroshi IshikawaNeural Rendering Novel View Synthesis
Department of Computer Science and Communications Engineering, Waseda University, Japan

Jun 8, 2026J/K move · Enter open · S save

Leveraging NeRF-Rendered Images for 3D Gaussian Splatting

Mizuki Morikawa, Yuta Shimizu, Chunyu Li, Yusuke Monno+1

Institute of Science Tokyo, Tokyo, Japan

3D Gaussian Splatting Novel View Synthesis

Open paper AskPDF arXiv

Neural radiance field (NeRF) and 3D Gaussian splatting (3DGS) are two mainstream approaches for novel view synthesis. They often show complementary performance, i.e., 3DGS demonstrating faster rendering speed and NeRF demonstrating higher rendering quality. Motivated by this, we propose leveraging NeRF-rendered images for 3DGS. Specifically, we target street scenes and utilize a pre-trained street-specific NeRF method to produce training images for a target 3DGS method. In our 3DGS training, NeRF-rendered images are used to remove transient objects in street-level input views and to generate bird's-eye views as additional views, inheriting the higher-quality rendering of NeRF into 3DGS. We further incorporate a diffusion-based image enhancement to improve the image quality of the additional views. Experimental results on one synthetic and two real datasets demonstrate that our proposed method improves street-scene rendering while preserving the speed of 3DGS and the quality of NeRF.

Abstract

3D Gaussian Splatting

Explore similar work

Leveraging NeRF-Rendered Images for 3D Gaussian Splatting

NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

Gaussian Splatting with Reflections Guided by What is Seen