NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

Pratik Singh Bisht; Andreas Kolb

cs.CVSep 29, 2026

NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

Authors: Pratik Singh Bisht, Andreas Kolb

Organizations: Computer Graphics Group University of Siegen

Abstract

We revisit the role of appearance modeling in 3D Gaussian Splatting (3DGS) and show that limited expressiveness in view-dependent reflectance is a key driver of representation redundancy. In standard 3DGS, low-order spherical harmonics (SH) are used, restricting the splats' ability to model high-frequency directional effects, which is typically compensated by increasing the number of splats. We propose \emph{NRF-GS: Neural Residual Fields for Gaussian Splatting}, a hybrid representation that replaces per-splat SH-bases with a shared neural residual field. Each Gaussian encodes a compact set of appearance features and a lambertian base color, while a lightweight \emph{global scene-level MLP} predicts view-dependent residuals conditioned on viewing direction, distance, and per-splat features. This formulation enhances directional reflectance modeling by combining diffuse per-splat reflectance representations with a shared global function for high-frequency details, enabling both higher expressiveness and parameter sharing across splats. Our key insight is that by accurately capturing high-frequency directional reflectance, especially in specular regions, the GS-representation becomes more expressive, reducing the need for geometrically redundant splats. As a result, NRF-GS achieves comparable or better rendering quality while reducing the number of Gaussians by up to 50%, and produces visibly improved specular and high-frequency details.

Figures & tables

Panel of Figure 1 — Figure 1 : Novel view from Mip-NeRF 360 dataset - Garden scene. Our method (d) achieves better specular representation on the foreground table as well as sharper details on the background wall with significantly lower number of points compared to other methods.

Figure 2 : NRF-GS architecture. Per Gaussian geometric attributes $(\mathbf{x}_{i},\mathbf{\Sigma}_{i})$ and appearance attributes ( $\mathbf{c}_{i}^{\text{base}}$ , $\alpha_{i}^{\text{base}}$ , and a latent feature $\mathbf{z}_{i}$ ). The viewing direction $\mathbf{d}_{i}$ is frequency-encoded and decomposed into low ( $\gamma_{\text{low}}(\mathbf{d}_{i})$ ) and high-frequency ( $\gamma_{\text{high}}(\mathbf{d}_{i})$ ) components. Two MLP branches ( $f_{\text{low}}$ , $f_{\text{high}}$ ) predict a low-frequency residual color and opacity residual $(\Delta\mathbf{c}_{i}^{\text{low}},\delta_{i}^{\alpha})$ and a high-frequency color residual with a learned modulation weight $(\Delta\mathbf{c}_{i}^{\text{high}},w_{i})$ . The final color and opacity are then rendered using the unchanged standard Gaussian splatting rasterizer.

Table 3

Panel of Figure 3 — Figure 3 : Simulation results for synthetic degree-5 SH (top row) and real BTF data Sattler et al. (2003) (bottom row). Each row shows ground truth, degree-3 SH and NRF reconstruction, and corresponding error maps.

Figure 4 : Point count growth for the Kitchen scene (MipNeRF360). NRF-GS converges to fewer Gaussians under identical densification/pruning.

Dataset	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
MipNeRF360	3DGS	3.359	26.46	0.813	0.184	0.030	27.40
	VDGS	3.548	41.27	0.813	0.186	0.029	27.65
	NRF-GS (ours)	1.656	31.38	0.814	0.191	0.029	27.86
MipNeRF360*	3DGS	2.605	24.22	0.812	0.199	0.030	27.74
	VDGS	2.828	36.70	0.813	0.200	0.029	28.03
	GSNB	2.667	59.06	0.800	0.201	0.031	27.71

Table 3 : Average test-set results on MipNeRF360, DL3DV, and Tanks and Temples. Points are reported in millions. MipNeRF360* excludes Bicycle and Garden , where GSNB goes out-of-memory. Best and second-best values in each dataset are highlighted.

Panel of Figure 5 — Figure 5 : Qualitative comparison across multiple scenes. Top to Bottom: MipNeRF360 - scenes ’kitchen’ and ’room’, TnT - scenes ’palace’ and ’panther’, DL3DV - scenes ’387e’ and ’06da’.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Scene	Method	MS-SSIM $\uparrow$		LPIPS $\downarrow$		L1 $\downarrow$		PSNR $\uparrow$
Scene	Method	Train	Test	Train	Test	Train	Test	Train	Test
Bicycle	Single	0.864	0.715	0.162	0.234	0.026	0.047	27.34	23.71
Bicycle	Ours	0.843	0.759	0.170	0.203	0.029	0.041	26.11	25.06
Bonsai	Single	0.950	0.942	0.138	0.144	0.012	0.013	34.54	33.23
Bonsai	Ours	0.950	0.944	0.136	0.137	0.012	0.014	34.10	33.24
Counter	Single	0.926	0.906	0.151	0.173	0.013	0.018	32.03	29.81

Appendix

Table 4 : Per-scene comparison of Single-branch vs Ours (NRF-GS) on MipNeRF360. Train/Test metrics are reported as sub-columns.

Dataset	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
MipNeRF360	3DGS	3.359 $\pm$ 1.986	26.46 $\pm$ 05.02	0.813 $\pm$ 0.130	0.184 $\pm$ 0.094	0.030 $\pm$ 0.015	27.40 $\pm$ 3.95
	VDGS	3.548 $\pm$ 1.869	41.27 $\pm$ 09.95	0.813 $\pm$ 0.132	0.186 $\pm$ 0.096	0.030 $\pm$ 0.015	27.65 $\pm$ 4.09
	NRF-GS (ours)	1.656 $\pm$ 1.015	31.38 $\pm$ 07.12	0.814 $\pm$ 0.131	0.191 $\pm$ 0.097	0.029 $\pm$ 0.016	27.86 $\pm$ 4.27
MipNeRF360*	3DGS	2.605 $\pm$ 1.503	24.22 $\pm$ 02.66	0.813 $\pm$ 0.147	0.200 $\pm$ 0.098	0.029 $\pm$ 0.017	27.74 $\pm$ 4.46
	VDGS	2.828 $\pm$ 1.372	36.69 $\pm$ 04.58	0.814 $\pm$ 0.149	0.200 $\pm$ 0.101	0.029 $\pm$ 0.017	28.04 $\pm$ 4.59
	GSNB	2.667 $\pm$ 1.315	59.06 $\pm$ 20.68	0.800 $\pm$ 0.162	0.201 $\pm$ 0.102	0.032 $\pm$ 0.020	27.71 $\pm$ 5.13

Appendix

Table 5 : Average test-set results (mean

\pm

std) on MipNeRF360, DL3DV, and Tanks and Temples. MipNeRF360* excludes Bicycle and Garden scenes where GSNB runs out-of-memory. Points are reported in millions.

Panel of Figure 6 — Figure 6 : Additional qualitative comparison across multiple scenes. Top to Bottom: MipNeRF360 - scenes ’bonsai’, TnT - scenes ’horse’ and ’lighthouse’, DL3DV - scenes ’5c3a’ and ’669c’.

Scene	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
Bicycle	3DGS	6.137 $\pm$ 0.076	34.08 $\pm$ 0.13	0.765 $\pm$ 0.001	0.175 $\pm$ 0.001	0.039 $\pm$ 0.001	25.17 $\pm$ 0.04
	VDGS	6.441 $\pm$ 0.069	58.88 $\pm$ 0.29	0.755 $\pm$ 0.002	0.195 $\pm$ 0.002	0.039 $\pm$ 0.001	25.14 $\pm$ 0.14
	NRF-GS	3.187 $\pm$ 0.024	44.12 $\pm$ 0.28	0.759 $\pm$ 0.003	0.203 $\pm$ 0.006	0.041 $\pm$ 0.001	25.06 $\pm$ 0.16
Bonsai	3DGS	1.257 $\pm$ 0.005	20.12 $\pm$ 0.10	0.943 $\pm$ 0.001	0.133 $\pm$ 0.001	0.015 $\pm$ 0.001	32.23 $\pm$ 0.10
	VDGS	1.617 $\pm$ 0.012	29.70 $\pm$ 0.33	0.945 $\pm$ 0.002	0.134 $\pm$ 0.005	0.014 $\pm$ 0.000	32.72 $\pm$ 0.15
	GSNB	1.477 $\pm$ 0.042	39.15 $\pm$ 0.18	0.945 $\pm$ 0.002	0.127 $\pm$ 0.003	0.014 $\pm$ 0.001	33.19 $\pm$ 0.05

Appendix

Table 6 : Per-scene test-set results on MipNeRF360.

Scene	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
06da	3DGS	1.330 $\pm$ 0.007	12.85 $\pm$ 0.09	0.905 $\pm$ 0.001	0.107 $\pm$ 0.001	0.024 $\pm$ 0.001	28.21 $\pm$ 0.01
	VDGS	1.687 $\pm$ 0.012	22.30 $\pm$ 0.05	0.906 $\pm$ 0.002	0.110 $\pm$ 0.002	0.023 $\pm$ 0.001	28.41 $\pm$ 0.13
	GSNB	1.404 $\pm$ 0.007	31.57 $\pm$ 0.50	0.908 $\pm$ 0.002	0.102 $\pm$ 0.003	0.021 $\pm$ 0.001	29.34 $\pm$ 0.01
	NRF-GS	0.703 $\pm$ 0.009	15.13 $\pm$ 0.13	0.928 $\pm$ 0.000	0.098 $\pm$ 0.001	0.017 $\pm$ 0.001	30.91 $\pm$ 0.03
093e	3DGS	0.897 $\pm$ 0.011	11.05 $\pm$ 0.13	0.952 $\pm$ 0.001	0.073 $\pm$ 0.001	0.016 $\pm$ 0.000	31.21 $\pm$ 0.02
	VDGS	0.957 $\pm$ 0.004	16.28 $\pm$ 0.10	0.955 $\pm$ 0.002	0.071 $\pm$ 0.001	0.016 $\pm$ 0.000	31.46 $\pm$ 0.15

Appendix

Table 7 : Per-scene test-set results on DL3DV.

Scene	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
Auditorium	3DGS	0.735 $\pm$ 0.005	10.92 $\pm$ 0.08	0.871 $\pm$ 0.002	0.205 $\pm$ 0.005	0.044 $\pm$ 0.001	24.28 $\pm$ 0.03
	VDGS	0.856 $\pm$ 0.002	17.42 $\pm$ 0.08	0.858 $\pm$ 0.003	0.209 $\pm$ 0.001	0.047 $\pm$ 0.001	24.10 $\pm$ 0.06
	GSNB	0.765 $\pm$ 0.004	21.17 $\pm$ 0.06	0.867 $\pm$ 0.002	0.199 $\pm$ 0.001	0.046 $\pm$ 0.001	24.25 $\pm$ 0.04
	NRF-GS	0.375 $\pm$ 0.003	12.17 $\pm$ 0.06	0.886 $\pm$ 0.002	0.193 $\pm$ 0.001	0.038 $\pm$ 0.001	25.69 $\pm$ 0.11
Ballroom	3DGS	3.077 $\pm$ 0.068	22.48 $\pm$ 0.08	0.824 $\pm$ 0.001	0.102 $\pm$ 0.003	0.038 $\pm$ 0.000	24.15 $\pm$ 0.06
	VDGS	3.802 $\pm$ 0.007	43.83 $\pm$ 0.13	0.830 $\pm$ 0.002	0.105 $\pm$ 0.001	0.035 $\pm$ 0.001	24.83 $\pm$ 0.58

Appendix

Table 8 : Per-scene test-set results on Tanks and Temples.

Scene	Method	Points (M) $\downarrow$	Time (min) $\downarrow$	MS-SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	PSNR $\uparrow$
Lighthouse	3DGS	0.870 $\pm$ 0.025	12.40 $\pm$ 0.22	0.843 $\pm$ 0.002	0.161 $\pm$ 0.001	0.052 $\pm$ 0.001	22.31 $\pm$ 0.02
	VDGS	1.201 $\pm$ 0.031	22.37 $\pm$ 0.10	0.841 $\pm$ 0.002	0.162 $\pm$ 0.001	0.051 $\pm$ 0.001	22.54 $\pm$ 0.01
	GSNB	0.819 $\pm$ 0.006	21.45 $\pm$ 0.13	0.846 $\pm$ 0.004	0.165 $\pm$ 0.002	0.052 $\pm$ 0.001	22.57 $\pm$ 0.01
	NRF-GS	0.522 $\pm$ 0.008	15.32 $\pm$ 0.18	0.854 $\pm$ 0.001	0.158 $\pm$ 0.001	0.045 $\pm$ 0.001	23.52 $\pm$ 0.01
M60	3DGS	1.652 $\pm$ 0.006	14.73 $\pm$ 0.06	0.903 $\pm$ 0.001	0.120 $\pm$ 0.001	0.023 $\pm$ 0.001	27.82 $\pm$ 0.05
	VDGS	2.349 $\pm$ 0.003	27.78 $\pm$ 0.08	0.904 $\pm$ 0.001	0.119 $\pm$ 0.001	0.022 $\pm$ 0.001	28.05 $\pm$ 0.05

Appendix

Table 9 : Per-scene test-set results on Tanks and Temples, continued.

Explore similar work

Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned Parameterization
Sep 14, 2026YuanHang Wang, Xin Cao, Yi ZhangDynamic 3D Gaussian Splatting Rasterization
University of Technology Sydney
RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects
Sep 16, 2026Xiaobiao Du, Yida Wang, Cheng Bi +2Gaussian Splatting Novel View Synthesis
University of Technology Sydney · Li Auto Inc. · Adelaide University
Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting
May 1, 2026Zhenhua Ning, Xin Li, Jun Yu +3Semantic Blending Heuristics
Pengcheng Laboratory, Shenzhen · Harbin Institute of Technology, Shenzhen

Sep 14, 2026, cs.AIJ/K move · Enter open · S save

Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned Parameterization

YuanHang Wang, Xin Cao, Yi Zhang

University of Technology Sydney

Dynamic 3D Gaussian Splatting Rasterization Controlled Study Taxonomy

Open paper AskPDF arXiv

Three-dimensional Gaussian Splatting (3DGS) combines explicit primitives with efficient rasterization, yet recent systems increasingly use neural networks to generate or share Gaussian parameters. We characterize this trend along five axes: attribute decoding, spatial sharing, view-conditioned decoding, topology generation, and amortized inference. An analysis of 19 representative methods shows that these choices address different limitations and cannot be reduced to a binary neural label. We also isolate three forms of neural parameterization in a controlled mip-NeRF 360 study. Sharing appearance and opacity improves reconstruction quality, while decoding geometric structure offers no further gain. The evidence favors selective neuralization: shared functions help when they capture reusable correlations without sacrificing the local geometric freedom of explicit splats.

Abstract

Explore similar work

Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned Parameterization

RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects

Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting