Compact Neural Appearance Models for Efficient Gaussian Splatting

Florian Hahlbohm; Jorge Condor; Linus Franke; Martin Eisemann; Marcus Magnor

cs.CVSep 4, 2026

Compact Neural Appearance Models for Efficient Gaussian Splatting

Authors: Florian Hahlbohm, Jorge Condor, Linus Franke, Martin Eisemann, Marcus Magnor

Organizations: TU Braunschweig · Universit`a della Svizzera italiana · University of Würzburg · University of New Mexico

Abstract

Explicit primitive-based radiance fields such as 3D Gaussian Splatting typically model view-dependent appearance using low-order spherical harmonics (SH). Although efficient to evaluate, SH coefficients dominate per-primitive storage and memory traffic, while their band-limited basis restricts angular detail. We present a thorough, end-to-end comparison of SH and recent spherical appearance models and introduce an implicit alternative that decodes compact per-primitive latent codes using a tiny shared MLP. We integrate all models into the same optimized pipeline, fusing their forward and backward passes into a differentiable CUDA rasterizer and provide a portable WebGL viewer for laptop and mobile GPUs. Our evaluation across reconstruction quality, memory use, and optimization and rendering performance shows that recent spherical models offer the strongest overall quality-efficiency trade-off. Our neural representation is the most compact model evaluated and, compared to third-degree SH, reduces the per-primitive appearance footprint from 192 to 28 bytes, accelerates optimization by 1.3 $\times$ , while improving reconstruction quality. We further analyze how appearance parametrization shapes optimization, identifying differences in recovered geometry and the tendency of expressive models to absorb non-static scene content. Together, our framework and analysis provide practical guidance for replacing SH beyond what image metrics alone can capture.

Figures & tables

Figure 1: 3DGS [ 31 ] employs third-degree spherical harmonics, totaling 45 view-dependent appearance parameters per Gaussian (left). Our model (right) achieves faster training

/\penalty

rendering with similar quality using only 8 features per Gaussian and a tiny MLP with shared weights.

		Mip-NeRF 360 [ 5 ]						Tanks & Temples [ 34 ] $/\penalty$ Deep Blending [ 24 ]						NeRF Synthetic [ 41 ]
Basis	Bytes $/\penalty$ G	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑
None	12	27.73	0.825	0.234	3m32s	4.6GiB	443.1	27.65	0.888	0.237	2m17s	3.3GiB	763.3	31.46	0.959	0.048	57s	1.3GiB	1734
SH	192	28.36	0.836	0.222	6m12s	6.7GiB	378.9	27.96	0.893	0.228	4m10s	4.9GiB	683.2	34.01	0.971	0.036	1m22s	1.5GiB	1696
SV	208	28.64	0.834	0.225	6m30s	6.8GiB	385.0	27.98	0.893	0.232	4m20s	5.0GiB	696.6	33.45	0.968	0.038	1m28s	1.6GiB	1576
NASG	44	28.50	0.836	0.223	3m54s	4.9GiB	439.9	27.97	0.893	0.227	2m36s	3.6GiB	757.4	32.69	0.966	0.043	59s	1.3GiB	1826
NASGabor	48	28.54	0.836	0.223	4m00s	5.0GiB	436.9	27.96	0.893	0.227	2m39s	3.6GiB	743.6	32.95	0.967	0.041	59s	1.3GiB	1852

Table 1: Quantitative results on established benchmark datasets using MCMC densification [ 32 ] . We employ PPISP [ 13 ] on real scenes to handle photometric variations and measure frames per second (FPS) at the native scene resolution. Since image metrics vary between runs, we report average values across five runs in this table to improve reproducibility.

		Bicycle (6M Gaussians)					Stump (4.5M Gaussians)					Bonsai (1.5M Gaussians)
	Device	SH	SV	NASG	NASGabor	Neural	SH	SV	NASG	NASGabor	Neural	SH	SV	NASG	NASGabor	Neural
CUDA	Desktop (RTX 5090)	2.11	2.16	1.85	1.87	1.97	2.27	2.29	2.11	2.00	1.98	1.46	1.42	1.32	1.28	1.36
	Desktop (RTX 4090)	2.75	2.80	2.25	2.26	2.38	2.95	3.00	2.55	2.43	2.36	1.76	1.71	1.51	1.46	1.55
	Desktop (RTX 3090)	4.27	4.36	3.82	3.83	4.06	4.54	4.66	4.28	4.06	4.02	2.90	2.85	2.66	2.60	2.75
WebGL	Desktop (RTX 4090)	4.0	4.5	4.0	4.0	4.0	3.6	4.0	3.6	3.6	3.9	1.3	1.4	1.2	1.3	1.2
	MacBook (M1 Pro)	12.2	13.1	12.0	12.0	13.2	11.1	12.1	11.0	11.0	11.9	6.1	6.4	6.1	6.1	6.4

Table 2: Average frame time at 720p in milliseconds across scenes, devices, and rendering backends. CPU-side depth sorting is excluded. For WebGL, we load the same checkpoint for all models, i.e . , geometry is identical, assigning dummy values for the appearance parameters, and exclude the CPU-side depth sorting as well as application of the PPISP module. Values rounded roughly w.r.t. to measurement precision.

Panel of Figure 2 — Figure 2: Greater angular expressivity can reduce geometric artifacts. Models with high angular expressivity separate view-dependent reflections from diffuse color, whereas less expressive models compensate through geometry. SH uses elongated white Gaussians to fake the reflection on the sketch block and misses the reflection on the bonsai pot, while NASGabor achieves the best diffuse $/\penalty$ specular decomposition.

Method	t Train	t Render	Angular Res.	Robustness	Simplicity	Compactness	Key Takeaway
SH	$-$	$-$	$\bm{--}$	$\bm{++}$	$+$	$-$	Overfits the least (strong SSIM $/\penalty$ LPIPS); geometry must compensate for sharp reflections. 1
SV	$-$	$-$	$+$	$-$	$-$	$-$	Parameter-efficient for sharp diffuse reflections; least robust method in our evaluation. 2
NASG(abor)	$+$	$+$	$\bm{++}$	$+$	$-$	$+$	Best quality $/\penalty$ speed trade-off; its sharp lobes generalize well and can explain capture artifacts. 3
Neural	$+$	$\circ$	$+$	$\circ$	$\bm{++}$	$\bm{++}$	Most compact in general settings; hard to regularize per scene. 4
1 Costly Adam updates must also touch Gaussians invisible in the current iteration; rendering incurs many memory fetches.
2 Ineffective with few sites (softmax requires $\geq 2$ ), making it slow and a poor fit for compact representations; explicit many-lobe models are hard to optimize via gradient descent.

Table 3: Study of the evaluated view-dependent appearance models. Ranking goes from best (

\bm{++}

) to worst (

\bm{--}

Panel of Figure 3 — Figure 3: Effect of color activation and opacity learning rate on a test view of the garden scene. The choice of parametrization has strong influence on how well the optimization recovers meaningful geometry as evident from the significantly improved details of the brick wall.

	SH		NASGabor		Neural
Method	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑
ReLU, lr=0.05	28.35	0.836	28.51	0.836	28.53	0.831
ReLU, lr=0.025	28.33	0.831	28.59	0.833	28.54	0.827
Softplus, lr=0.05	28.42	0.837	28.55	0.837	28.54	0.831
Softplus, lr=0.025	28.41	0.833	28.60	0.834	28.56	0.827
Hardsigmoid, lr=0.05	28.31	0.834	28.49	0.835	28.50	0.831

Table 4: Quality metrics under different color activations and opacity learning rates across Mip-NeRF 360 scenes [ 5 ] using MCMC densification and PPISP. The variant with lr=auto adjusts the opacity learning rate between 0.05 and 0.025 based on the scene-specific training view distribution as proposed by Miazga et al . [ 40 ] .

Panel of Figure 4 — Figure 4: Base color and view-dependent residual along a camera path between two training views of the bonsai scene, cropped to the books on the floor. The first and last crop show the reference views, the others show in-between views. The topmost book moved during the data capture. NASGabor’s expressivity allows partially fitting this artifact as a view-dependent effect, while SH requires the geometry to compensate through popping [ 48 ] , an insight that motivates employing expressive appearance models with pixel-sorted approaches [ 21 , 52 ] .

Panel of Figure 5 — Figure 5: Base color and view-dependent residual with SH appearance rendered separately for a top-down view of the garden scene. The residual reveals where the camera operator inadvertently modified the scene by stepping on the grass during the capture, i.e . , the optimizer fit changes in geometry through the appearance model.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Parameter	Learning rate	Schedule
SH	base color	$0.0025\cdot Y_{0}$	constant
SH	coefficients	0.000125	constant
SV	base color	0.0008	constant
	site colors	0.0008	constant
	sites	0.002	delayed cosine
	temperatures	0.006	delayed cosine

Appendix

Table 5: Learning rates and schedules of the appearance parameters. Cosine follows Miazga et al . [ 40 ] : decay from iteration 7k toward zero at 37k, i.e . , about 13% of the initial rate remain at the end of optimization. Delayed cosine follows Di Sario et al . [ 14 ] : frozen until 1k, decay to 0.1× at 27k, frozen after. Warm-up follows Mip-NeRF [ 4 ] : sinusoidal warm-up over 3k, then log-linear decay to 0.1× at 30k.

\bar{d}

is the mean distance between each training camera and its three nearest neighbors.

Model	Payload	Bytes $/\penalty$ G	Per-frame work
SH	45 coefficients at 7 $/\penalty$ 8 $/\penalty$ 6 bit	40	SH basis, weighted sum
SV	7 sites × 7 fp16	112	site distances, softmax
NASG	1 lobe × 12 fp16	32	lobe response
NASGabor	1 lobe × 13 fp16	32	lobe response, carrier
Neural	16 fp16 pre-activations	32	MLP

Appendix

Table 6: Per-Gaussian appearance payload of the WebGL viewer for the default configuration of each model and the per-frame work of the shader. Byte counts are padded to whole 16-byte texels, the base color is packed with the geometry, and the contents of each payload are described in the text.

	SH				NASGabor				Neural
Method	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓
MCMC [ 32 ]	28.35	0.836	0.222	2.91M	28.51	0.836	0.223	2.90M	28.53	0.831	0.227	2.96M
$\hookrightarrow$ with 0.1 $\times$ #Gs	27.30	0.798	0.309	0.32M	27.32	0.796	0.312	0.32M	27.30	0.793	0.316	0.32M
ADC [ 31 ]	27.88	0.818	0.250	2.79M	28.14	0.818	0.252	2.80M	28.04	0.812	0.258	2.88M
$\hookrightarrow$ with [ 23 ]	27.23	0.777	0.346	0.33M	27.29	0.779	0.343	0.33M	27.24	0.777	0.349	0.34M

Appendix

Table 7: Quality metrics across Mip-NeRF 360 scenes [ 5 ] with different densification strategies and PPISP [ 13 ] . #Gs refers to the number of potentially visible Gaussians in the final model, i.e . , those with opacity

\geq\nicefrac{{1}}{{255}}

after training.

Configuration	Outdoor	Indoor	All	Outdoor	Indoor	All	Outdoor	Indoor	All	Train ↓	VRAM ↓	FPS ↑	Bytes $/\penalty$ G	MAC $/\penalty$ G
	PSNR ↑			SSIM ↑			LPIPS ↓			Train ↓	VRAM ↓	FPS ↑	Bytes $/\penalty$ G	MAC $/\penalty$ G
1 hidden layer	25.37	32.15	28.39	0.753	0.933	0.833	0.232	0.219	0.226	4m14s	5.2GiB	432.8	28	304
3 hidden layers	25.25	32.71	28.57	0.745	0.935	0.829	0.238	0.217	0.229	4m24s	5.4GiB	435.7	28	816
32 neurons $/\penalty$ layer	25.38	32.67	28.62	0.749	0.935	0.832	0.234	0.217	0.226	4m29s	5.5GiB	449.0	28	2144
$F\!=\!16$ w $/\penalty$ o frequency encoding	25.31	32.63	28.56	0.748	0.935	0.831	0.236	0.217	0.227	4m42s	5.7GiB	438.5	44	560
Default w $/\penalty$ o fusion	25.34	32.58	28.56	0.748	0.935	0.831	0.234	0.218	0.227	16m08s	5.7GiB	112.1	28	560

Appendix

Table 8: Comparison of different model configurations for the neural appearance model across Mip-NeRF 360 scenes [ 5 ] . Our default configuration uses two hidden layers with 16 neurons and

F\!=\!8

frequency-encoded input features. MAC

/\penalty

G is the number of multiply-accumulate operations needed to evaluate the model for a single Gaussian in our WebGL implementation.

Panel of Figure 6 — Figure 6: Base color and view-dependent residual, visualized as in Fig. 2 , for a test view of four scenes. NASGabor achieves the best decomposition, with the residual limited to reflections such as those on the desk in Dr Johnson . The SV residual often also models diffuse appearance. The neural model incorrectly represents the highlight on the desk in Dr Johnson with the base color instead of the residual.

PSNR $\uparrow$ on Mip-NeRF 360 [ 5 ]
Method	Bicycle	Flowers	Garden	Stump	Treehill	Bonsai	Counter	Kitchen	Room	Average
None	25.65 $\pm$ 0.0507	21.66 $\pm$ 0.0276	27.50 $\pm$ 0.0226	27.14 $\pm$ 0.0364	23.29 $\pm$ 0.0578	31.89 $\pm$ 0.0919	28.39 $\pm$ 0.0394	31.47 $\pm$ 0.0310	32.58 $\pm$ 0.0373	27.73 $\pm$ 0.0230
SH	26.19 $\pm$ 0.0337	22.15 $\pm$ 0.0238	27.96 $\pm$ 0.0167	27.73 $\pm$ 0.0346	23.40 $\pm$ 0.0275	32.97 $\pm$ 0.0251	29.49 $\pm$ 0.0348	32.35 $\pm$ 0.0354	32.98 $\pm$ 0.0550	28.36 $\pm$ 0.0094
SV	26.10 $\pm$ 0.0368	22.08 $\pm$ 0.0293	28.04 $\pm$ 0.0201	27.46 $\pm$ 0.0383	23.23 $\pm$ 0.0928	34.11 $\pm$ 0.0272	30.77 $\pm$ 0.0379	32.81 $\pm$ 0.0617	33.16 $\pm$ 0.0591	28.64 $\pm$ 0.0266
NASG	26.12 $\pm$ 0.0150	21.98 $\pm$ 0.0145	27.94 $\pm$ 0.0313	27.53 $\pm$ 0.0628	23.32 $\pm$ 0.0362	33.61 $\pm$ 0.0446	30.28 $\pm$ 0.0328	32.52 $\pm$ 0.0595	33.20 $\pm$ 0.0505	28.50 $\pm$ 0.0117
NASGabor	26.12 $\pm$ 0.0522	21.96 $\pm$ 0.0326	27.98 $\pm$ 0.0160	27.63 $\pm$ 0.0152	23.30 $\pm$ 0.0634	33.66 $\pm$ 0.0807	30.43 $\pm$ 0.0274	32.60 $\pm$ 0.0829	33.20 $\pm$ 0.0814	28.54 $\pm$ 0.0255

Appendix

Table 9: Per-scene image quality metrics on Mip-NeRF 360 [ 5 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

PSNR $\uparrow$ on Tanks & Temples [ 34 ] $/\penalty$ Deep Blending [ 24 ]
Method	Train	Truck	Drjohnson	Playroom	Average
None	24.10 $\pm$ 0.1286	26.78 $\pm$ 0.0207	29.34 $\pm$ 0.0503	30.39 $\pm$ 0.1534	27.65 $\pm$ 0.0207
SH	24.48 $\pm$ 0.1266	27.13 $\pm$ 0.0268	29.43 $\pm$ 0.0867	30.80 $\pm$ 0.0714	27.96 $\pm$ 0.0294
SV	24.54 $\pm$ 0.0993	27.37 $\pm$ 0.0263	29.33 $\pm$ 0.0875	30.68 $\pm$ 0.1853	27.98 $\pm$ 0.0357
NASG	24.39 $\pm$ 0.2059	27.21 $\pm$ 0.0164	29.55 $\pm$ 0.1153	30.73 $\pm$ 0.1417	27.97 $\pm$ 0.0396
NASGabor	24.53 $\pm$ 0.1562	27.22 $\pm$ 0.0288	29.51 $\pm$ 0.0779	30.60 $\pm$ 0.3347	27.96 $\pm$ 0.0855

Appendix

Table 10: Per-scene image quality metrics on Tanks & Temples [ 34 ]

/\penalty

Deep Blending [ 24 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

PSNR $\uparrow$ on NeRF Synthetic [ 41 ]
Method	Chair	Drums	Ficus	Hotdog	Lego	Materials	Mic	Ship	Average
None	33.60 $\pm$ 0.0199	25.40 $\pm$ 0.0390	32.04 $\pm$ 0.0585	33.49 $\pm$ 0.4511	34.83 $\pm$ 0.0125	28.22 $\pm$ 0.0821	34.70 $\pm$ 0.0382	29.36 $\pm$ 0.1582	31.46 $\pm$ 0.0873
SH	35.98 $\pm$ 0.0111	26.28 $\pm$ 0.0259	35.46 $\pm$ 0.3506	38.35 $\pm$ 0.0920	36.14 $\pm$ 0.0059	30.91 $\pm$ 0.0060	37.48 $\pm$ 0.0638	31.44 $\pm$ 0.2776	34.01 $\pm$ 0.0478
SV	36.34 $\pm$ 0.0074	27.00 $\pm$ 0.0316	30.65 $\pm$ 2.3306	36.53 $\pm$ 4.2332	36.36 $\pm$ 0.0174	31.08 $\pm$ 0.0142	37.80 $\pm$ 0.0144	31.88 $\pm$ 0.0505	33.45 $\pm$ 0.6306
NASG	34.77 $\pm$ 0.0090	26.34 $\pm$ 0.0389	33.40 $\pm$ 0.0802	36.35 $\pm$ 0.9041	35.60 $\pm$ 0.0560	28.94 $\pm$ 0.0239	35.25 $\pm$ 3.1155	30.87 $\pm$ 0.1773	32.69 $\pm$ 0.3724
NASGabor	34.79 $\pm$ 0.0145	26.32 $\pm$ 0.0196	33.35 $\pm$ 0.0367	37.03 $\pm$ 0.6098	35.70 $\pm$ 0.0144	28.96 $\pm$ 0.0865	36.59 $\pm$ 0.0413	30.89 $\pm$ 0.1202	32.95 $\pm$ 0.0763

Appendix

Table 11: Per-scene image quality metrics on NeRF Synthetic [ 41 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

Explore similar work

Apr 16, 2026

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

We introduce a hybrid Gaussian-hash-grid radiance representation for reconstructing 2D Gaussian scene models from multi-view images. Similar to NeST splatting, our approach reduces the entanglement between geometry and appearance common in NeRF-based models, but adds per-Gaussian latent features alongside hash-grid features to bias the optimizer toward a separation of low- and high-frequency scene components. This explicit frequency-based decomposition reduces the tendency of high-frequency texture to compensate for geometric errors. Encouraging Gaussians with hard opacity falloffs further strengthens the separation between geometry and appearance, improving both geometry reconstruction and rendering efficiency. Finally, probabilistic pruning combined with a sparsity-inducing BCE opacity loss allows redundant Gaussians to be turned off, yielding a minimal set of Gaussians sufficient to represent the scene. Using both synthetic and real-world datasets, we compare against the state of the art in Gaussian-based novel-view synthesis and demonstrate superior reconstruction fidelity with an order of magnitude fewer primitives.

Neel Kelkar, Simon Niedermayr, Klaus Engel +1

Technical University of Munich, Munich, Germany · Siemens Healthineers, Erlangen, Germany

3D Gaussian Splatting Multi-View 3D Reconstruction Representation Disentanglement Novel View Synthesis Gaussian Splatting

Ask PDF

Sep 29, 2026

NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

We revisit the role of appearance modeling in 3D Gaussian Splatting (3DGS) and show that limited expressiveness in view-dependent reflectance is a key driver of representation redundancy. In standard 3DGS, low-order spherical harmonics (SH) are used, restricting the splats' ability to model high-frequency directional effects, which is typically compensated by increasing the number of splats. We propose \emph{NRF-GS: Neural Residual Fields for Gaussian Splatting}, a hybrid representation that replaces per-splat SH-bases with a shared neural residual field. Each Gaussian encodes a compact set of appearance features and a lambertian base color, while a lightweight \emph{global scene-level MLP} predicts view-dependent residuals conditioned on viewing direction, distance, and per-splat features. This formulation enhances directional reflectance modeling by combining diffuse per-splat reflectance representations with a shared global function for high-frequency details, enabling both higher expressiveness and parameter sharing across splats. Our key insight is that by accurately capturing high-frequency directional reflectance, especially in specular regions, the GS-representation becomes more expressive, reducing the need for geometrically redundant splats. As a result, NRF-GS achieves comparable or better rendering quality while reducing the number of Gaussians by up to 50%, and produces visibly improved specular and high-frequency details.

Pratik Singh Bisht, Andreas Kolb

Computer Graphics Group University of Siegen

3DGS Compression 3D Gaussian Splatting Neural Rendering

Ask PDF

Jun 8, 2026

Beyond Spherical Harmonics: Rethinking Appearance Models for Radiance Reconstruction

View-dependent appearance modeling remains a challenging problem in novel-view synthesis and reconstruction. Accurately representing complex angular effects often requires substantial memory and computational resources. For new learning-based methods, a common approach is to rely on SH. However, capturing high-frequency phenomena such as specular reflections demands high-order expansions, which increase memory usage and computational cost. Consequently, most methods employ low-order SH, which limits the ability to model complex view-dependent effects, resulting in overly smooth or diffuse representations. To address these limitations, we systematically evaluate a wide range of spherical functions in the context of scene reconstruction. Some of them are introduced to graphics and computer vision for the first time in this paper. Based on the insights from the experiment, we develop a novel spherical formulation, the Normalized Anisotropic Spherical Gabor function that enables efficient modeling and learning of high-frequency appearance effects while maintaining compact representation. Compared to existing approaches, our function achieves higher-quality reconstruction of view-dependent phenomena such as glints, while being up to five times more memory-efficient and more efficient to evaluate. We validate its performance in radiance-field reconstruction tasks.

Ewa Miazga, Jorge Condor, Piotr Didyk

1École Polytechnique Fédérale de Lausanne, Switzerland · 2Università della Svizzera Italiana, Lugano, Switzerland

Novel View Synthesis 3D Scene Representation Spherical Harmonics Neural Radiance Fields

Ask PDF