Compact Neural Appearance Models for Efficient Gaussian Splatting

Florian Hahlbohm; Jorge Condor; Linus Franke; Martin Eisemann; Marcus Magnor

cs.CVSep 4, 2026

Compact Neural Appearance Models for Efficient Gaussian Splatting

Authors: Florian Hahlbohm, Jorge Condor, Linus Franke, Martin Eisemann, Marcus Magnor

Organizations: TU Braunschweig · Universit`a della Svizzera italiana · University of Würzburg · University of New Mexico

Abstract

Explicit primitive-based radiance fields such as 3D Gaussian Splatting typically model view-dependent appearance using low-order spherical harmonics (SH). Although efficient to evaluate, SH coefficients dominate per-primitive storage and memory traffic, while their band-limited basis restricts angular detail. We present a thorough, end-to-end comparison of SH and recent spherical appearance models and introduce an implicit alternative that decodes compact per-primitive latent codes using a tiny shared MLP. We integrate all models into the same optimized pipeline, fusing their forward and backward passes into a differentiable CUDA rasterizer and provide a portable WebGL viewer for laptop and mobile GPUs. Our evaluation across reconstruction quality, memory use, and optimization and rendering performance shows that recent spherical models offer the strongest overall quality-efficiency trade-off. Our neural representation is the most compact model evaluated and, compared to third-degree SH, reduces the per-primitive appearance footprint from 192 to 28 bytes, accelerates optimization by 1.3 $\times$ , while improving reconstruction quality. We further analyze how appearance parametrization shapes optimization, identifying differences in recovered geometry and the tendency of expressive models to absorb non-static scene content. Together, our framework and analysis provide practical guidance for replacing SH beyond what image metrics alone can capture.

Figures & tables

Figure 1: 3DGS [ 31 ] employs third-degree spherical harmonics, totaling 45 view-dependent appearance parameters per Gaussian (left). Our model (right) achieves faster training

/\penalty

rendering with similar quality using only 8 features per Gaussian and a tiny MLP with shared weights.

		Mip-NeRF 360 [ 5 ]						Tanks & Temples [ 34 ] $/\penalty$ Deep Blending [ 24 ]						NeRF Synthetic [ 41 ]
Basis	Bytes $/\penalty$ G	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑	PSNR ↑	SSIM ↑	LPIPS ↓	Train ↓	VRAM ↓	FPS ↑
None	12	27.73	0.825	0.234	3m32s	4.6GiB	443.1	27.65	0.888	0.237	2m17s	3.3GiB	763.3	31.46	0.959	0.048	57s	1.3GiB	1734
SH	192	28.36	0.836	0.222	6m12s	6.7GiB	378.9	27.96	0.893	0.228	4m10s	4.9GiB	683.2	34.01	0.971	0.036	1m22s	1.5GiB	1696
SV	208	28.64	0.834	0.225	6m30s	6.8GiB	385.0	27.98	0.893	0.232	4m20s	5.0GiB	696.6	33.45	0.968	0.038	1m28s	1.6GiB	1576
NASG	44	28.50	0.836	0.223	3m54s	4.9GiB	439.9	27.97	0.893	0.227	2m36s	3.6GiB	757.4	32.69	0.966	0.043	59s	1.3GiB	1826
NASGabor	48	28.54	0.836	0.223	4m00s	5.0GiB	436.9	27.96	0.893	0.227	2m39s	3.6GiB	743.6	32.95	0.967	0.041	59s	1.3GiB	1852

Table 1: Quantitative results on established benchmark datasets using MCMC densification [ 32 ] . We employ PPISP [ 13 ] on real scenes to handle photometric variations and measure frames per second (FPS) at the native scene resolution. Since image metrics vary between runs, we report average values across five runs in this table to improve reproducibility.

		Bicycle (6M Gaussians)					Stump (4.5M Gaussians)					Bonsai (1.5M Gaussians)
	Device	SH	SV	NASG	NASGabor	Neural	SH	SV	NASG	NASGabor	Neural	SH	SV	NASG	NASGabor	Neural
CUDA	Desktop (RTX 5090)	2.11	2.16	1.85	1.87	1.97	2.27	2.29	2.11	2.00	1.98	1.46	1.42	1.32	1.28	1.36
	Desktop (RTX 4090)	2.75	2.80	2.25	2.26	2.38	2.95	3.00	2.55	2.43	2.36	1.76	1.71	1.51	1.46	1.55
	Desktop (RTX 3090)	4.27	4.36	3.82	3.83	4.06	4.54	4.66	4.28	4.06	4.02	2.90	2.85	2.66	2.60	2.75
WebGL	Desktop (RTX 4090)	4.0	4.5	4.0	4.0	4.0	3.6	4.0	3.6	3.6	3.9	1.3	1.4	1.2	1.3	1.2
	MacBook (M1 Pro)	12.2	13.1	12.0	12.0	13.2	11.1	12.1	11.0	11.0	11.9	6.1	6.4	6.1	6.1	6.4

Table 2: Average frame time at 720p in milliseconds across scenes, devices, and rendering backends. CPU-side depth sorting is excluded. For WebGL, we load the same checkpoint for all models, i.e . , geometry is identical, assigning dummy values for the appearance parameters, and exclude the CPU-side depth sorting as well as application of the PPISP module. Values rounded roughly w.r.t. to measurement precision.

Panel of Figure 2 — Figure 2: Greater angular expressivity can reduce geometric artifacts. Models with high angular expressivity separate view-dependent reflections from diffuse color, whereas less expressive models compensate through geometry. SH uses elongated white Gaussians to fake the reflection on the sketch block and misses the reflection on the bonsai pot, while NASGabor achieves the best diffuse $/\penalty$ specular decomposition.

Method	t Train	t Render	Angular Res.	Robustness	Simplicity	Compactness	Key Takeaway
SH	$-$	$-$	$\bm{--}$	$\bm{++}$	$+$	$-$	Overfits the least (strong SSIM $/\penalty$ LPIPS); geometry must compensate for sharp reflections. 1
SV	$-$	$-$	$+$	$-$	$-$	$-$	Parameter-efficient for sharp diffuse reflections; least robust method in our evaluation. 2
NASG(abor)	$+$	$+$	$\bm{++}$	$+$	$-$	$+$	Best quality $/\penalty$ speed trade-off; its sharp lobes generalize well and can explain capture artifacts. 3
Neural	$+$	$\circ$	$+$	$\circ$	$\bm{++}$	$\bm{++}$	Most compact in general settings; hard to regularize per scene. 4
1 Costly Adam updates must also touch Gaussians invisible in the current iteration; rendering incurs many memory fetches.
2 Ineffective with few sites (softmax requires $\geq 2$ ), making it slow and a poor fit for compact representations; explicit many-lobe models are hard to optimize via gradient descent.

Table 3: Study of the evaluated view-dependent appearance models. Ranking goes from best (

\bm{++}

) to worst (

\bm{--}

Panel of Figure 3 — Figure 3: Effect of color activation and opacity learning rate on a test view of the garden scene. The choice of parametrization has strong influence on how well the optimization recovers meaningful geometry as evident from the significantly improved details of the brick wall.

	SH		NASGabor		Neural
Method	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑
ReLU, lr=0.05	28.35	0.836	28.51	0.836	28.53	0.831
ReLU, lr=0.025	28.33	0.831	28.59	0.833	28.54	0.827
Softplus, lr=0.05	28.42	0.837	28.55	0.837	28.54	0.831
Softplus, lr=0.025	28.41	0.833	28.60	0.834	28.56	0.827
Hardsigmoid, lr=0.05	28.31	0.834	28.49	0.835	28.50	0.831

Table 4: Quality metrics under different color activations and opacity learning rates across Mip-NeRF 360 scenes [ 5 ] using MCMC densification and PPISP. The variant with lr=auto adjusts the opacity learning rate between 0.05 and 0.025 based on the scene-specific training view distribution as proposed by Miazga et al . [ 40 ] .

Panel of Figure 4 — Figure 4: Base color and view-dependent residual along a camera path between two training views of the bonsai scene, cropped to the books on the floor. The first and last crop show the reference views, the others show in-between views. The topmost book moved during the data capture. NASGabor’s expressivity allows partially fitting this artifact as a view-dependent effect, while SH requires the geometry to compensate through popping [ 48 ] , an insight that motivates employing expressive appearance models with pixel-sorted approaches [ 21 , 52 ] .

Panel of Figure 5 — Figure 5: Base color and view-dependent residual with SH appearance rendered separately for a top-down view of the garden scene. The residual reveals where the camera operator inadvertently modified the scene by stepping on the grass during the capture, i.e . , the optimizer fit changes in geometry through the appearance model.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Parameter	Learning rate	Schedule
SH	base color	$0.0025\cdot Y_{0}$	constant
SH	coefficients	0.000125	constant
SV	base color	0.0008	constant
	site colors	0.0008	constant
	sites	0.002	delayed cosine
	temperatures	0.006	delayed cosine

Appendix

Table 5: Learning rates and schedules of the appearance parameters. Cosine follows Miazga et al . [ 40 ] : decay from iteration 7k toward zero at 37k, i.e . , about 13% of the initial rate remain at the end of optimization. Delayed cosine follows Di Sario et al . [ 14 ] : frozen until 1k, decay to 0.1× at 27k, frozen after. Warm-up follows Mip-NeRF [ 4 ] : sinusoidal warm-up over 3k, then log-linear decay to 0.1× at 30k.

\bar{d}

is the mean distance between each training camera and its three nearest neighbors.

Model	Payload	Bytes $/\penalty$ G	Per-frame work
SH	45 coefficients at 7 $/\penalty$ 8 $/\penalty$ 6 bit	40	SH basis, weighted sum
SV	7 sites × 7 fp16	112	site distances, softmax
NASG	1 lobe × 12 fp16	32	lobe response
NASGabor	1 lobe × 13 fp16	32	lobe response, carrier
Neural	16 fp16 pre-activations	32	MLP

Appendix

Table 6: Per-Gaussian appearance payload of the WebGL viewer for the default configuration of each model and the per-frame work of the shader. Byte counts are padded to whole 16-byte texels, the base color is packed with the geometry, and the contents of each payload are described in the text.

	SH				NASGabor				Neural
Method	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓	PSNR ↑	SSIM ↑	LPIPS ↓	#Gs ↓
MCMC [ 32 ]	28.35	0.836	0.222	2.91M	28.51	0.836	0.223	2.90M	28.53	0.831	0.227	2.96M
$\hookrightarrow$ with 0.1 $\times$ #Gs	27.30	0.798	0.309	0.32M	27.32	0.796	0.312	0.32M	27.30	0.793	0.316	0.32M
ADC [ 31 ]	27.88	0.818	0.250	2.79M	28.14	0.818	0.252	2.80M	28.04	0.812	0.258	2.88M
$\hookrightarrow$ with [ 23 ]	27.23	0.777	0.346	0.33M	27.29	0.779	0.343	0.33M	27.24	0.777	0.349	0.34M

Appendix

Table 7: Quality metrics across Mip-NeRF 360 scenes [ 5 ] with different densification strategies and PPISP [ 13 ] . #Gs refers to the number of potentially visible Gaussians in the final model, i.e . , those with opacity

\geq\nicefrac{{1}}{{255}}

after training.

Configuration	Outdoor	Indoor	All	Outdoor	Indoor	All	Outdoor	Indoor	All	Train ↓	VRAM ↓	FPS ↑	Bytes $/\penalty$ G	MAC $/\penalty$ G
	PSNR ↑			SSIM ↑			LPIPS ↓			Train ↓	VRAM ↓	FPS ↑	Bytes $/\penalty$ G	MAC $/\penalty$ G
1 hidden layer	25.37	32.15	28.39	0.753	0.933	0.833	0.232	0.219	0.226	4m14s	5.2GiB	432.8	28	304
3 hidden layers	25.25	32.71	28.57	0.745	0.935	0.829	0.238	0.217	0.229	4m24s	5.4GiB	435.7	28	816
32 neurons $/\penalty$ layer	25.38	32.67	28.62	0.749	0.935	0.832	0.234	0.217	0.226	4m29s	5.5GiB	449.0	28	2144
$F\!=\!16$ w $/\penalty$ o frequency encoding	25.31	32.63	28.56	0.748	0.935	0.831	0.236	0.217	0.227	4m42s	5.7GiB	438.5	44	560
Default w $/\penalty$ o fusion	25.34	32.58	28.56	0.748	0.935	0.831	0.234	0.218	0.227	16m08s	5.7GiB	112.1	28	560

Appendix

Table 8: Comparison of different model configurations for the neural appearance model across Mip-NeRF 360 scenes [ 5 ] . Our default configuration uses two hidden layers with 16 neurons and

F\!=\!8

frequency-encoded input features. MAC

/\penalty

G is the number of multiply-accumulate operations needed to evaluate the model for a single Gaussian in our WebGL implementation.

Panel of Figure 6 — Figure 6: Base color and view-dependent residual, visualized as in Fig. 2 , for a test view of four scenes. NASGabor achieves the best decomposition, with the residual limited to reflections such as those on the desk in Dr Johnson . The SV residual often also models diffuse appearance. The neural model incorrectly represents the highlight on the desk in Dr Johnson with the base color instead of the residual.

PSNR $\uparrow$ on Mip-NeRF 360 [ 5 ]
Method	Bicycle	Flowers	Garden	Stump	Treehill	Bonsai	Counter	Kitchen	Room	Average
None	25.65 $\pm$ 0.0507	21.66 $\pm$ 0.0276	27.50 $\pm$ 0.0226	27.14 $\pm$ 0.0364	23.29 $\pm$ 0.0578	31.89 $\pm$ 0.0919	28.39 $\pm$ 0.0394	31.47 $\pm$ 0.0310	32.58 $\pm$ 0.0373	27.73 $\pm$ 0.0230
SH	26.19 $\pm$ 0.0337	22.15 $\pm$ 0.0238	27.96 $\pm$ 0.0167	27.73 $\pm$ 0.0346	23.40 $\pm$ 0.0275	32.97 $\pm$ 0.0251	29.49 $\pm$ 0.0348	32.35 $\pm$ 0.0354	32.98 $\pm$ 0.0550	28.36 $\pm$ 0.0094
SV	26.10 $\pm$ 0.0368	22.08 $\pm$ 0.0293	28.04 $\pm$ 0.0201	27.46 $\pm$ 0.0383	23.23 $\pm$ 0.0928	34.11 $\pm$ 0.0272	30.77 $\pm$ 0.0379	32.81 $\pm$ 0.0617	33.16 $\pm$ 0.0591	28.64 $\pm$ 0.0266
NASG	26.12 $\pm$ 0.0150	21.98 $\pm$ 0.0145	27.94 $\pm$ 0.0313	27.53 $\pm$ 0.0628	23.32 $\pm$ 0.0362	33.61 $\pm$ 0.0446	30.28 $\pm$ 0.0328	32.52 $\pm$ 0.0595	33.20 $\pm$ 0.0505	28.50 $\pm$ 0.0117
NASGabor	26.12 $\pm$ 0.0522	21.96 $\pm$ 0.0326	27.98 $\pm$ 0.0160	27.63 $\pm$ 0.0152	23.30 $\pm$ 0.0634	33.66 $\pm$ 0.0807	30.43 $\pm$ 0.0274	32.60 $\pm$ 0.0829	33.20 $\pm$ 0.0814	28.54 $\pm$ 0.0255

Appendix

Table 9: Per-scene image quality metrics on Mip-NeRF 360 [ 5 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

PSNR $\uparrow$ on Tanks & Temples [ 34 ] $/\penalty$ Deep Blending [ 24 ]
Method	Train	Truck	Drjohnson	Playroom	Average
None	24.10 $\pm$ 0.1286	26.78 $\pm$ 0.0207	29.34 $\pm$ 0.0503	30.39 $\pm$ 0.1534	27.65 $\pm$ 0.0207
SH	24.48 $\pm$ 0.1266	27.13 $\pm$ 0.0268	29.43 $\pm$ 0.0867	30.80 $\pm$ 0.0714	27.96 $\pm$ 0.0294
SV	24.54 $\pm$ 0.0993	27.37 $\pm$ 0.0263	29.33 $\pm$ 0.0875	30.68 $\pm$ 0.1853	27.98 $\pm$ 0.0357
NASG	24.39 $\pm$ 0.2059	27.21 $\pm$ 0.0164	29.55 $\pm$ 0.1153	30.73 $\pm$ 0.1417	27.97 $\pm$ 0.0396
NASGabor	24.53 $\pm$ 0.1562	27.22 $\pm$ 0.0288	29.51 $\pm$ 0.0779	30.60 $\pm$ 0.3347	27.96 $\pm$ 0.0855

Appendix

Table 10: Per-scene image quality metrics on Tanks & Temples [ 34 ]

/\penalty

Deep Blending [ 24 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

PSNR $\uparrow$ on NeRF Synthetic [ 41 ]
Method	Chair	Drums	Ficus	Hotdog	Lego	Materials	Mic	Ship	Average
None	33.60 $\pm$ 0.0199	25.40 $\pm$ 0.0390	32.04 $\pm$ 0.0585	33.49 $\pm$ 0.4511	34.83 $\pm$ 0.0125	28.22 $\pm$ 0.0821	34.70 $\pm$ 0.0382	29.36 $\pm$ 0.1582	31.46 $\pm$ 0.0873
SH	35.98 $\pm$ 0.0111	26.28 $\pm$ 0.0259	35.46 $\pm$ 0.3506	38.35 $\pm$ 0.0920	36.14 $\pm$ 0.0059	30.91 $\pm$ 0.0060	37.48 $\pm$ 0.0638	31.44 $\pm$ 0.2776	34.01 $\pm$ 0.0478
SV	36.34 $\pm$ 0.0074	27.00 $\pm$ 0.0316	30.65 $\pm$ 2.3306	36.53 $\pm$ 4.2332	36.36 $\pm$ 0.0174	31.08 $\pm$ 0.0142	37.80 $\pm$ 0.0144	31.88 $\pm$ 0.0505	33.45 $\pm$ 0.6306
NASG	34.77 $\pm$ 0.0090	26.34 $\pm$ 0.0389	33.40 $\pm$ 0.0802	36.35 $\pm$ 0.9041	35.60 $\pm$ 0.0560	28.94 $\pm$ 0.0239	35.25 $\pm$ 3.1155	30.87 $\pm$ 0.1773	32.69 $\pm$ 0.3724
NASGabor	34.79 $\pm$ 0.0145	26.32 $\pm$ 0.0196	33.35 $\pm$ 0.0367	37.03 $\pm$ 0.6098	35.70 $\pm$ 0.0144	28.96 $\pm$ 0.0865	36.59 $\pm$ 0.0413	30.89 $\pm$ 0.1202	32.95 $\pm$ 0.0763

Appendix

Table 11: Per-scene image quality metrics on NeRF Synthetic [ 41 ] , averaged over five training runs with the standard deviation across runs. The three best results are highlighted in green in descending order of saturation.

Explore similar work

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting
Apr 16, 2026Neel Kelkar, Simon Niedermayr, Klaus Engel +13D Gaussian Splatting Multi-View 3D Reconstruction
Technical University of Munich, Munich, Germany · Siemens Healthineers, Erlangen, Germany
NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting
Sep 29, 2026Pratik Singh Bisht, Andreas Kolb3DGS Compression 3D Gaussian Splatting
Computer Graphics Group University of Siegen
Beyond Spherical Harmonics: Rethinking Appearance Models for Radiance Reconstruction
Jun 8, 2026Ewa Miazga, Jorge Condor, Piotr DidykNovel View Synthesis 3D Scene Representation
1École Polytechnique Fédérale de Lausanne, Switzerland · 2Università della Svizzera Italiana, Lugano, Switzerland

Apr 16, 2026J/K move · Enter open · S save

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

Neel Kelkar, Simon Niedermayr, Klaus Engel, Rüdiger Westermann

Technical University of Munich, Munich, Germany · Siemens Healthineers, Erlangen, Germany

3D Gaussian Splatting Multi-View 3D Reconstruction Representation Disentanglement Novel View Synthesis Gaussian Splatting

Open paper AskPDF arXiv

We introduce a hybrid Gaussian-hash-grid radiance representation for reconstructing 2D Gaussian scene models from multi-view images. Similar to NeST splatting, our approach reduces the entanglement between geometry and appearance common in NeRF-based models, but adds per-Gaussian latent features alongside hash-grid features to bias the optimizer toward a separation of low- and high-frequency scene components. This explicit frequency-based decomposition reduces the tendency of high-frequency texture to compensate for geometric errors. Encouraging Gaussians with hard opacity falloffs further strengthens the separation between geometry and appearance, improving both geometry reconstruction and rendering efficiency. Finally, probabilistic pruning combined with a sparsity-inducing BCE opacity loss allows redundant Gaussians to be turned off, yielding a minimal set of Gaussians sufficient to represent the scene. Using both synthetic and real-world datasets, we compare against the state of the art in Gaussian-based novel-view synthesis and demonstrate superior reconstruction fidelity with an order of magnitude fewer primitives.

Abstract

Explore similar work

Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting

NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

Beyond Spherical Harmonics: Rethinking Appearance Models for Radiance Reconstruction