Birth-Death Structural Learning for 3D Gaussian Splatting

Quan Tran Hong; Long Nguyen-Chi; Binh T. Nguyen

cs.CVOct 25, 2026

Birth-Death Structural Learning for 3D Gaussian Splatting

Authors: Quan Tran Hong, Long Nguyen-Chi, Binh T. Nguyen

Organizations: Center for AI Research, VinUniversity, Hanoi, Vietnam

Abstract

3D Gaussian Splatting (3DGS) has become a standard representation for real-time novel-view synthesis. Yet, its quality-efficiency trade-off still relies heavily on adaptive density control. Existing cloning, splitting, and pruning rules use proxy signals, such as position gradients, opacity, and image-space error. Although effective in practice, these signals do not directly answer the structural question behind density control: where should new Gaussians be allocated to reduce the multi-view reconstruction loss, and which existing primitives can be pruned with minimal penalty. To address this, we propose first-variation birth-death control for 3DGS, a principled approach that replaces heuristic structural decisions with a variational score derived from the reconstruction loss. At each structural update, we freeze the current Gaussian population, attach auxiliary mass coordinates to each primitive, and differentiate the multiview loss with respect to these coordinates. We demonstrate that this score is a coordinate of the first variation of the Splat Regression Model, giving a rigorous local descent surrogate for birth-death moves: high-score Gaussians are selected as birth sites, while low-score, low-opacity Gaussians are selected for removal. Across standard 3DGS benchmarks, integrating this score into 3DGS-MCMC yields state-of-the-art reconstruction quality and outperforms existing methods, delivering the most significant gains under tight Gaussian budgets. Ultimately, our work provides compelling evidence that gradient-based variational metrics offer a principled alternative to proxy heuristics in 3DGS.

Figures & tables

Figure 1: Overview of the score-guided birth–death controller. At each structural update, we freeze the current Gaussian parameters and expose one auxiliary mass coordinate per primitive, implemented as $a_{i}(x)\mapsto m_{i}a_{i}(x)$ . A batched multiview backward pass computes the first-variation score $q_{i}=-\partial\widehat{F}_{t}(\mathbf{m})/\partial m_{i}|_{\mathbf{m}=\mathbf{1}}$ . Higher scores explicitly predict a larger first-order loss reduction from assigning mass to that primitive. Guided by these scores, the controller kills low-opacity, low-score slots and births new capacity at high-score parent sites using the 3DGS-MCMC transitions.

Figure 2: Qualitative comparison on Tanks & Temples (Truck, top row) and Mip-NeRF360 (middle and bottom rows) at a 200k-Gaussian cap with random initialization. All methods use the same view, scale, and crop coordinates in each row. Additional examples are in Appendix K .

Method	Mip-NeRF360			Tanks & Temples			DeepBlending
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Number of Gaussians limited to 100k
3DGS [ 15 ] (SfM) †	23.61	0.693	0.413	22.38	0.774	0.333	24.65	0.827	0.412
Improved ADC [ 10 ]	26.34	0.781	0.322	22.08	0.784	0.280	28.25	0.885	0.315
3DGS-MCMC [ 17 ] (SfM) †	27.06	0.800	0.303	22.50	0.780	0.332	28.94	0.876	0.333
EDGS [ 19 ] †	27.09	0.798	0.296	22.32	0.777	0.324	28.43	0.872	0.337

Table 1: Fixed-budget comparison under matched Gaussian caps with SfM initialization for Ours. Mip-NeRF360 averages use the seven scenes listed in Section 4.1 . Higher PSNR and SSIM are better; lower LPIPS is better. We highlight the best , second best and third best results within each dataset-budget block.

Figure 3: Matched PSNR–budget comparison on the seven-scene Mip-NeRF360 subset. The renderer, optimizer, relocation kernel, and training schedule are fixed; only the birth–death statistic changes from opacity to first variation.

Variant	Selection statistic	$\Delta$ PSNR (dB) $\uparrow$
(a) MCMC baseline	$\alpha_{i}$	–
(b) Position gradient	$\\|\nabla_{\mu_{i}}\widehat{F}_{t}\\|$	+0.04
(c) Opacity gradient (post-act.)	$-\partial\widehat{F}_{t}/\partial\alpha_{i}$	+0.11
(d) Birth only	$(q_{i})_{+}$	+0.28
(e) Death only	$q_{i}$	+0.05
(f) Ours (Full)	$q_{i}$	+0.33

Table 2: Controller-statistic ablation on Mip-NeRF360 (100k cap). Within-sweep gains over the matched opacity-controller rerun.

Figure 4: Qualitative comparison with absolute $L_{1}$ error maps. Rows, top to bottom: Dr. Johnson (1M), Counter (200k), and Bonsai (100k); all use random initialization. Brighter error-map pixels indicate larger error. Each row uses the same Gaussian cap, test view, and crop across methods.

Method	Tanks & Temples				Mip-NeRF360				DeepBlending
Method	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla 3DGS [ 15 ]	1577	23.79	0.853	0.169	2627	27.55	0.813	0.221	2475	29.82	0.907	0.238
LightGaussian-20% [ 8 ]	315	22.49	0.779	0.271	526	26.35	0.765	0.288	495	28.01	0.873	0.315
PUP-3DGS-20% [ 12 ]	315	22.60	0.809	0.228	525	26.67	0.790	0.257	495	29.72	0.905	0.248
Trimming the Fat-20% [ 1 ]	315	22.50	0.819	0.232	524	26.49	0.781	0.280	494	29.08	0.900	0.272
MesonGS-20% [ 31 ]	314	20.70	0.822	0.207	527	25.01	0.776	0.262	496	28.70	0.897	0.262

Table 3: Compact-model comparison with external results from GHAP [ 29 ] . Mip-NeRF360 averages all nine scenes here, including Flowers and Treehill. Ours uses a 500k cap; external counts and training protocols differ. Counts are in thousands. Colors denote 1st , 2nd , and 3rd among compact rows; Vanilla is excluded.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Value
Training iterations	30,000
Gaussian caps	100k, 200k, 500k, 1M, 2M
Initialization / seed	SfM or random initialization, seed 0
Controller interval $\Delta$	40 iterations
Score batch size $B$	16 views
EMA coefficient $\rho$	0.9

Appendix

Table 4: Experimental configuration and default hyperparameters.

	Reconstruction quality		Final #G (M)
Scene	Vanilla	Vanilla + $q_{i}$	Vanilla	+ $q_{i}$
Bicycle	25.13 / 0.747 / 0.242	25.29 / 0.774 / 0.202	4.769	3.271
Bonsai	32.46 / 0.947 / 0.180	32.67 / 0.952 / 0.151	2.167	1.064
Flowers	21.36 / 0.589 / 0.359	21.61 / 0.627 / 0.278	3.206	2.850

Appendix

Table 6: Uncapped Vanilla 3DGS transfer. Each quality entry reports PSNR

\uparrow

/ SSIM

\uparrow

/ LPIPS

\downarrow

; final Gaussian counts are in millions.

Scene	Position gradient	First-variation score $q_{i}$
500k parent-selection diagnostic
Bonsai	29.81 / 0.908 / 0.277	30.20 / 0.912 / 0.266
Counter	27.67 / 0.874 / 0.267	27.95 / 0.877 / 0.261
Garden	24.51 / 0.689 / 0.361	24.58 / 0.691 / 0.357
Kitchen	28.05 / 0.878 / 0.218	28.94 / 0.883 / 0.210
Room	30.17 / 0.894 / 0.277	30.67 / 0.897 / 0.273

Appendix

Table 7: Capped Vanilla transfer. Entries report PSNR

\uparrow

/ SSIM

\uparrow

/ LPIPS

\downarrow

; the mean uses the five 500k scenes only. Final Bicycle/100k populations are 99,812 (position gradient) and 99,910 (

q_{i}

Cap	Structural updates	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
500k	Full updates	21.88	0.820	0.216
500k	Stop at 15k	22.36	0.823	0.217
1M	Full updates	22.06	0.835	0.190
1M	Stop at 15k	22.24	0.831	0.196

Appendix

Table 8: Structural-update cutoff on Tanks & Temples Train. Continuous optimization runs for 30k iterations in both variants. These are within-diagnostic comparisons, not replacements for the default benchmark results.

Method	Tanks & Temples				Mip-NeRF360				DeepBlending
Method	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#G	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla 3DGS [ 15 ]	1577	23.79	0.853	0.169	2627	27.55	0.813	0.221	2475	29.82	0.907	0.238
LightGaussian [ 8 ]	158	22.11	0.756	0.306	263	25.67	0.735	0.331	248	28.01	0.869	0.327
PUP-3DGS [ 12 ]	158	21.52	0.767	0.280	262	25.33	0.753	0.309	248	29.15	0.895	0.274
Trimming the Fat [ 1 ]	156	21.54	0.776	0.293	263	25.26	0.731	0.343	247	28.06	0.887	0.302
MesonGS [ 31 ]	157	20.71	0.811	0.208	263	24.92	0.773	0.264	248	28.69	0.896	0.264

Appendix

Table 9: Extreme low-primitive comparison (

\sim

10% retention) on Mip-NeRF360, Tanks & Temples, and DeepBlending. External compact-baseline values are from GHAP [ 29 ] . Vanilla 3DGS is shown only as a full-model reference and is not included in the rank highlighting. Rank colors are computed only among the low-primitive rows: best , second best and third best . Higher PSNR/SSIM and lower LPIPS are better. All Gaussian counts are in thousands.

Setting	Protocol	Mean $\Delta$ PSNR	Severe views	View-luma std.	Mean bright-pixel frac.
Train, 2M	Default	$-0.85$	$13/38$	$0.088$	$0.094$
Train, 2M	Exp. comp.	$-0.29$	$6/38$	–	–
Truck, 2M	Default	$-0.05$	$0/32$	$0.061$	$0.048$
Mip-NeRF360 avg., 2M	Default	$+0.06$	$3/246$	$0.062$	$0.031$

Appendix

Table 10: Diagnostic evidence for the high-budget Tanks & Temples Train behavior.

\Delta

PSNR denotes Ours minus the matched 3DGS-MCMC rerun. The exposure-compensation row reruns both 3DGS-MCMC and Ours under the same exposure-compensation protocol; for Train at 2M, MCMC/Ours PSNR changes from

22.95/22.10

24.95/24.65

. This is a paired diagnostic, not a replacement for the default main-table protocol. Severe views denote

\Delta

PSNR

<-1

dB.

Figure 5: Photometric diagnostic for the high-budget Train case. Left and middle: Train has higher view-level brightness variation and a larger bright-pixel fraction than Truck and the Mip-NeRF360 average. Right: paired exposure-compensation reruns for both 3DGS-MCMC and Ours reduce the mean Train/SfM/2M gap from

-0.85

dB to

-0.29

dB and reduce severe views from 13 to 6. This supports the interpretation that the high-budget regression is tied to residual reliability under photometric variation, not to a general breakdown of first-variation allocation.

Dataset	Method	100k	500k	1M	2M
		(P / S / L)	(P / S / L)	(P / S / L)	(P / S / L)
Mip-NeRF360	3DGS [ 15 ] (SfM) †	23.61 / 0.693 / 0.413	28.22 / 0.821 / 0.260	28.71 / 0.846 / 0.222	29.01 / 0.861 / 0.200
	Improved ADC [ 10 ]	26.34 / 0.781 / 0.322	28.46 / 0.843 / 0.232	28.94 / 0.858 / 0.205	29.21 / 0.871 / 0.184
	3DGS-MCMC [ 17 ] (SfM) †	27.06 / 0.800 / 0.303	28.82 / 0.861 / 0.214	29.23 / 0.875 / 0.190	29.52 / 0.883 / 0.172
	EDGS [ 19 ] †	27.09 / 0.798 / 0.296	28.82 / 0.865 / 0.193	29.18 / 0.877 / 0.168	29.38 / 0.883 / 0.155
	ConeGS [ 2 ] †	27.74 / 0.809 / 0.285	29.08 / 0.870 / 0.190	29.37 / 0.880 / 0.168	29.43 / 0.881 / 0.158

Appendix

Table 11: Full fixed-budget comparison across datasets, capacities, and methods. Each cell reports PSNR

\uparrow

/ SSIM

\uparrow

/ LPIPS

\downarrow

. Rows marked

\dagger

are published ConeGS benchmark values; Improved ADC and Ours are evaluated at the same Gaussian caps.

Scene	ConeGS †	3DGS (SfM) †	3DGS-MCMC (SfM) †	EDGS †	Ours
Mip-NeRF360
Bicycle	24.05 / 0.652 / 0.379	20.99 / 0.470 / 0.543	23.34 / 0.630 / 0.400	23.68 / 0.635 / 0.393	23.57 / 0.636 / 0.393
Garden	24.69 / 0.709 / 0.336	22.87 / 0.589 / 0.445	24.51 / 0.710 / 0.350	24.47 / 0.706 / 0.341	25.12 / 0.747 / 0.306
Stump	25.71 / 0.707 / 0.351	22.79 / 0.547 / 0.498	25.16 / 0.680 / 0.370	25.08 / 0.673 / 0.377	25.41 / 0.695 / 0.363
Room	31.09 / 0.903 / 0.254	27.02 / 0.859 / 0.328	30.37 / 0.900 / 0.270	30.08 / 0.896 / 0.265	30.94 / 0.903 / 0.261
Counter	28.22 / 0.879 / 0.251	25.64 / 0.845 / 0.307	27.83 / 0.880 / 0.260	27.77 / 0.877 / 0.251	28.44 / 0.895 / 0.232

Appendix

Table 12: Per-scene fixed-budget comparison at 100k Gaussians. Each cell reports PSNR / SSIM / LPIPS. Baseline values for ConeGS, 3DGS (SfM), 3DGS-MCMC (SfM), and EDGS are published ConeGS per-scene results; Ours is evaluated at the same cap.

Scene	ConeGS †	3DGS (SfM) †	3DGS-MCMC (SfM) †	EDGS †	Ours
Mip-NeRF360
Bicycle	25.25 / 0.758 / 0.242	24.06 / 0.651 / 0.370	24.90 / 0.740 / 0.282	24.95 / 0.751 / 0.251	25.14 / 0.753 / 0.265
Garden	26.87 / 0.837 / 0.157	25.16 / 0.743 / 0.296	26.36 / 0.823 / 0.189	26.49 / 0.833 / 0.161	26.79 / 0.837 / 0.170
Stump	27.03 / 0.793 / 0.215	25.27 / 0.688 / 0.348	26.81 / 0.779 / 0.250	26.50 / 0.768 / 0.237	27.21 / 0.795 / 0.229
Room	32.02 / 0.924 / 0.205	31.47 / 0.913 / 0.234	31.72 / 0.921 / 0.221	31.36 / 0.923 / 0.203	31.84 / 0.922 / 0.219
Counter	28.87 / 0.906 / 0.194	28.86 / 0.901 / 0.213	28.95 / 0.907 / 0.205	29.05 / 0.912 / 0.184	29.06 / 0.911 / 0.200

Appendix

Table 13: Per-scene fixed-budget comparison at 500k Gaussians. Each cell reports PSNR / SSIM / LPIPS. Baseline values for ConeGS, 3DGS (SfM), 3DGS-MCMC (SfM), and EDGS are published ConeGS per-scene results; Ours is evaluated at the same cap.

Scene	ConeGS †	3DGS (SfM) †	3DGS-MCMC (SfM) †	EDGS †	Ours
Mip-NeRF360
Bicycle	25.45 / 0.778 / 0.198	24.40 / 0.688 / 0.325	25.34 / 0.768 / 0.238	25.26 / 0.778 / 0.199	25.51 / 0.780 / 0.220
Garden	27.42 / 0.861 / 0.115	26.70 / 0.827 / 0.172	26.83 / 0.847 / 0.147	27.05 / 0.857 / 0.118	27.45 / 0.862 / 0.126
Stump	27.23 / 0.802 / 0.184	25.64 / 0.719 / 0.301	27.21 / 0.798 / 0.213	26.71 / 0.783 / 0.203	27.56 / 0.814 / 0.193
Room	32.30 / 0.928 / 0.196	31.62 / 0.918 / 0.221	32.09 / 0.926 / 0.208	31.71 / 0.928 / 0.191	32.10 / 0.927 / 0.205
Counter	29.21 / 0.911 / 0.181	29.11 / 0.907 / 0.201	29.24 / 0.913 / 0.191	29.27 / 0.917 / 0.171	29.34 / 0.917 / 0.187

Appendix

Table 14: Per-scene fixed-budget comparison at 1M Gaussians. Each cell reports PSNR / SSIM / LPIPS. Baseline values for ConeGS, 3DGS (SfM), 3DGS-MCMC (SfM), and EDGS are published ConeGS per-scene results; Ours is evaluated at the same cap.

Figure 6: Quality as a function of Gaussian budget on the Mip-NeRF360 comparison set under random and SfM initialization. Each point is a seed-0 dataset average at a fixed Gaussian cap; error bands are not shown because matched multi-seed runs are not available for this sweep. Panels use tight y-axis ranges within each initialization and metric to make the matched Ours-vs-3DGS-MCMC gap readable. Across PSNR, SSIM, and LPIPS, the proposed controller improves over the matched 3DGS-MCMC backend, with the largest gains at lower budgets.

Figure 7: Quality–time tradeoff on the Mip-NeRF360 SfM setting. Ours improves PSNR at matched Gaussian caps while adding score-pass overhead during training; inference cost remains tied to the same fixed Gaussian budget.

Cap	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Train time	Overhead	Score-pass	Base train
100k	3DGS-MCMC	27.61	0.814	0.281	10.75m	1.00 $\times$	–	–
100k	Ours	27.94	0.818	0.276	13.65m	1.27 $\times$	2.90m	10.75m
200k	3DGS-MCMC	28.04	0.835	0.254	11.79m	1.00 $\times$	–	–
200k	Ours	28.37	0.841	0.249	15.01m	1.27 $\times$	3.22m	11.79m
500k	3DGS-MCMC	28.89	0.864	0.209	16.16m	1.00 $\times$	–	–
500k	Ours	29.11	0.869	0.203	20.47m	1.27 $\times$	4.31m	16.16m

Appendix

Table 15: Quality–time and computational efficiency values on the Mip-NeRF360 SfM setting. MCMC time is estimated by subtracting measured score-pass time from the matched Ours wall-clock logs. Peak memory and FPS are omitted because complete matched measurements were not logged for this timing protocol.

Figure 8: Utility-score validation for the fixed-budget controller. Top: choosing high-score Gaussians gives larger held-out loss reduction after a small test copy; choosing low-score Gaussians better identifies slots that are safe to recycle; and the score agrees with finite removal damage. Bottom: mean

\pm

standard error over disjoint-view probes. Copy and recycle use 18 samples; removal agreement uses 16 finite Spearman correlations.

Probe setting	Spearman $\uparrow$	Top-10% gain $\uparrow$	Death P@10 $\uparrow$
Same views as score batch	$0.954\pm 0.007$	$1.442\pm 0.081$	$0.929\pm 0.026$
Disjoint probe views	$0.613\pm 0.029$	$0.216\pm 0.107$	$0.560\pm 0.070$
Held-out test views	$0.632\pm 0.026$	$0.600\pm 0.108$	$0.615\pm 0.055$

Appendix

Table 16: Effect of probe-view choice in utility-score validation. The main paper uses disjoint probe views to avoid measuring the derivative on the same views used to compute the score. Values report mean

\pm

standard error over 18 validation samples.

Figure 9: Qualitative comparison under random and SfM initialization. For each scene, we compare the matched 3DGS-MCMC controller and our score-guided controller under both initialization regimes. This comparison verifies that the visual gains are not tied to a particular initialization choice. All columns use the same Gaussian cap, test view, and crop coordinates within each row.

Figure 10: Qualitative comparison across scenes and budgets. Each row shows one test-view crop with a fixed scene, Gaussian cap, and random initialization. From top to bottom, the rows correspond to Dr. Johnson (1M, random), Garden (500k, random), Counter (200k, random), Bonsai (100k, random), and Bicycle (100k, random). From left to right, we show the ground-truth image, the matched 3DGS-MCMC rendering, the per-pixel absolute $L_{1}$ error map $|I_{\mathrm{GT}}-I_{\mathrm{MCMC}}|$ , our rendering, and the per-pixel absolute $L_{1}$ error map $|I_{\mathrm{GT}}-I_{\mathrm{ours}}|$ . Brighter pixels in the error maps indicate larger reconstruction error. All methods use the same test view, scale, and crop coordinates within each row.

Figure 11: All-scene qualitative comparison on Tanks & Temples and DeepBlending at a 200k-Gaussian cap with random initialization. Each row uses the same test view and crop coordinates for the ground truth, 3DGS-MCMC, and our method. These examples show that the score-guided controller also improves visual fidelity outside Mip-NeRF360, especially on thin structures, object boundaries, and repeated textures.

Figure 12: All-scene qualitative comparison on Mip-NeRF360 at a 200k-Gaussian cap with random initialization. Each row corresponds to one Mip-NeRF360 scene, and all methods use identical test views and crop coordinates. This figure complements the representative examples in Figure 2 by showing that the visual improvement is not restricted to a small number of selected scenes.

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,019 papers

Jul 18, 2026

SPARE-GS: Structural Parsimony and Resource Efficiency for 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) achieves high-fidelity novel view synthesis in real-time; however its training efficiency and representation compactness are hindered by excessive primitive proliferation. To address this challenge, we formulate the structural evolution of 3DGS as a global budget-constrained optimization problem and derive an optimality condition, which requires the marginal utility of structural resources to be balanced across spatial regions under a finite primitive budget. Based on this formulation, we propose SPARE-GS, a general plug-and-play framework that dynamically aligns the distribution of 3D Gaussian primitives with regional representational demand. SPARE-GS estimates capacity-normalized regional demand, assigns adaptive target quotas, and uses regional budget deviations to coordinate densification, pruning and adaptive termination toward a more balanced structural allocation. Extensive experiments across standard, accelerated, and structure-enhanced 3DGS pipelines demonstrate that SPARE-GS reduces the Gaussian count and training time by an average of 30.38% and 23.81%, respectively, while improving the average PSNR. Moreover, the resulting compact representations reduce downstream processing time and improve the rate-distortion performance of diverse compression and pruning methods, demonstrating the broad applicability of global structural budget regulation.

Zhang Chen, Shuai Wan, Fuzheng Yang +3

School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · School of Telecommunication Engineering, Xidian University, Xi’an 710071, China +2

3D Gaussian Splatting 3D Gaussian Pruning Adaptive Gaussian Densification 3DGS Training

Ask PDF

Mar 3, 2026

Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

We present multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { -- } photometric rendering residuals, semantic priors, and geometric priors { -- } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the multimodal metric indicates recoverable detail; and (2) a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning. By prioritizing regions supported by consistent multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.

Kaiqiang Xiong, Zhanke Wang, Ronggang Wang

Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University, China · Peng Cheng Laboratory, China · Migu Culture Technology Co., Ltd., China

Novel View Synthesis 3D Gaussian Splatting Sparse-View 3D Reconstruction Adaptive Gaussian Densification Importance Sampling

Ask PDF

May 1, 2026

Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting

While 3D Gaussian Splatting (3DGS) has demonstrated impressive real-time rendering performance, its efficacy remains constrained by a reliance on heuristic density control. Despite numerous refinements to these handcrafted rules, such methods inherently lack the flexibility to adapt to diverse scenes with complex geometries. In this paper, we propose a paradigm shift for density control from rigid heuristics to fully learnable policies. Specifically, we introduce LeGS, a framework that reformulates density control as a parameterized policy network optimized via Reinforcement Learning (RL). Central to our approach is the tailored effective reward function grounded in sensitivity analysis, which precisely quantifies the marginal contribution of individual Gaussians to reconstruction quality. To maintain computational tractability, we derive a closed-form solution that reduces the complexity of reward calculation from

O(N^2)

O(N)

. Extensive experiments on the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets demonstrate that LeGS significantly outperforms state-of-the-art methods, striking a superior balance between reconstruction quality and efficiency. The code will be released at https://github.com/AaronNZH/LeGS

Zhenhua Ning, Xin Li, Jun Yu +3

Pengcheng Laboratory, Shenzhen · Harbin Institute of Technology, Shenzhen

3D Gaussian Splatting 3D Reconstruction Adaptive Gaussian Densification

Ask PDF

Abstract

3D Gaussian Splatting

Explore similar work

SPARE-GS: Structural Parsimony and Resource Efficiency for 3D Gaussian Splatting

Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting