NormLift: From Lifted Features To Semantic Reliability In 3D Gaussian Splatting

Yihan Zang; Da Li; Dominik Engel; Shinkyu Park; Ivan Viola

cs.CVSep 16, 2026

NormLift: From Lifted Features To Semantic Reliability In 3D Gaussian Splatting

Authors: Yihan Zang, Da Li, Dominik Engel, Shinkyu Park, Ivan Viola

Organizations: CEMSE Division, King Abdullah University of Science and Technology

Abstract

Training-free weighted aggregation is widely used to lift 2D semantic features onto 3D Gaussians for open-vocabulary scene understanding, yet its theoretical role remains insufficiently understood. Existing analyses typically justify this operation from the rendering side, treating Gaussian features as linearly composable Euclidean variables for reconstructing 2D feature maps. However, this view does not match downstream 3D usage, where each Gaussian is often queried independently in a cosine-based embedding space. We revisit feature lifting from the 3D side and formulate per-Gaussian assignment as a cosine alignment problem on the CLIP unit sphere. Under this objective, the $\ell_2$ -normalized semantic back-projected feature emerges as the closed-form solution, providing a complementary interpretation of the standard lifting rule from the perspective of per-Gaussian semantic assignment. This solution is still a convex combination of the observed directions, so every lifted feature is inherently a mixture. Such mixing is unavoidable for any per-Gaussian estimator, and the question is whether it can be measured. The same formulation answers this: the norm of the back-projected feature decomposes into intra-view and inter-view consistency, so the feature magnitude quantifies how mixed the observations are and serves as a semantic reliability signal. Calibrated by effective multi-view support, this reliability score identifies the Gaussians whose lifted direction has drifted and guides a mode-voting refinement that corrects them by copying a single observed CLIP direction instead of averaging. Experiments on open-vocabulary 3D semantic segmentation show that NormLift is an efficient, training-free framework that achieves strong performance across evaluation protocols. Our code is available at https://github.com/kaust-violab/NormLift_release

Figures & tables

Figure 1: Method overview of NormLift. Top : a) Per-Gaussian features are lifted via cosine alignment on the CLIP sphere, giving the closed-form $u_{j}^{\star}=f_{j}/\|f_{j}\|$ . b) The norm $\|f_{j}\|$ factors into intra- and inter-view consistency, calibrated to the reliability score $R(j)$ by effective view support. c) KNN mode voting copies the most-supported neighbor’s feature, avoiding the blur of linear averaging. d) NormLift supports 2D and 3D downstream tasks. Bottom : a visualization of the 3D segmentation pipeline. The framework is training-free and operates entirely at the per-Gaussian level.

Panel of Figure 2 — Figure 2: Semantic drift from linear interpolation of CLIP features. Left: mixing the source features of door and sofa as $f=\alpha f_{\mathrm{door}}+(1-\alpha)f_{\mathrm{sofa}}$ . The predicted class drifts to unrelated categories (e.g. picture , floor ) for $\alpha\in[0.3,0.7]$ , with neither source class dominating in this regime. Right: pairwise drift rate at $\alpha{=}0.5$ across the 19 ScanNet classes used in our protocol; the overall drift rate is $33.6\%$ .

		19 cls.		15 cls.		10 cls.
Method	T-F	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
LangSplat ( Qin et al., 2024 )	✗	3.78	9.11	5.35	13.20	8.40	22.06
OpenGaussian ( Wu et al., 2024b )	✗	24.73	41.54	30.13	48.25	38.29	55.19
LaGa ( Cen et al., 2025 )	✗	32.50	49.10	35.50	53.50	42.60	63.20
THGS ( Dai et al., 2025 )	✓	34.39	50.74	39.61	57.07	46.38	64.74
VALA ( Wang et al., 2026a )	✓	32.11	50.05	35.10	54.77	46.21	65.61

Table 1: Open-vocabulary 3D semantic segmentation on ScanNet under the OpenGaussian ( Wu et al., 2024b ) protocol. T-F denotes training-free methods. Best and second-best results are highlighted.

Figure 3: Qualitative comparison with SFS ( Xiong et al., 2026 ) under the same training-free protocol. NormLift improves text-query retrieval on LERF scenes and produces sharper, more complete 3D segmentation on ScanNet, especially in the zoomed-in regions.

Table 5

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Hyperparameter sensitivity on ScanNet (10-scene mean, 19 classes). Each subplot shows mIoU as a function of one hyperparameter, with the others fixed at their defaults.

Figure 5: Reliability calibration and distribution on ScanNet. Left: the proposed reliability score $R$ aligns more monotonically with downstream accuracy than the feature-norm-based score, showing the benefit of effective multi-view calibration. Right: Gaussian distribution over reliability intervals, with zero-evidence Gaussians ( $R=0$ ) shown in red and nonzero bins shown in blue. The results indicate that $R$ provides an informative per-Gaussian signal for identifying unreliable primitives and guiding refinement.

Group	Ratio	Mean opacity	% opacity $<0.1$	Mean see-count	Acc
All	$100\%$	$0.622$	$23.95\%$	$9.85$	$44.78\%$
$\mathcal{G}_{0}$	$15.94\%$	$0.042$	$86.32\%$	$0.112$	$35.61\%$
$R(j)>0$	$84.06\%$	$0.731$	$12.01\%$	$11.86$	$46.21\%$

Appendix

Table 4: Empirical characterization of the zero-evidence subset

\mathcal{G}_{0}

versus the remaining Gaussians on ScanNet, averaged over 10 scenes. Zero-evidence Gaussians have much lower opacity, see-count, and downstream accuracy, indicating that

R(j)=0

isolates a physically and semantically distinct regime.

	19 classes		15 classes		10 classes
Scene	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
0000	38.13	53.92	36.62	58.42	44.29	71.47
0062	34.81	68.57	34.63	68.63	53.33	82.53
0070	32.23	44.29	41.86	46.46	54.60	71.65
0097	39.63	59.29	37.44	64.74	53.29	72.70
0140	27.59	42.16	38.13	46.84	49.45	65.04

Appendix

Table 5: Per-scene results of NormLift on ScanNet (OpenGaussian protocol). The last row is the mean over the ten scenes and matches Table 1 .

	mIoU $\uparrow$			mAcc $\uparrow$			locAcc $\uparrow$
Scene	SFS	Occam’s	Ours	SFS	Occam’s	Ours	SFS	Occam’s	Ours
Figurines	0.569	0.593	0.588	0.635	0.669	0.678	0.886	0.843	0.899
Ramen	0.273	0.369	0.287	0.458	0.632	0.500	0.474	0.731	0.544
Teatime	0.636	0.639	0.650	0.775	0.790	0.796	0.923	0.857	0.906
Waldo	0.531	0.487	0.521	0.695	0.703	0.720	0.777	0.753	0.877
Mean	0.502	0.522	0.511	0.641	0.699	0.674	0.765	0.796	0.807

Appendix

Table 6: Semantic segmentation and localization on LERF scenes.

Figure 6: Qualitative comparison of semantic segmentation and localization across scenes. (a,b) Segmentation results on figurines and teatime . (c) Localization visualization on waldo kitchen . From left to right: input RGB image, ground-truth segmentation, SFS [ Xiong et al., 2026 ] , OccamLGS [ Cheng et al., 2024 ] , and ours. Our method produces comparable segmentation masks and yields more accurate and stable localization responses, as highlighted by the arrows.

Method	Scene	#Gauss.	Lift (s)	Rel. / Quant. (s)	Refine (s)	Total (s)	Mem (GB)
SFS	figurines	951,644	449.24	99.53	280.00	828.77	7.48
	ramen	479,180	131.02	25.78	92.17	248.97	4.71
	teatime	1,929,418	189.45	343.38	188.95	721.78	15.16
	waldo_kitchen	1,537,735	200.25	225.47	180.62	606.34	12.09
	Mean	1,224,494	242.49	173.54	185.44	601.46	9.86
NormLift	figurines	951,644	449.24	3.21	31.59	484.04	7.48

Appendix

Table 7: Wall-clock time and peak GPU memory on LERF-OVS, on a single NVIDIA RTX 4090. NormLift and SFS share the same lifting algorithm and thus the same lifting time per scene. The two methods differ in the post-lifting stages: SFS applies product quantization followed by refinement, whereas NormLift computes a reliability score and applies KNN mode-voting refinement.

Scheme	mIoU	mAcc
Spherical geometric median (adapted from VALA)	32.96	50.53
+ $R$ -guided refinement	35.34	53.18
NormLift	35.82	54.15

Appendix

Table 8:

R

-guided refinement on top of another aggregator (ScanNet, 19 classes, 10 scenes).

Training steps	mIoU	mAcc
7k	19.25	34.29
15k	19.86	34.80
30k	19.26	34.59

Appendix

Table 9: NormLift under three quality tiers of the 3DGS reconstruction (ScanNet++, two scenes).

Method	ScanNet++	HoliCity
SFS	19.14 / 25.53	37.92 / 43.92
NormLift	21.93 / 30.74	40.47 / 46.30

Appendix

Table 10: Generalization with unchanged hyperparameters (mIoU / mAcc).

	before: correct	before: wrong
after: correct	349.2k	50.9k
after: wrong	30.6k	501.9k

Appendix

Table 11: Before/after transition of voting edits (ScanNet, 19 classes, 10 scenes, labeled Gaussians).

Same-class neighbors	Share of Gaussians	Acc change
0–25%	1.5%	$+1.31$
25–50%	10.0%	$+2.79$
50–75%	23.5%	$+3.00$
$\geq 75\%$	65.0%	$+1.81$

Appendix

Table 12: Accuracy change of the refinement, stratified by the fraction of same-class neighbors (ScanNet, 19 classes).

Open-Vocabulary 3D Segmentation

Open-vocabulary 3D segmentation assigns language-queryable semantic and instance masks to 3D scenes, including objects unseen during training.

See all 74 papers

NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting
Apr 24, 2026Zaiyan Yang, Xinpeng Liu, Heng Guo +33D Gaussian Splatting 3D Semantic Segmentation
Beijing University of Posts and Telecommunications, China · The University of Osaka, Japan · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, China +1
Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation
Oct 22, 2026Dominik Engel, Yousef Rajeh, Leon Sick +1Open-Vocabulary 3D Segmentation VLM Calibration
King Abdullah University of Science and Technology (KAUST) · Ulm University
Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error
Oct 6, 2026Iván Verdugo Guerra, Ezequiel López Rubio, Jorge García González3D Gaussian Splatting 3D Semantic Segmentation
Department of Computer Languages and Computer Science University of Málaga

Apr 24, 2026J/K move · Enter open · S save

NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting

Zaiyan Yang, Xinpeng Liu, Heng Guo, Jinglei Shi+2

Beijing University of Posts and Telecommunications, China · The University of Osaka, Japan · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, China+1

3D Gaussian Splatting 3D Semantic Segmentation

Open paper AskPDF arXiv

We propose a neural regularization method that refines the noisy 3D semantic field produced by lifting multi-view inconsistent 2D features, in order to obtain an accurate and robust 3D semantic Gaussian Splatting. The 2D features extracted from vision foundation models suffer from multi-view inconsistency due to a lack of cross-view constraints. Lifting these inconsistent features directly into 3D Gaussians results in a noisy semantic field, which degrades the performance of downstream tasks. Previous methods either focus on obtaining consistent multi-view features in the preprocessing stage or aim to mitigate noise through improved optimization strategies, often at the cost of increased preprocessing time or expensive computational overhead. In contrast, we introduce a variance-aware conditional MLP that operates directly on the 3D Gaussians, leveraging their geometric and appearance attributes to correct semantic errors in 3D space. Experiments on different datasets show that our method enhances the accuracy of lifted semantics, providing an efficient and effective approach to robust 3D semantic Gaussian Splatting.

Abstract

Open-Vocabulary 3D Segmentation

Explore similar work

NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting

Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error