NormLift: From Lifted Features To Semantic Reliability In 3D Gaussian Splatting

Yihan Zang; Da Li; Dominik Engel; Shinkyu Park; Ivan Viola

cs.CVSep 16, 2026

NormLift: From Lifted Features To Semantic Reliability In 3D Gaussian Splatting

Authors: Yihan Zang, Da Li, Dominik Engel, Shinkyu Park, Ivan Viola

Organizations: CEMSE Division, King Abdullah University of Science and Technology

Abstract

Training-free weighted aggregation is widely used to lift 2D semantic features onto 3D Gaussians for open-vocabulary scene understanding, yet its theoretical role remains insufficiently understood. Existing analyses typically justify this operation from the rendering side, treating Gaussian features as linearly composable Euclidean variables for reconstructing 2D feature maps. However, this view does not match downstream 3D usage, where each Gaussian is often queried independently in a cosine-based embedding space. We revisit feature lifting from the 3D side and formulate per-Gaussian assignment as a cosine alignment problem on the CLIP unit sphere. Under this objective, the $\ell_2$ -normalized semantic back-projected feature emerges as the closed-form solution, providing a complementary interpretation of the standard lifting rule from the perspective of per-Gaussian semantic assignment. This solution is still a convex combination of the observed directions, so every lifted feature is inherently a mixture. Such mixing is unavoidable for any per-Gaussian estimator, and the question is whether it can be measured. The same formulation answers this: the norm of the back-projected feature decomposes into intra-view and inter-view consistency, so the feature magnitude quantifies how mixed the observations are and serves as a semantic reliability signal. Calibrated by effective multi-view support, this reliability score identifies the Gaussians whose lifted direction has drifted and guides a mode-voting refinement that corrects them by copying a single observed CLIP direction instead of averaging. Experiments on open-vocabulary 3D semantic segmentation show that NormLift is an efficient, training-free framework that achieves strong performance across evaluation protocols. Our code is available at https://github.com/kaust-violab/NormLift_release

Figures & tables

Figure 1: Method overview of NormLift. Top : a) Per-Gaussian features are lifted via cosine alignment on the CLIP sphere, giving the closed-form $u_{j}^{\star}=f_{j}/\|f_{j}\|$ . b) The norm $\|f_{j}\|$ factors into intra- and inter-view consistency, calibrated to the reliability score $R(j)$ by effective view support. c) KNN mode voting copies the most-supported neighbor’s feature, avoiding the blur of linear averaging. d) NormLift supports 2D and 3D downstream tasks. Bottom : a visualization of the 3D segmentation pipeline. The framework is training-free and operates entirely at the per-Gaussian level.

Panel of Figure 2 — Figure 2: Semantic drift from linear interpolation of CLIP features. Left: mixing the source features of door and sofa as $f=\alpha f_{\mathrm{door}}+(1-\alpha)f_{\mathrm{sofa}}$ . The predicted class drifts to unrelated categories (e.g. picture , floor ) for $\alpha\in[0.3,0.7]$ , with neither source class dominating in this regime. Right: pairwise drift rate at $\alpha{=}0.5$ across the 19 ScanNet classes used in our protocol; the overall drift rate is $33.6\%$ .

		19 cls.		15 cls.		10 cls.
Method	T-F	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
LangSplat ( Qin et al., 2024 )	✗	3.78	9.11	5.35	13.20	8.40	22.06
OpenGaussian ( Wu et al., 2024b )	✗	24.73	41.54	30.13	48.25	38.29	55.19
LaGa ( Cen et al., 2025 )	✗	32.50	49.10	35.50	53.50	42.60	63.20
THGS ( Dai et al., 2025 )	✓	34.39	50.74	39.61	57.07	46.38	64.74
VALA ( Wang et al., 2026a )	✓	32.11	50.05	35.10	54.77	46.21	65.61

Table 1: Open-vocabulary 3D semantic segmentation on ScanNet under the OpenGaussian ( Wu et al., 2024b ) protocol. T-F denotes training-free methods. Best and second-best results are highlighted.

Figure 3: Qualitative comparison with SFS ( Xiong et al., 2026 ) under the same training-free protocol. NormLift improves text-query retrieval on LERF scenes and produces sharper, more complete 3D segmentation on ScanNet, especially in the zoomed-in regions.

Table 5

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Hyperparameter sensitivity on ScanNet (10-scene mean, 19 classes). Each subplot shows mIoU as a function of one hyperparameter, with the others fixed at their defaults.

Figure 5: Reliability calibration and distribution on ScanNet. Left: the proposed reliability score $R$ aligns more monotonically with downstream accuracy than the feature-norm-based score, showing the benefit of effective multi-view calibration. Right: Gaussian distribution over reliability intervals, with zero-evidence Gaussians ( $R=0$ ) shown in red and nonzero bins shown in blue. The results indicate that $R$ provides an informative per-Gaussian signal for identifying unreliable primitives and guiding refinement.

Group	Ratio	Mean opacity	% opacity $<0.1$	Mean see-count	Acc
All	$100\%$	$0.622$	$23.95\%$	$9.85$	$44.78\%$
$\mathcal{G}_{0}$	$15.94\%$	$0.042$	$86.32\%$	$0.112$	$35.61\%$
$R(j)>0$	$84.06\%$	$0.731$	$12.01\%$	$11.86$	$46.21\%$

Appendix

Table 4: Empirical characterization of the zero-evidence subset

\mathcal{G}_{0}

versus the remaining Gaussians on ScanNet, averaged over 10 scenes. Zero-evidence Gaussians have much lower opacity, see-count, and downstream accuracy, indicating that

R(j)=0

isolates a physically and semantically distinct regime.

	19 classes		15 classes		10 classes
Scene	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
0000	38.13	53.92	36.62	58.42	44.29	71.47
0062	34.81	68.57	34.63	68.63	53.33	82.53
0070	32.23	44.29	41.86	46.46	54.60	71.65
0097	39.63	59.29	37.44	64.74	53.29	72.70
0140	27.59	42.16	38.13	46.84	49.45	65.04

Appendix

Table 5: Per-scene results of NormLift on ScanNet (OpenGaussian protocol). The last row is the mean over the ten scenes and matches Table 1 .

	mIoU $\uparrow$			mAcc $\uparrow$			locAcc $\uparrow$
Scene	SFS	Occam’s	Ours	SFS	Occam’s	Ours	SFS	Occam’s	Ours
Figurines	0.569	0.593	0.588	0.635	0.669	0.678	0.886	0.843	0.899
Ramen	0.273	0.369	0.287	0.458	0.632	0.500	0.474	0.731	0.544
Teatime	0.636	0.639	0.650	0.775	0.790	0.796	0.923	0.857	0.906
Waldo	0.531	0.487	0.521	0.695	0.703	0.720	0.777	0.753	0.877
Mean	0.502	0.522	0.511	0.641	0.699	0.674	0.765	0.796	0.807

Appendix

Table 6: Semantic segmentation and localization on LERF scenes.

Figure 6: Qualitative comparison of semantic segmentation and localization across scenes. (a,b) Segmentation results on figurines and teatime . (c) Localization visualization on waldo kitchen . From left to right: input RGB image, ground-truth segmentation, SFS [ Xiong et al., 2026 ] , OccamLGS [ Cheng et al., 2024 ] , and ours. Our method produces comparable segmentation masks and yields more accurate and stable localization responses, as highlighted by the arrows.

Method	Scene	#Gauss.	Lift (s)	Rel. / Quant. (s)	Refine (s)	Total (s)	Mem (GB)
SFS	figurines	951,644	449.24	99.53	280.00	828.77	7.48
	ramen	479,180	131.02	25.78	92.17	248.97	4.71
	teatime	1,929,418	189.45	343.38	188.95	721.78	15.16
	waldo_kitchen	1,537,735	200.25	225.47	180.62	606.34	12.09
	Mean	1,224,494	242.49	173.54	185.44	601.46	9.86
NormLift	figurines	951,644	449.24	3.21	31.59	484.04	7.48

Appendix

Table 7: Wall-clock time and peak GPU memory on LERF-OVS, on a single NVIDIA RTX 4090. NormLift and SFS share the same lifting algorithm and thus the same lifting time per scene. The two methods differ in the post-lifting stages: SFS applies product quantization followed by refinement, whereas NormLift computes a reliability score and applies KNN mode-voting refinement.

Scheme	mIoU	mAcc
Spherical geometric median (adapted from VALA)	32.96	50.53
+ $R$ -guided refinement	35.34	53.18
NormLift	35.82	54.15

Appendix

Table 8:

R

-guided refinement on top of another aggregator (ScanNet, 19 classes, 10 scenes).

Training steps	mIoU	mAcc
7k	19.25	34.29
15k	19.86	34.80
30k	19.26	34.59

Appendix

Table 9: NormLift under three quality tiers of the 3DGS reconstruction (ScanNet++, two scenes).

Method	ScanNet++	HoliCity
SFS	19.14 / 25.53	37.92 / 43.92
NormLift	21.93 / 30.74	40.47 / 46.30

Appendix

Table 10: Generalization with unchanged hyperparameters (mIoU / mAcc).

	before: correct	before: wrong
after: correct	349.2k	50.9k
after: wrong	30.6k	501.9k

Appendix

Table 11: Before/after transition of voting edits (ScanNet, 19 classes, 10 scenes, labeled Gaussians).

Same-class neighbors	Share of Gaussians	Acc change
0–25%	1.5%	$+1.31$
25–50%	10.0%	$+2.79$
50–75%	23.5%	$+3.00$
$\geq 75\%$	65.0%	$+1.81$

Appendix

Table 12: Accuracy change of the refinement, stratified by the fraction of same-class neighbors (ScanNet, 19 classes).

Open-Vocabulary 3D Segmentation

Open-vocabulary 3D segmentation assigns language-queryable semantic and instance masks to 3D scenes, including objects unseen during training.

See all 74 papers

Apr 24, 2026

NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting

We propose a neural regularization method that refines the noisy 3D semantic field produced by lifting multi-view inconsistent 2D features, in order to obtain an accurate and robust 3D semantic Gaussian Splatting. The 2D features extracted from vision foundation models suffer from multi-view inconsistency due to a lack of cross-view constraints. Lifting these inconsistent features directly into 3D Gaussians results in a noisy semantic field, which degrades the performance of downstream tasks. Previous methods either focus on obtaining consistent multi-view features in the preprocessing stage or aim to mitigate noise through improved optimization strategies, often at the cost of increased preprocessing time or expensive computational overhead. In contrast, we introduce a variance-aware conditional MLP that operates directly on the 3D Gaussians, leveraging their geometric and appearance attributes to correct semantic errors in 3D space. Experiments on different datasets show that our method enhances the accuracy of lifted semantics, providing an efficient and effective approach to robust 3D semantic Gaussian Splatting.

Zaiyan Yang, Xinpeng Liu, Heng Guo +3

Beijing University of Posts and Telecommunications, China · The University of Osaka, Japan · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, China +1

3D Gaussian Splatting 3D Semantic Segmentation

Ask PDF

Oct 22, 2026

Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

Lifting vision-language features such as CLIP into 3D Gaussian reconstructions enables open-vocabulary segmentation through natural-language queries. Although CLIP represents a broad range of semantic concepts, the lifted features of individual scenes can exhibit strong directional concentration, while raw cosine similarity does not account for the scene-specific feature distribution. We show that substantial segmentation gains can be obtained by changing how a fixed lifted feature field is compared with text. Our method partially subtracts a shared scene direction from visual features and text prototypes and whitens text prototypes using a fixed reference vocabulary of generic terms. It then uses soft clustering for multiclass prediction and CSLS with a relevance margin against the reference vocabulary for single-query prediction. Beyond reducing directional concentration, scene centering improves class-relative feature separation in every evaluated scene. Across seven lifting methods and four datasets, the resulting query-time calibration consistently improves mean intersection over union in both multiclass and single-query segmentation. These gains require no retraining, source-image re-encoding, or feature re-lifting, demonstrating the value of scene-conditioned comparison for making better use of existing lifted representations.

Dominik Engel, Yousef Rajeh, Leon Sick +1

King Abdullah University of Science and Technology (KAUST) · Ulm University

Open-Vocabulary 3D Segmentation VLM Calibration 3D VLMs Vision-Language Alignment

Ask PDF

Oct 6, 2026

Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error

The same Gaussian of a 3D Gaussian Splatting model is seen from many views, and these views do not always agree on the class it belongs to. The Gaussian may be occluded in some of them, and the confidence of the detector is not the same from one view to another. The ground truth, on the other hand, is given as an annotated mesh, because two training runs do not produce the same Gaussians. In this work, we propose a post-training lifting method that works with one target class at a time and combines the information coming from all the views. Target and non-target evidence are accumulated simultaneously, weighted by the visibility of each Gaussian in each view. After that, the Gaussians are filtered with two thresholds: a main threshold

β

selects the high-confidence seeds, and a lower one

γβ

adds the connected components around them. For the evaluation, the labels are transferred from the Gaussians to the mesh vertices that are both visible and annotated. With this design, we can separate three sources of error: the 2D detector, the lifting and the transfer between representations. The thresholds and the transfer operator are chosen on seven Replica validation scenes, and the method is evaluated on ten held-out ScanNet++ scenes with the same values for every scene and class. The mean mIoU on the validation scenes was 0.93 with masks from the dataset annotations and 0.65 with YOLO masks, and on the ScanNet++ test scenes it was 0.80 and 0.54. Compared with thresholding the evidence per view, as a previous version of the method did, the fraction improves the test mIoU by 0.24 and makes it possible to use a single threshold for all the classes and scenes of both datasets. Finally, the error analysis shows that most of the remaining error comes from the detector.

Iván Verdugo Guerra, Ezequiel López Rubio, Jorge García González

Department of Computer Languages and Computer Science University of Málaga

3D Gaussian Splatting 3D Semantic Segmentation

Ask PDF

Abstract

Open-Vocabulary 3D Segmentation

Explore similar work

NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting

Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error