Learning Hierarchical Orthogonal Prototypes for Generalized Few-Shot 3D Point Cloud Segmentation

Yifei Zhao; Fanyu Zhao; Zhongyuan Zhang; Shengtang Wu; Yixuan Lin; Yinsheng Li

cs.CVMar 20, 2026

Learning Hierarchical Orthogonal Prototypes for Generalized Few-Shot 3D Point Cloud Segmentation

Authors: Yifei Zhao, Fanyu Zhao, Zhongyuan Zhang, Shengtang Wu, Yixuan Lin, Yinsheng Li

Organizations: College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China

Abstract

Generalized few-shot 3D point cloud segmentation aims to adapt to novel classes from only a few annotations while maintaining strong performance on base classes, but this remains challenging due to the inherent stability-plasticity trade-off: adapting to novel classes can interfere with shared representations and cause base-class forgetting. We present HOP3D, a unified framework that learns hierarchical orthogonal prototypes with an entropy-based few-shot regularizer to enable robust novel-class adaptation without degrading base-class performance. HOP3D introduces hierarchical orthogonalization that decouples base and novel learning at both the gradient and representation levels, effectively mitigating base-novel interference. To further enhance adaptation under sparse supervision, we incorporate an entropy-based regularizer that leverages predictive uncertainty to refine prototype learning and promote balanced predictions. Extensive experiments on ScanNet200 and ScanNet++ demonstrate that HOP3D consistently outperforms state-of-the-art baselines under both 1-shot and 5-shot settings. The code is available at https://fdueblab-hop3d.github.io/.

Figures & tables

Fig. 1: Overview of the proposed HOP3D framework, integrating HOP-Net and HOP-Ent. The training pipeline consists of two phases: Phase 1 trains base classes with HOP-Rep and collects base-task gradients $\mathcal{G}$ to construct an orthonormal basis $B$ for HOP-Grad ; Phase 2 introduces novel classes for few-shot adaptation, applies HOP-Grad to project each novel gradient $g$ as $\tilde{g}$ (removing $B$ ), continues to use HOP-Rep for prototype subspace orthogonalization, and integrates HOP-Ent for entropy-guided refinement. (a) Gradient orthogonalization in HOP-Grad using $B$ and $\tilde{g}$ ; (b) Prototype similarity heatmaps and orthogonal prototype subspaces before/after HOP-Rep ; (c) Confidence distribution and class-frequency distribution before/after HOP-Ent .

Method	ScanNet200								ScanNet++
	5-shot				1-shot				5-shot				1-shot
	B	N	A	HM	B	N	A	HM	B	N	A	HM	B	N	A	HM
Fully Sup.	68.70	39.32	45.51	50.02	68.70	39.32	45.51	50.02	65.45	37.24	48.53	47.47	65.45	37.24	48.53	47.47
PIFS [ 5 ]	28.78	3.82	9.07	6.71	17.84	2.87	6.02	4.88	39.98	5.74	19.44	10.03	36.66	4.95	17.63	8.71
attMPTI [ 31 ]	37.13	4.99	11.76	8.79	54.84	3.28	14.14	6.17	55.89	4.19	24.87	7.78	53.16	3.55	23.40	6.66
COSeg [ 1 ]	57.67	5.21	16.25	9.54	47.03	4.03	13.09	7.42	59.34	6.96	27.91	12.45	58.49	6.24	27.14	11.26

TABLE I: Performance comparison on ScanNet200 and ScanNet++. Results under 1-shot and 5-shot settings.

Fig. 2: Qualitative comparison between GFS-VL and our HOP3D on ScanNet200. Class color legend is shown at the top. From left to right: query input, ground-truth labels, GFS-VL prediction, and HOP3D prediction.

HOP-Net	HOP-Ent	mIoU-B	mIoU-N	mIoU-A	HM
		67.64	28.80	36.98	40.39
✓ †		68.67	29.46	37.71	41.23
✓ ‡		68.79	29.19	37.53	40.98
✓		69.30	30.52	38.69	42.37
✓	✓ §	67.11	30.79	38.43	42.20
✓	✓	68.45	31.80	39.52	43.42

TABLE II: Ablation on ScanNet200 (1-shot) for HOP-Net and HOP-Ent.

Fig. 3: Qualitative results illustrating the effect of HOP-Net and HOP-Ent. From left to right: query, ground truth, prediction without HOP-Net/HOP-Ent, and HOP3D. Circles highlight corrected base–novel misclassifications.

Fig. 4: HOP-Net ablation. (a)–(d) Impact of

\lambda_{\mathrm{orth}}

. (e) Effect of AR. Shaded areas denote 95% confidence intervals.

Fig. 5: Cosine-similarity matrices of $\ell_{2}$ -normalized prototypes (red: higher similarity, blue: lower). Phase 1: base prototypes only; Phase 2: joint base+novel prototypes (base first, then novel). (a) P1 w/o HOP-Net; (b) P1 w/ HOP-Net; (c) P2 w/o HOP-Net; (d) P2 w/ HOP-Net.

Fig. 6: HOP-Ent analysis. (a) Confidence distribution, where higher mean confidence indicates better prediction certainty. (b) Class frequency distribution, where a lower coefficient of variation indicates better class balance. HOP-Ent improves both prediction certainty and class balance.

3D Point Cloud Segmentation

3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.

See all 68 papers

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
Mar 6, 2026Vishal Thengane, Zhaochong An, Tianjin Huang +5Few-Shot Semantic Segmentation 3D Semantic Segmentation
University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark +2
Uncertainty Gates the Gradient: Adaptive Updates for Generalized Few-Shot 3D Segmentation
Oct 17, 2026Yifei Zhao, Fanyu Zhao, Shengtang Wu +33D Point Cloud Segmentation 3D Semantic Segmentation
Fudan University · University of Science and Technology of China
Uncertainty-aware Prototype Learning with Variational Inference for Few-shot Point Cloud Segmentation
Mar 20, 2026Yifei Zhao, Fanyu Zhao, Yinsheng LiFew-Shot Semantic Segmentation 3D Point Cloud Segmentation
Fudan University, Shanghai, China

Mar 6, 2026J/K move · Enter open · S save

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

Vishal Thengane, Zhaochong An, Tianjin Huang, Son Lam Phung+4

University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark+2

Few-Shot Semantic Segmentation 3D Semantic Segmentation Class-Incremental Learning Prototype Learning

Open paper AskPDF arXiv

Incremental Few-Shot (IFS) segmentation aims to learn new categories over time from only a few annotations. Although widely studied in 2D, it remains underexplored for 3D point clouds. Existing methods suffer from catastrophic forgetting or fail to learn discriminative prototypes under sparse supervision, and often overlook a key cue: novel categories frequently appear as unlabelled background in base-training scenes. We introduce SCOPE (Scene-COntextualised Prototype Enrichment), a plug-and-play background-guided prototype enrichment framework that integrates with any prototype-based 3D segmentation method. After base training, a class-agnostic segmentation model extracts high-confidence pseudo-instances from background regions to build a prototype pool. When novel classes arrive with few labelled samples, relevant background prototypes are retrieved and fused with few-shot prototypes to form enriched representations without retraining the backbone or adding parameters. Experiments on ScanNet and S3DIS show that SCOPE achieves SOTA performance, improving novel-class IoU by up to 6.98% and 3.61%, and mean IoU by 2.25% and 1.70%, respectively, while maintaining low forgetting. Code is available https://github.com/Surrey-UP-Lab/SCOPE.

Abstract

3D Point Cloud Segmentation

Explore similar work

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

Uncertainty Gates the Gradient: Adaptive Updates for Generalized Few-Shot 3D Segmentation

Uncertainty-aware Prototype Learning with Variational Inference for Few-shot Point Cloud Segmentation