Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

Jiyang Huang; Hongru Chen; Wei Lin; Jia Wan; Antoni B. Chan

cs.CVMar 17, 2026

Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

Authors: Jiyang Huang, Hongru Chen, Wei Lin, Jia Wan, Antoni B. Chan

Organizations: Harbin Institute of Technology, Shenzhen, China · City University of Hong Kong, Kowloon Tong, Hong Kong

Abstract

Semi-supervised crowd analysis is a prominent area of research, as unlabeled data are typically abundant and inexpensive to obtain. However, traditional point-based annotations constrain performance because individual regions are inherently ambiguous, and consequently, learning fine-grained structural semantics from sparse anno tations remains an unresolved challenge. In this paper, we first propose an Exclusion-Constrained Dual-Prompt SAM (EDP-SAM), based on our Nearest Neighbor Exclusion Circle (NNEC) constraint, to generate mask supervision for current datasets. With the aim of segmenting individuals in dense scenes, we then propose Exclusivity-Guided Mask Learning (XMask), which enforces spatial separation through a discriminative mask objective. Gaussian smoothing and a differentiable center sampling strategy are utilized to improve feature continuity and training stability. Building on XMask, we present a semi-supervised crowd counting framework that uses instance mask priors as pseudo-labels, which contain richer shape information than traditional point cues. Extensive experiments on the ShanghaiTech A, UCF-QNRF, and JHU++ datasets (using 5%, 10%, and 40% labeled data) verify that our end-to-end model achieves state-of-the-art semi-supervised segmentation and counting performance, effectively bridging the gap between counting and instance segmentation within a unified framework.

Figures & tables

Figure 1: Conception of our approach. EDP-SAM is designed to generate data with mask supervision. To acquire segmentation capabilities, we propose a semi-supervised XMask method to learn a discriminative feature representation. Subsequently, we present a mask-constraint strategy composing the final step in our framework, coupling semantic and spatial information to enhance counting accuracy.

Figure 2: Overview of the proposed framework. Our semi-supervised pipeline consists of three stages: (1) pretraining and mask preparation, (2) crowd segmentation training, and (3) mask-constrained crowd counting optimization. In Stage 1, mask annotations are generated from point labels via the proposed EDP-SAM (yellow region). Stage 2 presents the semi-supervised XMask training process, including discriminative supervision and pseudo-mask generation. The trained segmentation model is then utilized in Stage 3 to produce pseudo-masks for unlabeled data, where newly designed loss functions are introduced to further enhance counting performance via mask constraint.

Figure 3: Point-Superpixel Dual-Prompt SAM with NNEC Constraint Method

Figure 4: MaskHead Discriminative Loss. First, use points to obtain the corresponding ROIs for computation. Based on the pseudo/ground truth mask, divide the ROIs into positive and negative regions. After applying Gaussian smoothing, extract the center feature and compute pixel-level L2 distance within feature map. Finally, applying pull and push force losses separately.

Figure 5: Pseudo-Mask Selection. We utilize a low threshold to obtain pseudo points, thereby generating all possible masks and ensuring the reliability of background. Then, based on the probability, we select the mask associated with the highest probability as the final valid mask.

Figure 6: Visualization of masks generated by FastSAM-based, XMask, and ground truth generated by EDP-SAM. Through feature learning, our method demonstrates strong robustness in human segmentation, potentially segmenting other irrelevant objects compared to FastSAM. Furthermore, incorporating NNEC enhances the segmentation’s dimensional adaptability.

Figure 7: Visualization of Counting results on ShTech A. The first row: GT location. The second row: predicted points. The third row: feature map from maskhead in our method. In both dense and normal crowd scenes, our method yields counting predictions that closely align with the ground-truth counts, while the learned feature maps maintain structural coherence with the semantic layout of crowd head regions.

Methods	NNEC	ShTech A [ 65 ]			UCF-QNRF [ 18 ]			JHU++ [ 47 ]
Methods	NNEC	5%	10%	40%	5%	10%	40%	5%	10%	40%
FastSAM [ 66 ]		0.279	0.281	0.291	0.203	0.211	0.213	0.186	0.185	0.189
FastSAM [ 66 ]	✓	0.287	0.290	0.300	0.223	0.229	0.233	0.197	0.182	0.201
S4M [ 61 ]		0.005	0.007	0.011	0.010	0.012	0.015	0.013	0.014	0.021
CrowdSAM [ 7 ]	✓	0.201	0.201	0.212	0.149	0.156	0.163	0.187	0.189	0.192
point only	✓	0.195	0.194	0.202	0.198	0.201	0.207	0.180	0.182	0.183

Table 1: Semi-supervised crowd instance segmentation comparison on [email protected].

labeled Pct.	Methods	ShTech A [ 65 ]		UCF-QNRF [ 18 ]		JHU++ [ 47 ]
labeled Pct.	Methods	MAE↓	MSE↓	MAE↓	MSE↓	MAE↓	MSE↓
5%	MT [ 50 ]	104.7	33.2	172.4	284.9	101.5	363.5
	L2R [ 34 ]	103.0	27.6	160.1	272.3	101.4	338.8
	DAC [ 27 ]	85.4	134.5	120.2	209.3	82.2	294.9
	OT-M [ 29 ]	83.7	133.3	118.4	195.4	82.7	304.5
	P2R [ 30 ]	69.9	119.5	100.1	182.5	77.8	293.5

Table 2: Semi-supervised crowd counting performance comparison between recent methods and our framework on three datasets under various labeled protocols.

Figure 8: Our method consistently outperforms SAM and FastSAM in segmentation efficiency across varying crowd densities, maintaining shorter processing times despite differences in density.

Table 3: Comprehensive ablation studies on different components and settings.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1 : Experiment hyperparameters of Pseudo Mask generation in the training of XMask on ShTech A. We evaluated IoU@0.5 and F1 of XMask with different settings of $\lambda$ and $\tau_{g}$ , and choose 1.0 and 0.8 respectively in the final XMask training. — Figure S1 : Experiment hyperparameters of Pseudo Mask generation in the training of XMask on ShTech A. We evaluated [email protected] and F1 of XMask with different settings of $\lambda$ and $\tau_{g}$ , and choose 1.0 and 0.8 respectively in the final XMask training.

Labeled-Pct.	Average L2 Feature Distance
Labeled-Pct.	within same instance	between different instances
40%	0.630	0.560
10%	0.600	0.570
5%	0.610	0.550

Appendix

Table S1 : Loss threshold test on ShTech A. Before XMask training, we tested the initial L2 feature distance on the untrained feature map with ground truth masks.

Panel of Figure 14 — Figure S2 : Experiments of Hyperparameters used in Segmentation Task on ShTech A

Figure S3 : Experiments on number of segment comparison to generate superpixel by SLIC. We implement n = 1000 in our approach, because smaller number leads to inaccurate mask without semantic, and larger number results in broken masks.

Figure S4 : Prompt and Constraint Ablation Study in EDP-SAM. The left table illustrates that our dual-prompt input results in both complete and accurate (in count) masks, and the right table shows the effectiveness of regional constraint and our NNEC circle lead to better masks with complete crowd head.

Figure S5 : Examples of Artificial Correction in EDP-SAM.

Figure S6 : Visualization of Semi-supervised Counting Comparison with GT

Crowd Counting

Crowd counting estimates how many people appear in images or video, supporting occupancy monitoring, safety planning, and crowd management.

See all 23 papers

Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation
Apr 2, 2026Hongru Chen, Jiyang Huang, Jia Wan +1Instance Segmentation Crowd Counting
Harbin Institute of Technology, Shenzhen · City University of Hong Kong
A Benchmark for Semi-supervised Multi-modal Crowd Counting
Jun 2, 2026Haoliang Meng, Xiaopeng Hong, Yabin Wang +1Crowd Counting Semi-Supervised Learning
Harbin Institute of Technology · Pengcheng Laboratory
Hybrid supervised instance segmentation by learning label noise suppression
Oct 14, 2026Linwei Chen, Ying Fu, Shaodi You +1Instance Segmentation Pseudo-Labeling
School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China · Computer Vision Research Group, Insititute of Informatics, University of Amsterdam, Amsterdam 1098 XH, Netherlands · Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing 100101, China

Apr 2, 2026J/K move · Enter open · S save

Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation

Hongru Chen, Jiyang Huang, Jia Wan, Antoni B. Chan

Harbin Institute of Technology, Shenzhen · City University of Hong Kong

Instance Segmentation Crowd Counting

Open paper AskPDF arXiv

Crowd instance segmentation is a crucial task with a wide range of applications, including surveillance and transportation. Currently, point labels are common in crowd datasets, while region labels (e.g., boxes) are rare and inaccurate. The masks obtained through segmentation help to improve the accuracy of region labels and resolve the correspondence between individual location coordinates and crowd density maps. However, directly applying currently popular large foundation models such as SAM does not yield ideal results in dense crowds. To this end, we first propose Dense Point-to-Mask Optimization (DPMO), which integrates SAM with the Nearest Neighbor Exclusive Circle (NNEC) constraint to generate dense instance segmentation from point annotations. With DPMO and manual correction, we obtain mask annotations from the existing point annotations for traditional crowd datasets. Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework trained with Group Relative Policy Optimization (GRPO), which selects the best predicted point from a sampling of the initial point prediction. Through extensive experiments, we achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets. Furthermore, we design new loss functions supervised by masks that boost counting performance across different models, demonstrating the significant role of mask annotations in enhancing counting accuracy.

Abstract

Crowd Counting

Explore similar work

Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation

A Benchmark for Semi-supervised Multi-modal Crowd Counting

Hybrid supervised instance segmentation by learning label noise suppression