Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation

Hongru Chen; Jiyang Huang; Jia Wan; Antoni B. Chan

cs.CVApr 2, 2026

Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation

Authors: Hongru Chen, Jiyang Huang, Jia Wan, Antoni B. Chan

Organizations: Harbin Institute of Technology, Shenzhen · City University of Hong Kong

Abstract

Crowd instance segmentation is a crucial task with a wide range of applications, including surveillance and transportation. Currently, point labels are common in crowd datasets, while region labels (e.g., boxes) are rare and inaccurate. The masks obtained through segmentation help to improve the accuracy of region labels and resolve the correspondence between individual location coordinates and crowd density maps. However, directly applying currently popular large foundation models such as SAM does not yield ideal results in dense crowds. To this end, we first propose Dense Point-to-Mask Optimization (DPMO), which integrates SAM with the Nearest Neighbor Exclusive Circle (NNEC) constraint to generate dense instance segmentation from point annotations. With DPMO and manual correction, we obtain mask annotations from the existing point annotations for traditional crowd datasets. Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework trained with Group Relative Policy Optimization (GRPO), which selects the best predicted point from a sampling of the initial point prediction. Through extensive experiments, we achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets. Furthermore, we design new loss functions supervised by masks that boost counting performance across different models, demonstrating the significant role of mask annotations in enhancing counting accuracy.

Figures & tables

Figure 1: Pipeline of our proposed DPMO-RPS framework for crowd instance segmentation, including DPMO segmentation paradigm using NNEC constraint and RPS network trained with modified GRPO. Supervised by manually modified mask annotations, we train the OPET Net to obtain more accurate predicted points and perform sampling. Then, we train the Point Policy Network with modified GRPO to select the best point from each group as a final point prompt to assist SAM in segmentation in DPMO. Finally, we can achieve effective crowd instance segmentation.

dataset	model	IoU $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
SHA ( Zhang et al. 2016 )	SAM ( Kirillov et al. 2023 ) +Classification	0.3987	0.3630	0.3498	0.3563
	FastSAM ( Zhao et al. 2023 )	0.3624	0.3034	0.2748	0.2884
	CrowdSAM ( Cai et al. 2024 )	0.3530	0.2997	0.1422	0.1928
	Ours	0.5431	0.5495	0.5572	0.5533
SHB ( Zhang et al. 2016 )	SAM ( Kirillov et al. 2023 ) +Classification	0.3776	0.3223	0.2606	0.2882
	FastSAM ( Zhao et al. 2023 )	0.4269	0.4220	0.4397	0.4307

Table 1: Segmentation performance comparison between our model and other models on crowd datasets.

model	Loss Function	ShanghaiTech A		UCF-QNRF		NWPU-Crowd
model	Loss Function	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
CSRNet ( Li et al. 2018 )	-	68.2	115.0	-	-	121.3	387.8
BL+ ( Ma et al. 2019 )	-	62.8	101.8	88.7	154.8	105.4	454.2
DMCount ( Wang et al. 2020a )	-	59.7	95.7	85.6	148.3	88.4	388.6
P2PNet ( Song et al. 2021 )	-	52.74	85.06	85.32	154.5	77.44	362.0
GL ( Wan et al. 2021 )	-	61.3	95.4	84.3	147.5	79.3	346.1

Table 2: Results of crowd counting comparison using the proposed mask loss function supervised by mask annotations.

Figure 2: Segmentation results contrast with different models, including the SAM-optimized model, FastSAM, CrowdSAM, and our DPMO-RPS model. The robust mask annotations are in the rightmost column.

datasets	time (hours)	correction ratio (%)	correction consistency (IoU)	quality gap (IoU)
SHA	8.5	22.5	0.9823	0.9588
SHB	12.5	18.0	0.9934	0.9762
UCF	30	25.5	0.9725	0.9623
JHU	75.25	30.5	0.9807	0.9532
NWPU	86.15	35.0	0.9695	0.9486

Table 3: Details of manual mask correction for datasets.

Figure 3: Results contrast for three hyperparameters in DPMO-RPS. (a) Different radii when sampling in RPS, (b) Different radii in the NNEC constraint of DPMO, (c) Different weights of the KL loss.

model	IoU $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
SAM+classification	0.3254	0.2509	0.2437	0.2472
FastSAM	0.3823	0.3125	0.2604	0.2841
CrowdSAM	0.3048	0.2237	0.1098	0.1473
Ours	0.5466	0.5823	0.5328	0.5565

Table 4: Segmentation results on independent human annotated subset generated from crowd counting datasets.

network	annotations	MAE $\downarrow$	RMSE $\downarrow$
ZIP ( Ma et al. 2025 )	points	47.81	75.04
	boxes	48.68	75.84
	masks	46.95	73.83

Table 5: Loss with different annotations on SHA dataset.

	IoU $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
w/o DPMO and RPS	0.0256	0.0133	0.0208	0.0162
w/o DPMO	0.3867	0.3725	0.3778	0.3751
w/o RPS	0.5131	0.5005	0.5081	0.5043
ours	0.5431	0.5495	0.5572	0.5533

Table 6: Segmentation results without different parts of DPMO-RPS framework on ShanghaiTech A dataset.

Method	IoU $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
GL ( Wan et al. 2021 ) + SAM	0.1866	0.2241	0.0308	0.0542
GL ( Wan et al. 2021 ) + ours	0.3187	0.3053	0.0849	0.1328
CLTR ( Liang et al. 2022 ) + SAM	0.2135	0.2069	0.0524	0.0836
CLTR ( Liang et al. 2022 ) + ours	0.3065	0.2867	0.1048	0.1535
PET ( Liu et al. 2023 ) + SAM	0.3024	0.2916	0.2831	0.2873
PET ( Liu et al. 2023 ) + ours	0.4769	0.4426	0.4396	0.4411

Table 7: Segmentation results with different point prediction models on UCF-QNRF dataset.

Instance Segmentation

Instance segmentation identifies each object and assigns it a separate pixel-level mask in images or video.

See all 159 papers

Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting
Mar 17, 2026Jiyang Huang, Hongru Chen, Wei Lin +2Crowd Counting Instance Segmentation
Harbin Institute of Technology, Shenzhen, China · City University of Hong Kong, Kowloon Tong, Hong Kong
SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness
Feb 25, 2026Zhaoyang Wei, Xumeng Han, Xuehui Yu +4Instance Segmentation Weakly Supervised Image Segmentation
the School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Science (UCAS), Beijing, 100049, China. · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University (SJTU), Shanghai, 200240, China. · the School of Computer Science and Technology, University of Chinese Academy of Science (UCAS), Beijing, 100049, China.
DualCount: Structurally Consistent Density and Point Modeling for Zero-Shot Object Counting
Sep 14, 2026Xuan Cuong NgoObject Counting
University of Arkansas, USA

Mar 17, 2026J/K move · Enter open · S save

Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

Jiyang Huang, Hongru Chen, Wei Lin, Jia Wan+1

Harbin Institute of Technology, Shenzhen, China · City University of Hong Kong, Kowloon Tong, Hong Kong

Crowd Counting Instance Segmentation Promptable Image Segmentation

Open paper AskPDF arXiv

Semi-supervised crowd analysis is a prominent area of research, as unlabeled data are typically abundant and inexpensive to obtain. However, traditional point-based annotations constrain performance because individual regions are inherently ambiguous, and consequently, learning fine-grained structural semantics from sparse anno tations remains an unresolved challenge. In this paper, we first propose an Exclusion-Constrained Dual-Prompt SAM (EDP-SAM), based on our Nearest Neighbor Exclusion Circle (NNEC) constraint, to generate mask supervision for current datasets. With the aim of segmenting individuals in dense scenes, we then propose Exclusivity-Guided Mask Learning (XMask), which enforces spatial separation through a discriminative mask objective. Gaussian smoothing and a differentiable center sampling strategy are utilized to improve feature continuity and training stability. Building on XMask, we present a semi-supervised crowd counting framework that uses instance mask priors as pseudo-labels, which contain richer shape information than traditional point cues. Extensive experiments on the ShanghaiTech A, UCF-QNRF, and JHU++ datasets (using 5%, 10%, and 40% labeled data) verify that our end-to-end model achieves state-of-the-art semi-supervised segmentation and counting performance, effectively bridging the gap between counting and instance segmentation within a unified framework.

Abstract

Instance Segmentation

Explore similar work

Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness

DualCount: Structurally Consistent Density and Point Modeling for Zero-Shot Object Counting