Zero-shot object counting (ZOC) aims to count instances of unseen categories using only a class name. Existing exemplar-based methods discover candidate exemplars through random image patches or open-vocabulary detector proposals. However, random patch sampling often yields poorly delineated instances, while detector-based discovery may produce multi-instance regions and, in dense scenes, provide only a few or even no suitable single-instance candidates. Thus, exemplar selection is constrained by candidate quality and coverage, while counting itself remains highly sensitive to exemplar choice. To this end, we propose CountZES, an inference-only framework that jointly addresses candidate discovery and exemplar selection through density-guided candidate generation and complementary exemplar-selection mechanisms. Starting from a text-grounded anchor, CountZES progressively discovers and refines visual exemplars through three stages: Detection-Anchored Exemplar (DAE), Density-Guided Exemplar (DGE), and Feature-Consensus Exemplar (FCE). DAE establishes a reliable single-instance anchor, DGE leverages density responses to expand candidate discovery beyond the detector-constrained pool and select a statistically consistent exemplar, and FCE promotes representativeness through feature-level consensus. Together, they yield complementary exemplars balancing semantic grounding, count consistency, and feature representativeness. CountZES requires no ZOC-specific optimization and adopts a modular design that supports different pretrained components. Extensive evaluations across natural, aerial, and medical domains demonstrate strong cross-domain generalization and superior performance among inference-only ZOC methods.
Figures & tables
Figure 1: Motivation for CountZES. (a) Detector-driven exemplar selection is constrained by the initial candidate pool, as GroundingDINO [ 22 ] may provide few or no clean single-instance proposals in dense scenes. (b) Counting is sensitive to exemplar choice, with visually similar exemplars yielding different count estimates. (c) CountZES addresses both challenges through progressive discovery and selection, yielding complementary exemplars from semantic, count-driven, and feature cues. (Best viewed in zoom).
Figure 2: CountZES comparison with T2ICount [ 31 ] , GeCo [ 29 ] , and TFOC [ 35 ] across natural, aerial, and medical domains.
Figure 3: Overview of CountZES. Given an image and text prompt, CountZES progressively discovers and selects visual exemplars through three stages. DAE refines GroundingDINO proposals using CLIP similarity and SSES to obtain a text-aligned single-instance anchor. Conditioned on this anchor, DGE leverages the density response and P2P prompting to discover candidate instances, applies RoI-based single-instance filtering, and selects a statistically consistent exemplar through GGES. FCE further promotes visual representativeness by projecting density-guided candidates onto SAM’s feature space and selecting a representative exemplar through FRES. The exemplars selected across the three stages form a complementary set for final density estimation and counting.
Figure 4: Overview of GGES in DGE. Per-box counts from Bsingle estimate a pseudo-GT, while a SAM similarity map conditioned on bDAE captures visual correspondence. A composite score combining pseudo-GT proximity and similarity-map entropy guides density-based exemplar selection.
Figure 5: Overview of FRES in the FCE stage. Single-instance boxes Bsingle are projected onto the upsampled SAM feature map Φup to obtain ℓ2 -normalized regional embeddings. These embeddings are grouped via binary clustering, and the box closest to the majority-cluster centroid is selected as the representative exemplar.
Method
Venue
Inference
MAE
RMSE
only
ZSOC † [ 39 ]
CVPR’23
✗
22.09
115.17
CLIP-Count [ 14 ]
ACM’23
✗
17.78
106.62
CounTX [ 1 ]
BMVC’23
✗
15.88
106.29
VLCounter [ 15 ]
AAAI’24
✗
17.05
106.16
PseCo [ 12 ]
CVPR’24
✗
16.58
129.77
Table 1: Comparison of CountZES with SOTA ZOC methods on FSC-147 dataset. Best results are in bold and second-best are underlined. † indicates ZOC methods based on exemplar discovery. * indicates that CountingDINO is not zero-shot (box prompts).
Method
Inference-only
MAE
RMSE
CLIP-Count [ 14 ]
✗
11.96
16.61
CounTX [ 1 ]
✗
8.13
10.87
G-DINO [ 22 ]
✗
29.72
31.60
VLCounter [ 15 ]
✗
6.46
8.68
VA-Count † [ 44 ]
✗
10.63
13.20
GeCo [ 29 ]
✗
10.34
14.73
Table 2: CountZES compared to SOTA on CARPK dataset.
Density-based
Class-wise
Method (Inf-only)
Low
Med
High
Overall
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
GeCo (✗)
11.63
17.84
10.30
20.61
12.38
24.13
13.12
28.36
T2ICount (✗)
30.04
122.91
25.16
76.95
20.62
52.23
24.95
88.21
TFOC (✓)
7.20
10.36
16.07
21.63
36.88
63.82
17.79
35.74
CountZES (✓)
6.86
19.22
10.26
14.44
20.36
43.65
12.29
28.32
Table 3: CountZES compared to SOTA on PerSense-D dataset.
Method (Inf-only)
VGG
MBM
MAE
RMSE
MAE
RMSE
GeCo (✗)
67.32
77.53
92.59
104.52
T2ICount (✗)
151.35
162.81
30.01
44.44
CountSE (✗)
64.75
74.51
61.90
69.97
CountGD++ (✗)
173.70
183.74
121.50
125.76
TFOC (✓)
144.91
158.52
64.25
75.67
Table 4: Comparison of CountZES with SOTA ZOC methods on medical cell counting datasets VGG and MBM.
Setting
Parallel
Seq. 1
Seq. 2
MAE
RMSE
MAE
RMSE
MAE
RMSE
In-domain
17.12
92.57
16.58
91.90
15.77
91.40
Cross-domain
22.31
112.01
21.87
111.23
21.09
110.14
Table 5: Ablation on FSC-147. Left: Effect of stage interaction and ordering. Parallel independently conditions both DGE and FCE on DAE. Seq. 1 follows DAE → FCE → DGE, while Seq. 2 follows DAE → DGE → FCE, propagating the selected exemplar sequentially between stages. Right: Stage-wise contribution.
Configuration
MAE
RMSE
DAE only
21.73
118.03
DGE only
17.38
108.56
FCE only
19.42
116.53
CountZES
15.77
91.40
Table 6: Ablation studies of CountZES. (a) Advantage of cross-stage collaboration over single-stage exemplar selection. (b) Robustness to poor detector initialization. (c) Effect of the number of peaks k in SSES. (d) Effect of density-guided candidate discovery in DGE. (e) Flexible design with different segmentation backbones.
Figure 6: Qualitative comparison of CountZES with T2ICount [ 31 ] and GeCo [ 29 ] across diverse scenarios, including natural (FSC-147 and PerSense-D), aerial (CARPK), and medical cell-counting (VGG and MBM) images.
Figure 7: Left: For each stage, the top three exemplars are shown along with their predicted counts. One exemplar from each stage (arrows) provides complementary cues, and their combination in CountZES improves overall counting performance (last column). Right: Performance–runtime tradeoff among inference-only zero-shot counting methods. (best viewed in zoom).
Figure 8: Overview of SSES in the DAE stage. Starting from a detector-derived coarse box (green), CLIP similarity maps are used to extract high-confidence peaks. These peaks guide SAM to generate candidate masks, which are scored based on semantic alignment and spatial consistency. The highest-scoring mask is selected as the refined single-instance exemplar.
Figure 9: Overview of P2P prompting and single-instance filtering. Density peaks extracted from the density map serve as point prompts to SAM, whose masks are filtered via RoI-count to retain only single-instance regions before being passed to the GGES module for final selection in the DGE stage.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Box Descriptor Definition
FSC-147 (in-domain)
MAE
RMSE
Avg pool + ℓ2 -norm
15.77
91.40
Max pool + ℓ2 -norm
15.89
91.63
Appendix
Table 7: Sensitivity analysis of descriptor definition in FRES on FSC-147 dataset.
α
In-domain
Cross-domain
MAE
RMSE
MAE
RMSE
0.25
16.36
92.22
21.48
110.96
0.5
15.77
91.40
21.09
110.14
0.75
16.18
92.35
21.44
111.15
1
17.82
94.61
22.97
113.37
Appendix
Table 8: Left: Sensitivity to weight value α . Right: Effect of number of clusters in FRES.
Figure 10: Additional qualitative results of CountZES on FSC-147 dataset.
Figure 11: Failure case illustrating selection of incomplete exemplar boxes for the class “watches.” Due to SAM’s limited semantic awareness, point prompts from density peaks segment only the watch dial, excluding the wristband, which produces partial masks and incomplete boxes that lead to suboptimal exemplar selection.
Figure 12: Additional failure cases highlighting CountZES selection of incomplete boxes as exemplars. For “nail polishes,” SAM-based point prompt segmentation in CountZES captures only partial objects; for “comic books,” point prompts isolate characters or cover details instead of the full book; for “eggs,” SAM segments only the yolk region; and for “sauce bottles,” it segments only the label instead of the whole bottle. These examples show how imperfect masks can impair downstream exemplar selection in CountZES.
Northwestern Polytechnical University · Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China +1