Organizations: Northwestern Polytechnical University · Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China · Fudan University
Zero-shot object counting (ZSOC) aims to enumerate objects of arbitrary categories specified by text descriptions without requiring visual exemplars. However, existing methods often treat counting as a coarse retrieval task, suffering from a lack of fine-grained quantity awareness. Furthermore, they frequently exhibit spatial insensitivity and degraded generalization due to feature space distortion during model adaptation.To address these challenges, we present QICA, a novel framework that synergizes quantity perception with robust spatial cast aggregation. Specifically, we introduce a Synergistic Prompting Strategy (SPS) that adapts vision and language encoders through numerically conditioned prompts, bridging the gap between semantic recognition and quantitative reasoning. To mitigate feature distortion, we propose a Cost Aggregation Decoder (CAD) that operates directly on vision-text similarity maps. By refining these maps through spatial aggregation, CAD prevents overfitting while preserving zero-shot transferability. Additionally, a multi-level quantity alignment loss (LMQA) is employed to enforce numerical consistency across the entire pipeline. Extensive experiments on FSC-147 demonstrate competitive performance, while zero-shot evaluation on CARPK and ShanghaiTech-A validates superior generalization to unseen domains.
Figures & tables
Figure 1: A comparison between (a) standard ZSOC and (b) QICA. (a) Existing methods typically rely on unimodal prompting and direct feature interaction, suffering from a lack of fine-grained quantity awareness and spatial insensitivity , which often leads to overfitting. (b) QICA introduces numerically conditional collaborative prompts and achieves accurate density estimation through cost aggregation decoding via LMQA .
Figure 2: Overall architecture of QICA . (a) SPS jointly adapts the frozen vision and language encoders by mapping quantity-aware text prompts to visual prompts via a coupling function ( Φ ). (b) CAD first computes a similarity map between dense visual features ( V ) and category-only text embeddings ( Tcat ), then refines this map through spatial aggregation and multi-scale upsampling to predict the final density map. The entire framework is supervised by LMQA . Notably, the pathways involving explicit quantity information (Quantity Embed and Tfull ) are active only during training, while the model uses only category information at inference to ensure zero-shot generalization.
Figure 3: Visualization of the CAD pipeline . (a) Original image. (b) The similarity map derived from the fine-tuned CLIP exhibits fine-grained quantity awareness , effectively highlighting target instances while suppressing noise. (c) The aggregated cost map and (d) the final fused map demonstrate the recovery of fine-grained spatial structure , transforming coarse activations into precise density predictions.
Scheme
Method
Venue
Exemplar
Backbone
Val
Test
Avg.
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
Few-shot
FamNet [ 50 ]
CVPR’21
Visual
ResNet-50
24.32
70.94
22.56
101.54
23.44
86.24
CountTR [ 34 ]
WACV’22
Visual
ViT-B/16
13.13
49.83
11.95
91.23
12.54
70.53
LOCA [ 10 ]
ICCV’23
Visual
ResNet-50
10.24
32.56
10.97
56.97
10.61
44.77
PseCo [ 22 ]
CVPR’24
Visual
SAM-ViT-H/16
15.31
68.34
13.05
112.89
14.18
90.62
CountGD [ 2 ]
NeurIPS’24
Visual & Text
GDINO-Swin-B
7.10
26.08
6.75
43.65
6.93
34.87
Table 1: Performance comparison of QICA with other SOTA models on FSC-147 dataset. The best performance for each scheme is highlighted in bold , and the second-best performance for the zero-shot setting is underlined .
Table 2: Comparison with SOTA methods on CARPK.
Method
Category
MAE ↓
RMSE ↓
MCNN [ 70 ]
Specific
221.4
357.8
CrowdClip [ 31 ]
Specific
217.0
322.7
RCC [ 19 ]
Open-set
240.1
366.9
CounTX [ 1 ]
Open-set
219.8
351.0
CLIP-Count [ 25 ]
Open-set
192.6
308.4
CountGD* [ 2 ]
Open-set
141.9
258.0
Table 3: Comparison with SOTA methods including specific category counting on ShanghaiTech-A test set. * means the results are reproduced by open-source codes.
TP
SP
CAD
LMQA
Val
Test
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
○
○
○
○
16.82
68.45
16.15
124.32
●
○
○
○
15.46
63.28
14.89
115.63
○
●
○
○
14.27
59.14
13.72
107.85
○
●
●
○
13.41
57.62
12.88
99.74
○
●
●
●
12.98
56.35
12.41
97.28
Table 4: Ablation Study on FSC-147. ‘TP’ refers ‘Text Prompts only’ (no quantity), ‘SP’ represents ‘Synergistic Prompting’. Baseline/intermediate configurations (rows 1-3) process V directly, while CAD configurations (rows 4-5) operate on V - Tcat similarity maps.
Figure 4: Ablation on prompt depth (a) and prompt length (b) in QICA. We report results on the validation sets of FSC-147.
K
1
3
5
7
Val
MAE ↓
13.76
13.21
12.98
13.12
RMSE ↓
58.24
57.16
56.35
56.89
Test
MAE ↓
13.18
12.67
12.41
12.58
RMSE ↓
101.45
98.92
97.28
98.16
Table 5: Results for different value of K on FSC-147. More results are in the appendix.
Figure 5: Sensitivity analysis of loss weights on FSC-147 validation set. Left shows the effect of λ1 on Lencqty while right illustrates λ2 on Ldecqty . ★ Gold stars mark optimal configurations.
Figure 6: Qualitative comparison of QICA with VLCounter [ 28 ] and T2ICount [ 47 ] . Our text-image ‘similarity map’ exhibits reduced noise and more precise object delineation, which results a more accurate density estimation.
Figure 7: Efficiency analysis of QICA compared to SOTA methods. (a) Relationship between trainable parameters, MAE performance, and computational cost represented. (b) Comprehensive efficiency metrics including FLOPs and inference speed (FPS). ★ represents QICA-L (ViT-L/14) while • is QICA-B (ViT-B/16).
Zero-shot object counting (ZOC) aims to count instances of arbitrary object categories specified only through textual prompts. Recent training-free approaches leverage foundation models such as SAM to reformulate counting as a prompt-driven segmentation task, eliminating the need for costly counting-specific training data with point-level annotations. More recently, SAM3 introduced promptable concept segmentation, enabling the zero-shot segmentation of all instances corresponding to a text-defined concept. However, SAM3 struggles in densely populated scenes containing numerous small objects, where limited image resolution and insufficient attention to target-relevant regions often lead to missed instances and poor instance separation, hindering accurate object counting. To address this limitation, we propose AdaCount, a training-free framework for ZOC based on similarity-guided spatial and feature adaptation. AdaCount first estimates a prototype-driven similarity map that identifies target-relevant regions. This similarity map subsequently guides two complementary adaptations: (i) similarity-guided spatial warping, which reallocates image resolution toward target instances, and (ii) feature modulation, which amplifies target-relevant encoder representations. Together, these adaptations enable SAM3 to devote greater representational capacity to target-relevant regions while preserving global image context, without requiring any model retraining. Extensive experiments across six diverse counting benchmarks establish AdaCount as a new SOTA among training-free ZOC approaches.
Muhammad Ibraheem Siddiqui, Muhammad Haris Khan
Mohamed Bin Zayed University of Artificial Intelligence
Zero-shot object counting aims to estimate the number of objects specified by a text query without category-specific training. Recent approaches primarily rely on density regression or detection-style instance prediction. While effective, density-based models often suffer from spatial ambiguity and background leakage due to weakly regulated mass allocation, leading to fragmented or part-biased representations that increase counting error in complex scenes. In this work, we propose an instance-aware dual-decoder framework that structurally couples density and point representations for zero-shot object counting. Instead of treating density estimation as independent pixel-wise regression, we interpret it as a structured mass allocation problem over a latent set of object instances. Predicted instance centers induce a soft instance-wise decomposition of the density map, upon which we enforce two geometric constraints: (1) per-instance mass conservation, ensuring each object contributes approximately one unit of density mass, and (2) center-of-mass alignment, encouraging each density component to concentrate around its corresponding predicted center. These constraints introduce instance-level geometric consistency and lead to more accurate mass allocation, thereby reducing counting error. Extensive experiments on FSC-147, PUCPR+, and CARPK show that our approach consistently reduces counting error and establishes new state-of-the-art performance in zero-shot object counting.
Text-guided zero-shot object counters excel at spatial localization but categorize poorly on novel or fine-grained classes: natural language is too coarse to fully specify visual identity, so they fail to separate visually similar distractors. Few-shot counters sidestep this with visual exemplars, but require manual annotations on every image. To resolve this dilemma, we introduce RECOUNT, a plug-and-play framework for image-guided zero-shot counting. Rather than specify a category with a text prompt, our key insight is to specify it visually, from a single off-scene reference image. However, we find that a lone reference image provides narrow coverage of a category's appearance and is unreliable across diverse scenes. We therefore repurpose a diffusion model as an automated contrastive data engine that expands the reference into a diverse exemplar gallery, supplying the discriminative detail that text cannot. RECOUNT preserves the class-agnostic proposals of any frozen counter and offloads categorization to a separate visual module (a frozen backbone with a lightweight head trained on this synthetic data) that matches each proposal against the target and distractor galleries. Applied to a frozen counter, RECOUNT attains the best zero-shot accuracy on both benchmarks, cutting counting error (MAE) by 55% on LookAlikes and 21% on PairTally relative to the strongest prior zero-shot counter.
Adriano D'Alessandro, Ali Mahdavi-Amiri, Ghassan Hamarneh