Boosting Quantitive and Spatial Awareness for Zero-Shot Object Counting

Da Zhang; Bingyu Li; Feiyu Wang; Zhiyuan Zhao; Junyu Gao

cs.CVMar 17, 2026

Boosting Quantitive and Spatial Awareness for Zero-Shot Object Counting

Authors: Da Zhang, Bingyu Li, Feiyu Wang, Zhiyuan Zhao, Junyu Gao

Organizations: Northwestern Polytechnical University · Institute of Artificial Intelligence (TeleAI), China Telecom · University of Science and Technology of China · Fudan University

Abstract

Zero-shot object counting (ZSOC) aims to enumerate objects of arbitrary categories specified by text descriptions without requiring visual exemplars. However, existing methods often treat counting as a coarse retrieval task, suffering from a lack of fine-grained quantity awareness. Furthermore, they frequently exhibit spatial insensitivity and degraded generalization due to feature space distortion during model adaptation.To address these challenges, we present QICA, a novel framework that synergizes quantity perception with robust spatial cast aggregation. Specifically, we introduce a Synergistic Prompting Strategy (SPS) that adapts vision and language encoders through numerically conditioned prompts, bridging the gap between semantic recognition and quantitative reasoning. To mitigate feature distortion, we propose a Cost Aggregation Decoder (CAD) that operates directly on vision-text similarity maps. By refining these maps through spatial aggregation, CAD prevents overfitting while preserving zero-shot transferability. Additionally, a multi-level quantity alignment loss ( $\mathcal{L}_{MQA}$ ) is employed to enforce numerical consistency across the entire pipeline. Extensive experiments on FSC-147 demonstrate competitive performance, while zero-shot evaluation on CARPK and ShanghaiTech-A validates superior generalization to unseen domains.

Figures & tables

Figure 1: A comparison between (a) standard ZSOC and (b) QICA. (a) Existing methods typically rely on unimodal prompting and direct feature interaction, suffering from a lack of fine-grained quantity awareness and spatial insensitivity , which often leads to overfitting. (b) QICA introduces numerically conditional collaborative prompts and achieves accurate density estimation through cost aggregation decoding via $\mathcal{L}_{MQA}$ .

Figure 2: Overall architecture of QICA . (a) SPS jointly adapts the frozen vision and language encoders by mapping quantity-aware text prompts to visual prompts via a coupling function ( $\Phi$ ). (b) CAD first computes a similarity map between dense visual features ( V ) and category-only text embeddings ( $\text{T}^{cat}$ ), then refines this map through spatial aggregation and multi-scale upsampling to predict the final density map. The entire framework is supervised by $\mathcal{L}_{MQA}$ . Notably, the pathways involving explicit quantity information (Quantity Embed and $\text{T}^{full}$ ) are active only during training, while the model uses only category information at inference to ensure zero-shot generalization.

Figure 3: Visualization of the CAD pipeline . (a) Original image. (b) The similarity map derived from the fine-tuned CLIP exhibits fine-grained quantity awareness , effectively highlighting target instances while suppressing noise. (c) The aggregated cost map and (d) the final fused map demonstrate the recovery of fine-grained spatial structure , transforming coarse activations into precise density predictions.

Scheme	Method	Venue	Exemplar	Backbone	Val		Test		Avg.
Scheme	Method	Venue	Exemplar	Backbone	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
Few-shot	FamNet [ 50 ]	CVPR’21	Visual	ResNet-50	24.32	70.94	22.56	101.54	23.44	86.24
	CountTR [ 34 ]	WACV’22	Visual	ViT-B/16	13.13	49.83	11.95	91.23	12.54	70.53
	LOCA [ 10 ]	ICCV’23	Visual	ResNet-50	10.24	32.56	10.97	56.97	10.61	44.77
	PseCo [ 22 ]	CVPR’24	Visual	SAM-ViT-H/16	15.31	68.34	13.05	112.89	14.18	90.62
	CountGD [ 2 ]	NeurIPS’24	Visual & Text	GDINO-Swin-B	7.10	26.08	6.75	43.65	6.93	34.87

Table 1: Performance comparison of QICA with other SOTA models on FSC-147 dataset. The best performance for each scheme is highlighted in bold , and the second-best performance for the zero-shot setting is underlined .

Panel of Table 2 — Table 2: Comparison with SOTA methods on CARPK.

Method	Category	MAE $\downarrow$	RMSE $\downarrow$
MCNN [ 70 ]	Specific	221.4	357.8
CrowdClip [ 31 ]	Specific	217.0	322.7
RCC [ 19 ]	Open-set	240.1	366.9
CounTX [ 1 ]	Open-set	219.8	351.0
CLIP-Count [ 25 ]	Open-set	192.6	308.4
CountGD* [ 2 ]	Open-set	141.9	258.0

Table 3: Comparison with SOTA methods including specific category counting on ShanghaiTech-A test set. * means the results are reproduced by open-source codes.

TP	SP	CAD	$\mathcal{L}_{MQA}$	Val		Test
				MAE $\downarrow$	RMSE $\downarrow$	MAE $\downarrow$	RMSE $\downarrow$
○	○	○	○	16.82	68.45	16.15	124.32
●	○	○	○	15.46	63.28	14.89	115.63
○	●	○	○	14.27	59.14	13.72	107.85
○	●	●	○	13.41	57.62	12.88	99.74
○	●	●	●	12.98	56.35	12.41	97.28

Table 4: Ablation Study on FSC-147. ‘TP’ refers ‘Text Prompts only’ (no quantity), ‘SP’ represents ‘Synergistic Prompting’. Baseline/intermediate configurations (rows 1-3) process

\mathbf{V}

directly, while CAD configurations (rows 4-5) operate on

\mathbf{V}

\mathbf{T}^{cat}

similarity maps.

Panel of Figure 4 — Figure 4: Ablation on prompt depth (a) and prompt length (b) in QICA. We report results on the validation sets of FSC-147.

	K	1	3	5	7
Val	MAE $\downarrow$	13.76	13.21	12.98	13.12
	RMSE $\downarrow$	58.24	57.16	56.35	56.89
Test	MAE $\downarrow$	13.18	12.67	12.41	12.58
	RMSE $\downarrow$	101.45	98.92	97.28	98.16

Table 5: Results for different value of K on FSC-147. More results are in the appendix.

Figure 5: Sensitivity analysis of loss weights on FSC-147 validation set. Left shows the effect of

\lambda_{1}

\mathcal{L}_{\text{enc}}^{\text{qty}}

while right illustrates

\lambda_{2}

\mathcal{L}_{\text{dec}}^{\text{qty}}

. ★ Gold stars mark optimal configurations.

Figure 6: Qualitative comparison of QICA with VLCounter [ 28 ] and T2ICount [ 47 ] . Our text-image ‘similarity map’ exhibits reduced noise and more precise object delineation, which results a more accurate density estimation.

Panel of Figure 7 — Figure 7: Efficiency analysis of QICA compared to SOTA methods. (a) Relationship between trainable parameters, MAE performance, and computational cost represented. (b) Comprehensive efficiency metrics including FLOPs and inference speed (FPS). ★ represents QICA-L (ViT-L/14) while • is QICA-B (ViT-B/16).

Object Counting

Object counting estimates the number and locations of target instances in images or video, from dense crowds to fine-grained categories.

See all 58 papers

AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting
Jul 2, 2026Muhammad Ibraheem Siddiqui, Muhammad Haris KhanTest-Time Adaptation Vision Foundation Model Adaptation
Mohamed Bin Zayed University of Artificial Intelligence
DualCount: Structurally Consistent Density and Point Modeling for Zero-Shot Object Counting
Sep 14, 2026Xuan Cuong NgoObject Counting
University of Arkansas, USA
RECOUNT: Reference-guided Counting with Synthetic Visual Exemplars
Aug 20, 2026Adriano D'Alessandro, Ali Mahdavi-Amiri, Ghassan HamarnehObject Counting
Simon Fraser University

Jul 2, 2026J/K move · Enter open · S save

AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting

Muhammad Ibraheem Siddiqui, Muhammad Haris Khan

Mohamed Bin Zayed University of Artificial Intelligence

Test-Time Adaptation Vision Foundation Model Adaptation Object Counting

Open paper AskPDF arXiv

Zero-shot object counting (ZOC) aims to count instances of arbitrary object categories specified only through textual prompts. Recent training-free approaches leverage foundation models such as SAM to reformulate counting as a prompt-driven segmentation task, eliminating the need for costly counting-specific training data with point-level annotations. More recently, SAM3 introduced promptable concept segmentation, enabling the zero-shot segmentation of all instances corresponding to a text-defined concept. However, SAM3 struggles in densely populated scenes containing numerous small objects, where limited image resolution and insufficient attention to target-relevant regions often lead to missed instances and poor instance separation, hindering accurate object counting. To address this limitation, we propose AdaCount, a training-free framework for ZOC based on similarity-guided spatial and feature adaptation. AdaCount first estimates a prototype-driven similarity map that identifies target-relevant regions. This similarity map subsequently guides two complementary adaptations: (i) similarity-guided spatial warping, which reallocates image resolution toward target instances, and (ii) feature modulation, which amplifies target-relevant encoder representations. Together, these adaptations enable SAM3 to devote greater representational capacity to target-relevant regions while preserving global image context, without requiring any model retraining. Extensive experiments across six diverse counting benchmarks establish AdaCount as a new SOTA among training-free ZOC approaches.

Abstract

Object Counting

Explore similar work

AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting

DualCount: Structurally Consistent Density and Point Modeling for Zero-Shot Object Counting

RECOUNT: Reference-guided Counting with Synthetic Visual Exemplars