Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges

Joana Konadu Owusu; Shivanand Venkanna Sheshappanavar

cs.CVAug 24, 2026

Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges

Authors: Joana Konadu Owusu, Shivanand Venkanna Sheshappanavar

Organizations: Geometric Intelligence Research Lab., Dept. of Electrical Engineering and Computer Science, University of Wyoming, Laramie, WY, USA.

Abstract

Object-counting methods have rapidly shifted from class-specific density regression to open-vocabulary, foundation-model-backed counters. These methods now enumerate instances from various visual and textual prompts. While this shift marks major conceptual progress, our survey argues that claims of universal generality have outpaced the evaluative infrastructure. Most progress metrics rely on a few saturated benchmarks that models exploit for statistical regularities. Newly introduced diagnostic datasets reveal systematic failures in semantic grounding, temporal identity, and spatial reasoning with occlusion. To address these failures, we introduce a five-axis taxonomy (modality, mechanism, prompting, supervision level, and generalization setting). We use this taxonomy to audit the literature across application domains, including microscopy, remote sensing, crowd counting, and agriculture. This formalizes prevailing challenges into six structural contradictions. From these, we propose a roadmap for compositional scene understanding, active counting agents, and unified multimodal evaluation protocols. The main imperative is to build a robust evaluation infrastructure to distinguish open-world generalization from benchmark-specific optimization, rather than simple incremental engineering.

Figures & tables

Era	Years	Representative papers	N
Class-specific density regression	2013–date	[ 2 , 3 , 4 , 16 , 17 , 18 ]	176
Class-agnostic/ few-shot	2018–date	[ 19 , 20 , 21 , 22 , 23 ]	244
Text-guided / open-vocab.	2023–date	[ 25 , 26 , 27 , 28 , 29 ]	76
Multimodal reasoning-based	2024–date	[ 30 , 31 , 32 , 33 ]	4

TABLE I: The four methodological eras, with representative papers reviewed in depth across Sections V – VIII .

Fig. 1: Evolution of object counting methodological eras, 2010–2026. The timeline marks the approximate onset of each of the four eras and their representative benchmark traced in this survey.

Group	Surveys	Modality	Years
Density and Crowd Counting	[ 54 , 55 , 49 , 56 , 57 ]	RGB images	2017–25
Class Agnostic Counting	[ 48 ]	RGB images	2026
Application-Specific	[ 51 , 58 , 59 , 60 , 61 , 62 , 50 , 63 ]	RGB images, Video	2020–25
Multimodal and Foundation-Models	[ 48 , 64 ]	RGB images	2025–26
Cross Domain	Ours	RGB images, Video, 3D, Thermal, Audio-Visual	2026

TABLE II: Comparison of existing counting surveys.

Fig. 2: Object counting as visual reasoning. The same scene can yield different valid counts depending on the query and evidence protocol.

Fig. 3: The four core algorithmic paradigms of object counting. The paradigms differ primarily in how the query $q$ interacts with the visual evidence before producing the scalar count $\hat{c}$ .

Paradigm	MAE	RMSE	Loc. AP	Track Acc.
Density-Map Reg. (P1)	✓	✓	—	—
Cross-Attention (P2)	✓	✓	—	—
Promptable Det./Seg. (P3)	✓	$\sim$	✓	—
Multimodal Reas. (P4)	✓	$\sim$	—	—

TABLE III: Metric–paradigm incompatibility matrix.

Fig. 4: Semantic grounding illusion: A model may achieve zero error while counting the wrong instances

Fig. 5: Illustration of the five-axis taxonomy of object counting. A counting claim is defined by the evidence (sensing modality) $X$ , target query $q$ , mechanism $m$ , and supervision $S$ consumed by the model $f_{\theta}$ to produce a prediction $\hat{c}$ . The generalization setting $G$ and evaluation protocol $E$ are shown as external layers: $G$ specifies the semantic scope the model is claimed to handle, and $E$ determines how $\hat{c}$ is judged; neither is an input to $f_{\theta}$ itself.

Method	Venue	Prompting	Sup.	Gen.
a. Density-map and Regression counting (§ V-A )
Wang et al. [ 128 ]	ICIP’16	Exemplar	Spatial	CA
CSRNet [ 129 ]	CVPR’18	Fixed class	Spatial	CS
Ma et al. [ 130 ]	ICCV’19	Fixed class	Spatial	CS
DM-Count [ 131 ]	NeurIPS’20	Fixed class	Spatial	CS
RCC [ 105 ]	arXiv’22	Reference-less	Weak	CA

TABLE IV: Image-based object-counting methods, grouped by counting mechanism and cross-referenced against the taxonomy axes. CS = Class-specific; CA = Class-agnostic.

Fig. 6: Frame-visible and tracking-based video counting. Frame-level methods estimate visible occupancy at each time step, while tracking and re-identification methods estimate distinct identities across a sequence.

Method	Venue	Prompting	Sup.	Gen.
a. Frame-level density and short-range modeling (§ VI-A )
Liu et al. [ 149 ]	ECCV’20	Fixed class	Spatial	CS
DroneBird [ 75 ]	ICLR’25	Fixed class	Spatial	CS
Shu & Chan [ 150 ]	CVPR’26	Fixed class	Spatial	CS
b. Tracking and unique-instance counting (§ VI-B )
Li et al. [ 42 ]	J. Database’23	Fixed class	Spatial	CS

TABLE V: Video and temporal counting methods, grouped by temporal counting mechanism and cross-referenced against the taxonomy axes. CS = Class-specific; CA = Class-agnostic.

Fig. 7: Trajectory-event and spatial-aggregation counting. Event methods count valid crossings, while mosaicking counts over a reconstructed region.

Fig. 8: Image-plane vs. depth occlusion. RGB reveals frontmost objects, while depth or point clouds can separate stacked instances.

Fig. 9: Illustration of geometric counting mechanisms. RGB-D methods add depth cues to 2D counting, point-cloud methods count directly from 3D structure, and multi-view methods fuse viewpoints to reduce single-view ambiguity.

Method	Venue	Prompting	Sup.	Gen.
a. Depth-aware and RGB-D counting (§ VII-A )
Lian et al. [ 39 ]	CVPR’19	Fixed class	Spatial	CS
Zhang et al. [ 167 ]	Sensors’23	Fixed class	Spatial	CS
Nan et al. [ 168 ]	ICASSP’25	Text	Spatial	CA
Ma et al. [ 78 ]	TCSVT’25	Fixed class	Spatial	CS
Sun et al. [ 169 ]	IJCV’26	Fixed class	Spatial	CS

TABLE VI: 3D, depth-aware, and multi-view counting methods, grouped by modality and cross-referenced against the taxonomy axes. CS = Class-specific; CA = Class-agnostic.

Method	Venue	Prompting	Sup.	Gen.
LVLM-Count [ 176 ]	TMLR’26	Text (QA)	None	CA
ValidCounter [ 177 ]	Inf. Sci.’25	Exemplar	None	CA
GroundCount [ 178 ]	arXiv’26	Text (QA)	None	CS
WS-COC [ 179 ]	ICLR’26	Text (QA)	Weak	CA
SITUATE [ 180 ]	VISAPP’26	Text (QA)	Weak	CA
AV-Reasoner [ 30 ]	CVPR’26	Multimodal QA	Mixed	CA

TABLE VII: Foundation-model-based counting architectures, cross-referenced against the taxonomy axes. CS = Class-specific; CA = Class-agnostic.

Method	Venue	Test MAE	$\Delta$
Three-exemplar counting (visual-only)
FamNet [ 20 ]	CVPR’21	22.08	–
BMNet+ [ 22 ]	CVPR’22	14.62	$-34\%$
CounTR [ 21 ]	BMVC’22	11.95	$-18\%$
LOCA [ 65 ]	ICCV’23	10.79	$-10\%$
CACViT [ 135 ]	AAAI’24	9.13	$-15\%$

TABLE VIII: Longitudinal progress on FSC-147 under visual-only and text-only counting settings.

\Delta

is the relative change in Test MAE versus the immediate preceding row.

Dataset	Venue	Annotation	Scale	Metrics	Description
a. Images (§ X-B )
FSC-147 [ 20 ]	CVPR’21	Points + exemplar boxes	6,135 img; 147 cat.	MAE, RMSE	Class-agnostic counting
MixCount [ 190 ]	arXiv’26	Text + instance labels	Synthetic; mixed-category scenes	MAE, RMSE	Open-vocabulary counting
b. Video and Temporal (§ X-C )
DroneCrowd [ 192 ]	CVPR’21	boxes + trajectories	112 vid.; 33,600 frm; 4.8M ann.	MAE, MSE; tracking metrics	Drone crowd counting
CroHD [ 193 ]	CVPR’21	Head boxes + track IDs	11,463 frm; 2.27M head boxes	HOTA, MOTA, IDF1	Dense-crowd tracking

TABLE IX: Object-counting datasets and benchmarks, grouped by input modality. N/R = not reported.

Object Counting

Object counting estimates the number and locations of target instances in images or video, from dense crowds to fine-grained categories.

See all 61 papers

Count Anything at Any Granularity
May 11, 2026Chang Liu, Haoning Wu, Weidi XieObject Counting Multimodal Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University, China · CMIC, Shanghai Jiao Tong University, China
Count Anything
May 29, 2026Mengqi Lei, Shuokun Cheng, Wei Bao +4Object Counting
1{BNRist, THUIBCS, BLBCI, School of Software}, Tsinghua University · School of Computer Science and Technology, China University of Geosciences, Wuhan · 3State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
The MixCount Dataset: Bridging the Data Gap for Open-Vocabulary Object Counting
May 18, 2026Corentin Dumery, Niki Amini-Naieni, Shervin Naini +1Synthetic Data Augmentation Object Counting
EPFL · University of Oxford · Northwestern University

May 11, 2026J/K move · Enter open · S save

Count Anything at Any Granularity

Chang Liu, Haoning Wu, Weidi Xie

School of Artificial Intelligence, Shanghai Jiao Tong University, China · CMIC, Shanghai Jiao Tong University, China

Object Counting Multimodal Large Language Models Fine-Grained Visual Perception Multimodal Understanding

Open paper AskPDF arXiv

Open-world object counting remains brittle: despite rapid advances in vision-language models (VLMs), reliably counting the objects a user intends is far from solved. We argue that a central reason is that counting granularity is left implicit; users may refer to a specific identity, an attribute, an instance type, a category, or an abstract concept, yet most methods treat "what to count" as a single, category-level matching problem. In this work, we redefine open-world counting as multi-grained counting, where visual exemplars specify target appearance and fine-grained text, with optional negative prompts, specifies the intended semantic granularity across five explicit levels. Making granularity explicit, however, exposes a critical data bottleneck: existing counting datasets lack the multi-category scenes, controlled distractors, and instance-level annotations needed to verify fine-grained prompt semantics. To address this, we propose the first fully automatic data-scaling pipeline that integrates controllable 3D synthesis with consistent image editing and VLM-based filtering, and use it to construct KubriCount, the largest and most comprehensively annotated counting dataset to date, supporting both training and multi-grained evaluation. Systematic benchmarking reveals that both multimodal large language models and specialist counting models exhibit severe prompt-following failures under fine-grained distinctions. Motivated by these findings, we train HieraCount, a multi-grained counting model that jointly leverages text and visual exemplars as complementary target specifications. HieraCount substantially improves multi-grained counting accuracy and generalizes robustly to challenging real-world scenarios. The project page is available here: https://verg-avesta.github.io/KubriCount/.

Abstract

Object Counting

Explore similar work

Count Anything at Any Granularity

Count Anything

The MixCount Dataset: Bridging the Data Gap for Open-Vocabulary Object Counting