UniCounting: Instance-Aware Proposal Consolidation for Image-Query-Free Multi-Category Counting

Jinshi Liu; Pan Liu; Lei He; Weichao Luo; Rui Qian

cs.CVOct 6, 2026

UniCounting: Instance-Aware Proposal Consolidation for Image-Query-Free Multi-Category Counting

Authors: Jinshi Liu, Pan Liu, Lei He, Weichao Luo, Rui Qian

Organizations: Shenzhen University · The Hong Kong University of Science and Technology (Guangzhou) · Hunan University of Science and Technology · Peng Cheng Laboratory · Fudan University

Abstract

Visual counting is commonly formulated as counting a single specified target, with a model receiving an image-specific exemplar, text query, or target category and returning a single count. We instead study fixed-vocabulary image-query-free multi-category counting. A global vocabulary is fixed for each run, and, given only an RGB image, the model predicts a complete category--count vector without being told which categories appear. We present UniCounting, which casts counting as instance-aware structural inference over an over-complete proposal set. Generic segmenters produce duplicate masks, partial views, and proposals from neighboring instances; semantic scores can name them but cannot determine which denote the same object. Frozen SAM~2.1 generates masks, while frozen DINOv2 and OpenCLIP provide relation and category features. A 3,267-parameter category-shared relation head predicts same-instance affinities from instance-mask-derived supervision. Sparse graph construction, representative selection, labeling, and background-margin admission then convert each admitted component into one count with replayable group evidence. Only the relation head is trained, without count or density-map targets. On COCO clean500, UniCounting obtains lower point-estimate vector $\ell_1$ error and absent-class false mass than calibrated OWLv2-All80, with comparable micro presence F1. Under a matched decoder, the learned relation reduces both errors relative to mask containment, mask IoU, CLIP, and DINO, while revealing a fragmentation--merge trade-off. We also report transfer diagnostics on OmniCount-sub, FSC-147, and CARPK.

Figures & tables

Figure 1: Qualitative counting examples. In the upper example, colored boxes mark representative proposals for the displayed groups, and arrows connect each representative to its fixed-vocabulary category and predicted count. In the lower example, one predicted component merges proposals supporting at least five distinct ground-truth hot-dog instances. The remaining discrepancy between the ground-truth count of 11 and the single predicted component may also include upstream proposal misses.

Figure 2: UniCounting modules and data flow. Frozen SAM 2.1 generates over-complete candidates, while frozen DINOv2 and OpenCLIP encode the three proposal views for relation features. The fixed category head uses only the masked- and box-view OpenCLIP embeddings to compute fixed-vocabulary category scores. The trained relation head supplies the same-instance affinity used by thresholding, top2-OR, and union-find. Deterministic maximum-category-margin representative selection, raw-cosine labeling, and background-margin admission then produce the count vector and its group evidence. Snowflake, lightning, and gray modules denote frozen, trained, and parameter-free computation, respectively.

Method	Vector $\ell_{1}\downarrow$	Present MAE $\downarrow$	False mass $\downarrow$	Total MAE $\downarrow$	Macro-F1 $\uparrow$	Micro-F1 $\uparrow$
Zero vector	7.018	2.425	0.000	7.018	0.000	0.000
Fixed-list detector (OWLv2-All80)	6.860	2.205	0.474	5.800	0.472	0.415
UniCounting	6.485 $\pm$ 0.058	2.129 $\pm$ 0.005	0.325 $\pm$ 0.062	5.868 $\pm$ 0.050	0.407 $\pm$ 0.007	0.416 $\pm$ 0.005
Faster R-CNN (fully supervised ref.)	3.098	0.916	0.446	2.078	0.809	0.832

Table 1: Primary complete-vector results on COCO clean500. Every row predicts the same COCO80 vector. OWLv2 receives the full raw COCO80 list identically for every image, while UniCounting uses a fixed taxonomy and no image-specific cue. Its row reports mean

\pm

sample standard deviation over three relation-head seeds; seed-specific thresholds are given in the supplement. The point estimate for UniCounting minus OWLv2 vector

\ell_{1}

-0.375

(6.485 versus 6.860). The separated Faster R-CNN row uses full COCO box supervision and is reported only as a supervised reference, not as a like-for-like training-regime comparison.

Edge/grouping rule	$\tau$	Vector $\ell_{1}\downarrow$	False mass $\downarrow$	Total MAE $\downarrow$	Micro-F1 $\uparrow$
No grouping / singletons	–	29.187	20.347	24.440	0.319
Mask containment	0.005	9.935	4.697	5.020	0.427
Mask IoU	0.005	13.649	8.265	7.965	0.374
CLIP cosine	0.500	7.335	1.452	4.987	0.431
DINO cosine	0.310	6.814	0.809	5.463	0.434
Relation-head score $+$ overlap guard	0.465	9.597	4.361	4.635	0.414

Table 2: Controlled edge and grouping ablation on COCO clean500. The two relation-head rows are controlled variants rather than the primary UniCounting configuration: both use a common edge threshold

\tau

=0.465, whereas primary UniCounting uses seed-specific instance thresholds selected on inner45. The table reports end-task metrics after a shared downstream decoder; full pre-admission partition diagnostics are provided in the supplement. Mask IoU is the binary-mask intersection over union; the overlap guard instead adds a positive-intersection constraint to the relation-head score. Every row is the arithmetic mean of seed-specific point metrics for seeds 17, 42, and 73. For containment, mask IoU, CLIP, DINO, relation-head, singleton, and overlap-guard rows, the candidate pool, pair universe, graph construction, representative and labeling rules, and run-matched background-margin admission are fixed; only the edge score, common edge threshold, or stated grouping control changes.

Figure 3: Fragmentation–merge trade-off under the matched decoder on COCO clean500. Each point represents an edge or grouping rule; the horizontal and vertical axes report the fragmented-instance rate and component merge error, respectively, so lower-left is preferred. Point annotations report the resulting complete-vector

\ell_{1}

error after the shared downstream decoder. The relation-head score produces the lowest fragmentation and vector error among the tested edge scores, while incurring more merges than the conservative mask-containment and mask-IoU rules.

Stage	Selected variant	Vector $\ell_{1}\downarrow$	Micro-F1 $\uparrow$
Graph	same-instance edges $\star$	$6.867{\pm}0.06$	$0.417{\pm}0.005$
Represent.	max category margin	$6.837{\pm}0.09$	$0.423{\pm}0.004$
Labeling	raw cosines	$6.837{\pm}0.09$	$0.423{\pm}0.004$
Admission	background-margin	$6.793{\pm}0.05$	$0.426{\pm}0.007$

Table 3: Decoder choices selected on inner45.

\star

refers to all eligible/top2-OR. Each stage carries forward the choices selected at preceding stages. The table is a staged selection trace rather than a cumulative-improvement analysis. The complete candidate matrix and frozen target-set results are reported in the supplement.

Figure 4: Stage-wise effects of the decoder choices selected on inner45 when audited on the frozen clean500 and OmniCount-sub target sets. Panel (a) reports vector-

\ell_{1}

improvement, defined as the baseline error minus the selected-variant error; panel (b) reports micro-F1 improvement, defined as the selected-variant score minus the baseline score. Positive values therefore denote improvement in both panels. Baselines are all-edges grouping, maximum-area representation, component-mean labeling, and no admission, respectively. Representative selection and labeling provide the largest consistent gains, whereas the effect of top2-OR sparsification is metric- and dataset-dependent.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Selected successful complete-vector predictions on COCO clean500. All examples use the primary UniCounting decoder with the fixed seed-17 checkpoint and thresholds selected on inner45. Colored contours indicate the representatives of admitted proposal components, and the accompanying summaries compare all nonzero ground-truth and predicted coordinates. The examples were filtered using fixed correctness and consolidation-complexity criteria and then selected for visual legibility and diversity; they are intended to illustrate successful behaviors rather than estimate average performance. Ground-truth annotations are used only for visualization and evaluation.

Setting	Value
SAM 2.1 ( Ravi et al. 2025 ) checkpoint	sam2.1-hiera-small , SHA-256 prefix 0a4067b1
Input resolution	checkpoint-native preprocessing
Points per side / crop layers	24 / 0
Predicted-IoU threshold	0.7
Stability threshold	0.8
Box NMS threshold	0.7

Appendix

Table S2: Automatic mask generation and filtering.

Setting	Value
GPU / memory	NVIDIA GeForce RTX 4090 / 24,564 MiB (approximately 24 GB)
CPU	Intel Core i9-14900KF
System memory	62 GiB available (approximately 64 GB physical memory)
Operating system	Ubuntu 24.04.4 LTS under WSL2
Linux kernel	6.6.114.1- microsoft-standard-WSL2
Python	3.12.3

Appendix

Hardware and software environment used for the experiments.

Edge score	threshold	vector $\ell_{1}\downarrow$	micro-F1 $\uparrow$
Mask containment	0.005	12.5333	0.4289
Mask IoU	0.005	15.8222	0.3920
CLIP cosine	0.500	7.2444	0.4632
DINO cosine	0.310	7.1185	0.4160
Relation-head score	0.465	6.7930	0.4260

Appendix

Table S4: Frozen threshold selection for the controlled edge-score study in inner45. Mask containment is

|m_{i}\cap m_{j}|/\min(|m_{i}|,|m_{j}|)

, whereas mask IoU is

|m_{i}\cap m_{j}|/|m_{i}\cup m_{j}|

. CLIP and DINO use cosine similarity between normalized

0.05

masked-view plus

0.95

box-view features.

Method	Vector $\ell_{1}\downarrow$	Micro-F1 $\uparrow$
Zero vector	6.447	0.000
UniCounting	6.715 $\pm$ 0.123	0.283 $\pm$ 0.011

Appendix

Table S5: OmniCount-sub complete-vector diagnostic. Both rows predict the same 191-dimensional vector. OmniCount-sub is a project holdout used only as a transfer diagnostic, not as an official benchmark.

Method / seed	TP	FP	FN	Micro-P	Micro-R	Micro-F1
OWLv2-All80	432	202	1015	0.681	0.299	0.415
UniCounting / 17	429	187	1,018	0.696	0.296	0.416
UniCounting / 42	418	165	1,029	0.717	0.289	0.412
UniCounting / 73	420	125	1,027	0.771	0.290	0.422

Appendix

Table S6: Pooled image–category presence diagnostics on COCO clean500. TP, FP, and FN are reported for each prediction vector; precision, recall, and F1 are computed from those pooled counts. Its low micro-F1 is primarily associated with low micro recall rather than excessive false-positive presence.

Grouping rule	Pair F1 $\uparrow$	Purity $\uparrow$	Merge error $\downarrow$	Comp./instance $\downarrow$	Fragmented $\downarrow$	Rep. IoU $\uparrow$
No grouping / singletons	0.0000	1.0000	0.0000	4.0770	0.6320	0.1680
Mask containment	0.6910	0.9090	0.1150	1.4150	0.1350	0.2020
Mask IoU	0.4984	0.9558	0.0515	1.8104	0.2282	0.1758
CLIP cosine	0.5040	0.6740	0.4680	1.6320	0.4300	0.1450
DINO cosine	0.6270	0.6510	0.5060	1.1950	0.1690	0.1760
Relation-head score $+$ overlap guard	0.6820	0.8500	0.1660	1.4180	0.1830	0.2030

Appendix

Table S7: Full pre-admission partition diagnostics for the controlled edge and grouping ablation on COCO clean500. Rows follow Table 2 of the main paper. Every value is the arithmetic mean of seed-specific point metrics for seeds 17, 42, and 73. The candidate pool, pair universe, graph construction, representative rule, and labeling rule are fixed; only the edge score, common edge threshold, or stated grouping control changes. Admission is not applied for these partition diagnostics.

		Inner45		COCO clean500		OmniCount-sub 2K
Stage	Variant	$\ell_{1}\downarrow$	Micro-F1 $\uparrow$	$\ell_{1}\downarrow$	Micro-F1 $\uparrow$	$\ell_{1}\downarrow$	Micro-F1 $\uparrow$
Graph	hard semantic $\land$ instance	32.978 $\pm$ 0.868	0.300 $\pm$ 0.003	30.597 $\pm$ 1.040	0.291 $\pm$ 0.003	26.553 $\pm$ 1.073	0.144 $\pm$ 0.003
	same-instance edges, all	6.889 $\pm$ 0.000	0.365 $\pm$ 0.000	6.686 $\pm$ 0.000	0.342 $\pm$ 0.000	6.769 $\pm$ 0.000	0.257 $\pm$ 0.000
	same-instance edges, top2 $\star$	6.867 $\pm$ 0.059	0.417 $\pm$ 0.005	6.810 $\pm$ 0.010	0.350 $\pm$ 0.002	6.976 $\pm$ 0.009	0.250 $\pm$ 0.002
Represent.	maximum area	7.296 $\pm$ 0.078	0.332 $\pm$ 0.008	6.837 $\pm$ 0.011	0.345 $\pm$ 0.002	7.293 $\pm$ 0.012	0.144 $\pm$ 0.003
	learned completeness	7.400 $\pm$ 0.102	0.323 $\pm$ 0.007	6.858 $\pm$ 0.003	0.339 $\pm$ 0.003	7.305 $\pm$ 0.014	0.141 $\pm$ 0.003

Appendix

Table S9: Complete decoder selection and frozen target-set audit. Inner45 is the only split used for selecting graph construction, representative, labeling, and admission variants. In the graph block, the representative is maximum-category-margin, labeling uses its raw cosines, and admission is disabled. The representative block carries forward the starred graph and uses representative raw-cosine labeling without admission; the labeling block additionally carries forward the starred representative; and the admission block carries forward all preceding starred choices. The clean500 and OmniCount-sub columns are reported only after all choices are frozen. Stars denote the variants selected on inner45; target-set winners are not used for model selection.

Explore similar work

May 29, 2026cs.CV

Count Anything

Object counting remains fragmented across domain-specific datasets and task formulations, despite rapid progress in generalist vision models. Existing counting models are often tailored to scenarios such as crowds, vehicles, cells, crops, or remote-sensing objects, and thus struggle to generalize across categories, visual domains, object scales, and density distributions. In this paper, we study text-guided object counting across domains, where a model takes an image and a natural-language query as input and returns an instance-grounded set of target points whose cardinality gives the count. This formulation unifies category-conditioned counting with interpretable spatial localization. To support this setting, we construct CLOC, a Cross-domain Large-scale Object Counting dataset that reorganizes diverse public data sources into a unified benchmark. CLOC covers six visual domains: General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology, with about 220K images, 619 categories, and 15M object instances. Based on CLOC, we propose Count Anything, a generalist model for text-guided object counting. Unlike density-map-based methods, which dominate counting models, Count Anything adopts discrete instance points and performs dual-granularity instance enumeration. A Region-level Sparse Counter provides object-level anchors for large and sparse targets, while a Pixel-level Dense Counter handles small, crowded, and weakly bounded targets via dense point prediction. A point-centric supervision strategy enables learning from heterogeneous annotations, and Complementary Count Fusion combines both counters in a parameter-free manner. Extensive experiments show that Count Anything achieves strong accuracy and multi-domain generalization, outperforming existing open-world counting methods. Code is available at: https://github.com/Mengqi-Lei/count-anything.

Mengqi Lei, Shuokun Cheng, Wei Bao +4

1{BNRist, THUIBCS, BLBCI, School of Software}, Tsinghua University · School of Computer Science and Technology, China University of Geosciences, Wuhan · 3State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University

Counting Vision Datasets Microscopy

Ask PDF

Jul 18, 2026cs.CV

Spatially-Aware Class-Agnostic Object Counting

Generalised object counting aims to estimate the number of instances of an arbitrary object category from a single image, but many recent methods can struggle on structurally complex objects due to limited spatial modelling. We present \textit{UpCount}, a class-agnostic counter designed to better preserve spatial structure. UpCount strengthens the visual representation by extracting multi-layer features from a ViT-B/16 encoder and reassembling them into a refined multi-scale pyramid that is spatially refined using Dense Prediction Transformers and FeatUp, yielding features with improved structural and spatial sensitivity; a proposal--verification counting head then identifies repeated patterns and produces a density map for the final count. On FSC-147, UpCount achieves 12.39 MAE and 100.89 RMSE on the test set, and it transfers effectively to vehicle counting on CARPK (6.27 MAE, 8.79 RMSE). Code: https://github.com/r28112072-rgb/upcount

Robert Wijaya, Md. Tanvir Hossain, Amanda Kau +1

Singapore University of Technology and Design, Singapore · The University of Canterbury, Christchurch, New Zealand · The Australian National University, Canberra, Australia

Feature Pyramid Counting Objects

Ask PDF

May 11, 2026cs.CV

Count Anything at Any Granularity

Open-world object counting remains brittle: despite rapid advances in vision-language models (VLMs), reliably counting the objects a user intends is far from solved. We argue that a central reason is that counting granularity is left implicit; users may refer to a specific identity, an attribute, an instance type, a category, or an abstract concept, yet most methods treat "what to count" as a single, category-level matching problem. In this work, we redefine open-world counting as multi-grained counting, where visual exemplars specify target appearance and fine-grained text, with optional negative prompts, specifies the intended semantic granularity across five explicit levels. Making granularity explicit, however, exposes a critical data bottleneck: existing counting datasets lack the multi-category scenes, controlled distractors, and instance-level annotations needed to verify fine-grained prompt semantics. To address this, we propose the first fully automatic data-scaling pipeline that integrates controllable 3D synthesis with consistent image editing and VLM-based filtering, and use it to construct KubriCount, the largest and most comprehensively annotated counting dataset to date, supporting both training and multi-grained evaluation. Systematic benchmarking reveals that both multimodal large language models and specialist counting models exhibit severe prompt-following failures under fine-grained distinctions. Motivated by these findings, we train HieraCount, a multi-grained counting model that jointly leverages text and visual exemplars as complementary target specifications. HieraCount substantially improves multi-grained counting accuracy and generalizes robustly to challenging real-world scenarios. The project page is available here: https://verg-avesta.github.io/KubriCount/.

Chang Liu, Haoning Wu, Weidi Xie

School of Artificial Intelligence, Shanghai Jiao Tong University, China · CMIC, Shanghai Jiao Tong University, China

Counting Fine-Grained Perception

Ask PDF