Many scientific and policy decisions require estimating the total number of target objects across a large image collection. Exhaustive human labeling is often infeasible, while raw model predictions can be biased, especially under domain shift. Recent human--AI estimation methods combine imperfect model predictions with limited human verification to obtain reliable collection-level estimates. However, existing work provides little guidance on how practitioners should choose among models, estimators, and adaptation strategies for a new deployment. We study these choices within a common human-in-the-loop counting framework on CLC-12, a benchmark of 12 tasks spanning biomedical microscopy, ground-level and aerial wildlife surveys, and overhead imagery of urban structures. Our results show that modern multimodal models provide a strong, training-free starting point for many tasks, while fine-tuned detectors become more advantageous at larger labeling budgets and across repeated deployments. Estimation efficiency improves further when model predictions are used to prioritize which units humans label, rather than sampling uniformly. We also find that each model's ranking quality can be estimated with a small pilot set, yielding practical guidance for selecting strategies on new counting tasks. In a field deployment on aerial pronghorn surveys, a task-specific adaptation of these methods meets the survey's ± 5% tolerance after reviewing a median of only 3.2% of collected images.
Figures & tables
Figure 1: Samples from the 12 counting tasks. Each in-image label identifies the task (dataset/class), tile index, and exact ground-truth count. Red points mark each object.
Estimator
Sampler
Model
Monte Carlo
uniform
—
{model-only, diff, ratio, reg}
uniform
{fine-tuned, frozen}
AIS, spotAIS
qt
fine-tuned
DIS, spotDIS
qt
frozen
Table 1: Estimator–sampler–model configurations evaluated. Uniform-sampling estimators are run with both fine-tuned and frozen models, while the IS variants are model-specific, with AIS paired to fine-tuned detectors and DIS to frozen models. Here diff, ratio, and reg denote the incremental difference, ratio, and regression estimators, defined in § 3.3 .
Tier (% ∅ )
Dataset
Class
% ∅
μ
N
full ( <5% )
Malaria
trophozoite
0.0
2.42
655
Malaria
difficult
0.0
1.29
347
Malaria
red blood cell
2.1
49.19
1,688
partial ( 5 – 60% )
Birds
bird
24.4
14.36
1,302
DOTA
tennis court
37.5
4.63
689
DOTA
harbor
52.1
3.90
2,084
Table 2: The 12 tasks of CLC-12. % ∅ is the empty fraction, μ the average object count per unit, and N the total number of units. Full per-task statistics are in Appendix E .
Figure 2: Top: mean absolute fractional error over 1,000 trajectories versus labeling budget on Birds/bird , one curve per estimator, for fine-tuned Faster R-CNN (left) and frozen InternVL3.5-8B (right). Bottom: each family’s best case, averaged over 12 tasks; at every task and budget the family is represented by its lowest-error {model, estimator} pair, so a curve is a per-budget optimum, not one fixed model. The fine-tuned detector curve starts at 3.7% labeled (warmup).
Figure 3: Overview of the main results. A case is one {model, task, budget} triple; its winner is the estimator (or, when comparing models, the model) with the lowest error on it. Left: error relative to model-free Monte Carlo, one panel per occupancy tier (log scale; below 1× the model helps). Every model is scored with its best estimator; the solid and dashed lines are the family’s best and worst model, as medians over the tier’s 3 tasks. The bar beside each panel is the share (%) of the tier’s 390 cases ( 3 tasks ×13 models ×10 budgets, t=5%,…,50% ) won by each estimator. Top right: the same shares at each budget t , over the 156 cases ( 13 models ×12 tasks) per t . Bottom right: share of the 12 tasks whose winning model belongs to each family, at each budget; the hatched band is the one task with no unique winner.
Figure 4: Number of the 120 (task, budget) settings on which each model, with its best estimator, has the lowest error of all 13 . The 10 settings with no unique winner (see text) count for no model. Solid: won at ≤15% labeled; faded: above that. Models with no wins are omitted.
Figure 5: Left: a pronghorn (by Alan D. Wilson, CC BY-SA 3.0) and the Iron Springs survey area. Vertical lines are flight transects; each dot is the ground footprint of one survey image (blue: empty unit; green: empty but used for camera pose calibration; red: populated unit). Center: estimated pronghorn count from the calibrated model-only baseline (gray) and two AIS runs, with the ground truth. Whiskers are 95% CI. Right: the two AIS trajectories and their 95% CIs (shaded) versus % of imagery reviewed. Both reached ±5% uncertainty after reviewing 3.2% of the imagery, the median over all 100 runs ( Fig. A7 ).
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Coverage
Half-width / Ftrue
Estimator
full
part
spar
rare
full
part
spar
rare
Monte Carlo
0.94
0.94
0.93
0.86
0.07
0.15
0.20
0.49
difference
0.89
0.93
0.88
0.88
0.09
0.12
0.18
0.47
ratio
0.87
0.92
0.88
0.86
0.12
0.12
0.16
0.43
regression
0.90
0.92
0.88
0.86
0.07
0.11
0.16
0.41
DIS
0.92
0.92
0.93
0.86
0.08
0.11
0.09
0.19
Appendix
Table A1: Behavior of 95% confidence intervals by occupancy tier. Coverage is the fraction of the 1,000 trials of a {model, task, budget} case (a {task, budget} case for Monte Carlo, which uses no model) whose interval contains Ftrue , averaged over the cases in a tier. Half-width is 1.96(Var1:tsimp)1/2/Ftrue (with Var1:tMC for Monte Carlo), summarized by its median within a case and again across cases. The excl. spot row pools all cases of the six estimators other than spotDIS and spotAIS.
Coverage
Half-width / Ftrue
Estimator
5%
10%
20%
50%
5%
10%
20%
50%
Monte Carlo
0.86
0.90
0.92
0.94
0.38
0.28
0.19
0.10
difference
0.82
0.87
0.90
0.92
0.37
0.27
0.18
0.09
ratio
0.81
0.86
0.89
0.91
0.41
0.31
0.19
0.10
regression
0.81
0.86
0.89
0.92
0.36
0.26
0.17
0.08
DIS
0.87
0.89
0.91
0.92
0.31
0.21
0.14
0.07
Appendix
Table A2: Behavior of 95% confidence intervals by labeling budget, at four of the ten evaluated budgets ( 5% to 50% in steps of 5% ). All rows are defined and summarized over cases as in Table A1 , but pooled across tiers.
Tier
Dataset
Class
% ∅
μ
CV
N
Ftrue
img. pool
img. total
tile px
full ( <5% empty)
Malaria
red blood cell
2.1
49.19
0.77
1,688
83,034
1,328
1,328
1,168
Malaria
trophozoite
0.0
2.42
0.72
655
1,584
655
655
1,488
Malaria
difficult
0.0
1.29
0.44
347
446
347
347
1,424
Malaria
schizont
0.0
1.14
0.35
166
190
166
166
1,536
Malaria
gametocyte
0.0
1.06
0.23
147
156
147
147
1,408
Malaria
leukocyte
0.0
1.10
0.27
94
103
94
94
1,248
Appendix
Table A3: All candidate classes, grouped by occupancy tier and ordered by decreasing pool total Ftrue within each tier. Selected benchmark classes are shown in bold . μ is the average target-object count per unit and CV its coefficient of variation across units (dispersion); N is the total number of estimation units; Ftrue is its exact target-object count within the N units; img. pool and img. total are the numbers of included and available source images, respectively; and tile px is the class-specific tile side length. Pools may slightly exceed the 5,000 -unit target because the final admitted image is retained in full.
Figure A1: Landing positions ur at the 30% budget, underlying the Shift column of Table A4 . Each point is one genuine trajectory, placed relative to the 199 replays of its own cached model path in replay standard deviations ( 0 is the replay mean). The shaded band spans ±2 standard deviations for an individual trajectory, not the ±0.28 interval for the mean shift; points outside it are red. The model-assisted estimators resemble the exact Monte Carlo control, whereas the model-only baseline has a pronounced negative tail.
MobileNetV3
ResNet-50
Estimator
Cov.
Shift
Cov.
Shift
faithful replay, in expectation
.90
0
.90
0
model-only
.61
−1.47
.64
−0.96
Monte Carlo (exact, measured)
.91
+0.10
.91
+0.10
difference
.86
+0.23
.89
+0.04
ratio
.86
+0.17
.88
+0.10
Appendix
Table A4: Replay validation for the two fine-tuned detectors. Cov. is held-out coverage and Shift is landing shift, both defined in Appendix G . The first row gives their values under faithful replay; shifts within ±0.28 are consistent with it. Monte Carlo ignores the detector, so its entries are the same under both.
Figure A2: Total count predicted by each of the eight MLLMs as a ratio to the true total, on every task (rows, ordered full to rare; log axis, 1= exact); circles are open-source MLLMs and diamonds are proprietary ones, each tinted by model.
MLLM + OVD ( n=11 )
MLLM ( n=8 )
10% labeled
30% labeled
10% labeled
30% labeled
Proxy
ρ
beats
win
ratio
ρ
beats
win
ratio
ρ
beats
win
ratio
ρ
beats
win
ratio
H(p,q)
0.85
0.84
0.53
1.00
0.81
0.84
0.59
1.00
0.83
0.82
0.48
1.00
0.76
0.82
0.54
1.00
H(p,qraw)
0.66
0.79
0.47
1.02
0.62
0.78
0.46
1.06
0.65
0.77
0.42
1.02
0.56
0.75
0.41
1.06
RMSE
0.44
0.72
0.39
1.15
0.40
0.72
0.39
1.18
0.53
0.76
0.36
1.11
0.47
0.74
0.35
1.12
MAE
0.41
0.70
0.37
1.15
0.36
0.70
0.37
1.22
0.52
0.75
0.33
1.13
0.44
0.72
0.32
1.18
Appendix
Table A5: Reliability of the five pilot proxies, and of the chi-square alternative (last row), for choosing a frozen model to run with DIS, on two candidate sets of n models at two labeling budgets. The metrics are defined in Appendix H ; the ideal value of each is 1 . q is the DIS proposal, which floors each prediction at 1 , and qraw is the normalized prediction without the floor. Bold marks the best of the five proxies in each column; the chi-square row is excluded from bolding.
Figure A3: Total-count gap on the full pool vs. mean AUC of DIS (area under the fractional-error curve up to 50% labeled, averaged over 100 trials). Each point is one frozen model on one task; lower is better on both axes.
Figure A4: Total-count gap on the full pool vs. fractional error of DIS at 10% labeled. Each point is one frozen model on one task; lower is better on both axes.
Figure A5: Hellinger distance H(p,qraw) on the full pool vs. mean AUC of DIS (area under the fractional-error curve up to 50% labeled, averaged over 100 trials). Each point is one frozen model on one task; lower is better on both axes.
Figure A6: Hellinger distance H(p,qraw) on the full pool vs. fractional error of DIS at 10% labeled. Each point is one frozen model on one task; lower is better on both axes.
Model
$/1,000 units
Units/$1
Normalized
Faster R-CNN
MobileNetV3
$0.017
59,507
581.3 ×
ResNet-50
$0.017
57,444
561.1 ×
Open-vocabulary detectors
Grounding DINO
$0.126
7,913
77.3 ×
Florence-2
$0.410
2,440
23.8 ×
Appendix
Table A6: Per-unit inference cost, ordered from least to most expensive within each family. Normalized is units per dollar relative to Gemini 3.1 Pro. Locally run models assume one A100 at 3perGPU−hour;proprietaryMLLMsusetheAPIcostperunitmeasuredonthe60$ -unit sample (Table A7 ).
Model
Origin
Size
GPU-h/unit
API cost/unit
Extraction method
Qwen3-VL-8B
HuggingFace
8B
7.94×10−5
—
Two-pass in-model count → choice
InternVL3.5-8B
HuggingFace
8B
7.00×10−5
—
Two-pass in-model count → choice
Molmo-7B-D-0924
HuggingFace
7B
8.00×10−5
—
Two-pass in-model count → choice
Aya Vision-8B
HuggingFace
8B
8.58×10−5
—
Two-pass in-model count → choice
Molmo2-8B
HuggingFace
8B
8.56×10−5
—
Two-pass in-model count → choice
GPT-5.2
OpenRouter
—
—
$0.004645
“Final answer:” line + GPT-5.2 extractor pass
Appendix
Table A7: MLLM inference time and cost. Open-source models ran on a single A100 over all 33,605 units of the benchmark, and GPU-hours per unit are seconds per unit divided by 3,600 . Proprietary models were queried via OpenRouter on a 60 -unit sample, and API cost per unit is the total API cost divided by the number of sampled units.
Prompt
Placeholders and details
Template
Open-source count (stage 1)
The placeholder {object_name} is replaced with the task-specific object name (e.g. “birds”, “storage tanks”).
How many {object_name} are in this image? Only output the number and no other text.
Open-source count extraction (stage 2, in-model second pass)
The placeholder {nlg} is the free-form text generated in stage 1, and {max_choice} is the upper bound of the structured choice list (default 300). The vLLM batched variant omits the “from 0 to {max_choice}” clause.
What number was this response trying to say? Response: {nlg} Answer only with one integer from 0 to {max_choice}.
Proprietary count (stage 1)
The placeholder {object_name} is replaced with the task-specific object name.
How many {object_name} are in this image? Think briefly from visible evidence, then finish with a line exactly like Final answer: <number>.
The placeholder {response} is the full reasoning text produced in stage 1.
Extract the count from the response below. Response: {response} Use the number on the final answer line when present. If the final answer is a range or alternative like ’2 or 3’, output the first number. Answer only with one number. If the final answer has no digit, answer unknown.
The placeholder {answer_text} is the extracted count (or “unknown”) and {response} is the stage-1 reasoning text.
Does the response explicitly contain this extracted count as its answer? Extracted count: {answer_text} Response: {response} Answer yes if the exact count is plainly present in the response. Answer no if the response does not plainly contain that count.
Appendix
Table A8: Prompts used for MLLM counting and response processing. Open-source models use a count prompt followed by an in-model count-extraction pass; proprietary models use a count prompt followed by a GPT-5.2 extractor pass and, optionally, a GPT-5.2 verifier pass.
Figure A7: Trajectories of 100 independent AIS (active measurement) runs on the pronghorn survey, each with a different random seed. Every run reached the ±5% estimated-uncertainty stopping condition: 96 after reviewing 3.0–3.4% of the imagery (median 3.2% over all 100 runs), and four after 27.2–28.5%. Of the 100 final 95% confidence intervals, 93 contain the true count; the seven that do not, including all four slow runs, are shown in green.