RS-OVC: Open-Vocabulary Counting for Remote-Sensing Data

Tamir Shor; George Leifman; Genady Beryozkin

cs.CVApr 9, 2026

RS-OVC: Open-Vocabulary Counting for Remote-Sensing Data

Authors: Tamir Shor, George Leifman, Genady Beryozkin

Organizations: Technion – Israel Institute of Technology · Google Research

Abstract

Object-Counting for remote-sensing (RS) imagery is attracting increasing research interest due to its crucial role in a wide and diverse set of applications. While several promising methods for RS object-counting have been proposed, existing methods focus on a closed, pre-defined set of object classes. This limitation necessitates costly re-annotation and model re-training to adapt current approaches for counting of novel objects that have not been seen during training, and severely inhibits their application in dynamic, real-world monitoring scenarios. To address this gap, in this work we propose RS-OVC - the first Open Vocabulary Counting (OVC) model for Remote-Sensing and aerial imagery. We show that our model is capable of accurate counting of novel object classes, that were unseen during training, based solely on textual and/or visual conditioning.

Figures & tables

Figure 1: Object confidence maps - illustrating spatial correspondence with aggregated textual (i.e. prompt) and visual (i.e. exemplar) conditioning. Visual exemplars are marked in red bounding-boxes on each image (red arrows highlight exemplars).

Figure 2: RS-OVC Pipeline - Our modifications from the original CountGD architecture are highlighted with an orange background. Image and text encoders remain frozen during optimization, other parameters are finetuned.

Figure 3: Curated dataset class-wise mean and standard-deviation (error bars) for object instance counts across images, for the training (top) and test (bottom) splits.

Method	Boat		House		Truck		Stadium		Airplane		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
CountGD	32.58	63.56	23.54	35.66	11.38	29.59	2.5	3.16	1.97	2.52	20.16	37.1
CountGD (RS-FT)	36.25	76.01	21.75	34.88	8.80	18.58	1.45	2.13	1.19	1.85	18.71	36.89
LAE	40.03	77.90	27.11	38.73	12.72	20.70	5.50	5.83	4.96	5.21	23.44	39.73
RS-OVC (RS-only)	32.47	72.56	18.56	31.13	10.21	17.32	4	4.69	6.5	7.0	17.07	34.01
RS-OVC (Ours)	7.5	15.44	15.52	26.87	8.74	24.2	5.52	6.37	3.08	3.55	12.42	24.72

Table 1: Object-Counting performance on NWPU-MOC test classes. CountGD indicates frozen off-the-shelf pretrained CountGD , RS-FT indicates original pretrained CountGD finetuned over RS data, RS-only indicates RS-OVC using only an RS pretrained image encoder, without feature fusion.

Method	Tractor		Boat		Truck		Airplane		Baseball-Field		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
CountGD	52.74	73.42	18.22	35.57	39.78	55.90	3.89	9.93	7.61	14.81	21.65	39.93
CountGD (RS-FT)	37.03	86.25	18.12	45.42	5.79	12.61	4.27	9.64	1.39	1.68	6.57	19.06
LAE	41.58	89.57	19.16	45.53	10.65	15.88	1.29	6.74	0.48	0.82	7.56	19.68
RS-OVC (RS-only)	37.55	86.66	18.14	45.23	6.04	12.72	4.39	9.75	1.58	1.88	6.74	19.08
RS-OVC (Ours)	26.84	47.98	12.79	27.45	5.54	10.41	4.21	10.22	3.00	3.06	5.81	13.58

Table 2: Object-Counting performance on FAIR-1M test classes.

Method	Storage Tank		Airplane		Bridge		Soccer Ball Field		Roundabout		Baseball Diamond		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
CountGD	57.71	112.68	33.21	55.19	14.74	25.14	56.39	92.20	21.29	43.15	52.09	87.43	36.46	71.97
CountGD (RS-FT)	51.47	116.75	35.60	77.28	11.54	20.41	10.57	28.00	4.18	5.36	8.00	17.40	28.14	72.72
LAE	57.18	121.67	18.86	59.40	14.80	23.91	7.21	9.01	6.82	8.06	4.67	6.69	23.59	67.94
RS-OVC (RS-only)	56.18	111.30	32.14	73.74	16.75	21.58	18.75	26.80	9.22	9.88	10.63	11.80	23.82	60.85
RS-OVC (Ours)	49.76	111.28	24.13	41.66	11.51	18.61	5.50	6.27	5.67	6.93	5.33	6.65	23.16	58.15

Table 3: Object-Counting performance on DOTA test classes.

Method	Storage Tank		Bridge		Airplane		Stadium		Chimney		Baseball Field		Windmill		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
CountGD	14.28	30.17	13.93	35.29	3.43	8.67	8.00	8.00	2.02	3.61	28.00	81.35	3.10	4.08	11.70	39.02
CountGD (RS-FT)	18.94	39.14	3.29	3.70	6.24	10.69	6.00	6.00	5.40	5.74	4.38	4.83	2.63	3.23	9.18	22.96
LAE	12.80	36.24	4.15	5.00	1.16	3.24	3.00	3.00	1.09	2.17	1.14	2.79	0.89	2.12	5.05	20.68
RS-OVC (RS-only)	20.46	42.42	7.76	9.50	6.65	11.42	4.00	4.00	2.98	3.24	2.60	2.98	2.01	2.53	8.37	22.71
RS-OVC (Ours)	19.95	42.20	2.50	2.94	6.23	11.10	7.00	7.00	2.86	3.08	3.05	3.37	2.69	3.21	9.15	24.62

Table 4: Object-Counting performance on DIOR test classes.

CountGD		CountGD (RS-FT)		LAE		RS-OVC (RS-only)		RS-OVC (Ours)
MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
13.52	17.80	12.38	18.39	15.39	21.84	10.94	15.91	10.51	15.30

Table 5: Object-Counting performance on the RSOC dataset building class.

Figure 4: Object confidence maps - for the standard setting and for joint visual–textual conditioning. Correct predictions require local object-level semantic understanding. For instance, in the top row our model correctly selects the single truck that has a red front (and not other trucks or red cars) given the textual prompt. Red markings indicate visual exemplars. Yellow markings upscale important image regions.

Figure 5: Object confidence maps - for the standard setting and for joint visual–textual conditioning. Correct predictions require global scene-level semantic understanding. The two rightmost columns introduce textual prompts that require relational reasoning. For instance, in the top row the model must identify absolute and relative spatial positions and orientations of baseball fields and trees to count correctly.

Figure 6: Object confidence maps - for the standard setting and for joint visual–textual conditioning that require basic reasoning. For instance, in the first row the model must attribute the bottom boat’s proximity to a pier to infer it is docking.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Boat		House		Truck		Stadium		Airplane		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
LAE	40.03	77.90	27.11	38.73	12.72	20.70	5.50	5.83	4.96	5.21	23.44	39.73
RS-OVC - zero-shot (Ours)	32.04	68.23	21.98	34.72	7.43	16.43	18.66	24.99	1.31	2.06	18.46	35.22
RS-OVC (Ours)	7.5	15.44	15.52	26.87	8.74	24.2	5.52	6.37	3.08	3.55	12.42	24.72

Appendix

Table 6: Zero-shot object-Counting performance on NWPU-MOC test classes, compared to 3-shot RS-OVC and LAE. Our method outperforms LAE in the zero-shot setup.

Method	Tractor		Boat		Truck		Airplane		Baseball-Field		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
LAE	41.58	89.57	19.16	45.53	10.65	15.88	1.29	6.74	0.48	0.82	7.56	19.68
RS-OVC-zero-shot (Ours)	36.84	85.36	18.40	41.97	6.01	11.88	6.08	11.07	5.90	7.49	7.51	18.31
RS-OVC (Ours)	26.84	47.98	12.79	27.45	5.54	10.41	4.21	10.22	3.00	3.06	5.81	13.58

Appendix

Table 7: Zero-shot performance on FAIR-1M test classes.

Method	Storage Tank		Airplane		Bridge		Soccer Ball Field		Roundabout		Baseball Diamond		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
LAE	57.18	121.67	18.86	59.40	14.80	23.91	7.21	9.01	6.82	8.06	4.67	6.69	23.59	67.94
RS-OVC-zero-shot (Ours)	51.62	113.30	35.81	77.73	12.64	21.04	14.54	26.44	8.16	21.37	5.62	9.63	27.69	67.79
RS-OVC (Ours)	49.76	111.28	24.13	41.66	11.51	18.61	5.50	6.27	5.67	6.93	5.33	6.65	23.16	58.15

Appendix

Table 8: Zero-shot performance on DOTA test classes.

Method	Storage Tank		Bridge		Airplane		Stadium		Chimney		Baseball Field		Windmill		Mean
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
LAE	12.80	36.24	4.15	5.00	1.16	3.24	3.00	3.00	1.09	2.17	1.14	2.79	0.89	2.12	5.05	20.68
RS-OVC-zero-shot (Ours)	21.29	42.98	2.59	4.00	7.25	12.48	0.00	0.00	1.48	2.05	1.85	2.97	2.15	3.20	9.50	25.20
RS-OVC (Ours)	19.95	42.20	2.50	2.94	6.23	11.10	7.00	7.00	2.86	3.08	3.05	3.37	2.69	3.21	9.15	24.62

Appendix

Table 9: Zero-shot performance on DIOR test classes.

LAE		RS-OVC-zero-shot (Ours)		RS-OVC (Ours)
MAE	RMSE	MAE	RMSE	MAE	RMSE
15.39	21.84	15.16	21.49	10.51	15.30

Appendix

Table 10: Zero-shot performance on the RSOC dataset building class.

Figure 7: MAE as a function of number of instances in the scene - aggregated across NWPU-MOC, FAIR-1M, DOTA, DIOR and RSOC-BUilding datasets. Data points are aggregated into logarithmically spaced bins to visualize performance across widely varying object counts, with shaded regions indicating standard deviation. We report results for RS-OVC (our method, in blue), LAE (OVD baseline, in yellow), off-the-shelf CountGD (without any RS adaptation, in red), CountGD finetuned over RS data (RSFT, black) and CountGD finetuned over RS data with a pre-trained RS-designated image encoder (RS-only, green).

Remote Sensing

Remote sensing uses satellite, aerial, and ground-based observations to measure and map Earth and other planetary surfaces.

See all 528 papers

OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation
Oct 21, 2026Yimin Wei, Hongruixuan Chen, Hao Zheng +5Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation
The University of Tokyo · RIKEN AIP · East China Normal University +1
COBICount: Separating Object and Background Responses for Remote Sensing Object Counting Without Training on Target Data
Sep 30, 2026Junjing Zheng, Zhiyi Zhou, Ningrui Yang +1Domain Generalization Object Counting
School of International Studies, Chongqing University of Posts and Telecommunications, Chongqing 400065, China · Brunel University London, Uxbridge UB8 3PH, U.K. · Department of Electronic and Electrical Engineering, College of Engineering, Design and Physical Sciences, Brunel University of London, Uxbridge UB8 3PH, U.K.
ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation
Mar 31, 2026Wenyang Chen, Zhanxuan Hu, Yaping Zhang +2Image Segmentation Remote Sensing Image Segmentation
Yunnan Normal University · Xi’an University of Posts & Telecommunications

Oct 21, 2026J/K move · Enter open · S save

OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation

Yimin Wei, Hongruixuan Chen, Hao Zheng, Hao Liu+4

The University of Tokyo · RIKEN AIP · East China Normal University+1

Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation Remote Sensing Benchmarks Multimodal Semantic Segmentation Fine-Grained Semantic Segmentation

Open paper AskPDF arXiv

In remote sensing, open-vocabulary segmentation is particularly important for flexible and scalable land-cover understanding across diverse regions, as it improves generalization to novel categories without requiring exhaustive pixel-level annotations. Despite its great promise, progress in this area remains largely constrained by existing datasets with limited category coverage and overly coarse taxonomies, and therefore struggles to support fine-grained semantic understanding required in real-world applications such as urban functional zoning, carbon accounting, and post-disaster assessment. To address these challenges, we present OpenEarth-OVS44, a multimodal benchmark with 290,587 expert-verified segments from 2,832 RGB images and 1,341 strictly registered RGB-SAR pairs at sub-meter spatial resolution, covering 40 cities across France, Japan, and the United States under a unified 44-class taxonomy organized into 9 base categories. Three evaluation tracks are provided: an RGB-only fine-grained benchmark, a multimodal track with diverse synthetic cloudy conditions for weather-robust evaluation, and a held-out cross-region track. Extensive experiments on representative open-vocabulary segmentation models demonstrate that training on OpenEarth-OVS44 substantially improves fine-grained segmentation performance over the strongest training-free baseline, with improvements of up to 7.1 mIoU points on unseen categories and 27.6 mIoU points on seen categories. The dataset and code will be released.

Abstract

Remote Sensing

Explore similar work

OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation

COBICount: Separating Object and Background Responses for Remote Sensing Object Counting Without Training on Target Data

ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation