ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation

Wenyang Chen; Zhanxuan Hu; Yaping Zhang; Hailong Ning; Yonghang Tai

cs.CVMar 31, 2026

ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation

Authors: Wenyang Chen, Zhanxuan Hu, Yaping Zhang, Hailong Ning, Yonghang Tai

Organizations: Yunnan Normal University · Xi’an University of Posts & Telecommunications

Abstract

Training-free open-vocabulary remote sensing segmentation (OVRSS), empowered by vision-language models, has emerged as a promising paradigm for achieving category-agnostic semantic understanding in remote sensing imagery. Existing approaches mainly focus on enhancing feature representations or mitigating modality discrepancies to improve patch-level prediction accuracy. However, such independent prediction schemes are fundamentally misaligned with the intrinsic characteristics of remote sensing data. In real-world applications, remote sensing scenes are typically large-scale and exhibit strong spatial as well as semantic correlations, making isolated patch-wise predictions insufficient for accurate segmentation. To address this limitation, we propose ConInfer, a context-aware inference framework for OVRSS that performs joint prediction across multiple spatial units while explicitly modeling their inter-unit semantic dependencies. By incorporating global contextual cues, our method significantly enhances segmentation consistency, robustness, and generalization in complex remote sensing environments. Extensive experiments on multiple benchmark datasets demonstrate that our approach consistently surpasses state-of-the-art per-pixel VLM-based baselines such as SegEarth-OV, achieving average improvements of 2.80% and 6.13% on open-vocabulary semantic segmentation and object extraction tasks, respectively. The implementation code is available at: https://github.com/Dog-Yang/ConInfer

Figures & tables

Figure 1: Compared with the direct predictions of MaskCLIP, the segmentation maps generated by clustering patch features from CLIP or DINOv3 exhibit superior spatial consistency.

Figure 2: Overview of the proposed Context-Aware Inference ( ConInfer ) framework. ConInfer performs joint, training-free inference by integrating semantic priors from CLIP with structural regularization from DINO. This formulation enables globally consistent open-vocabulary segmentation without any retraining.

Methods	OpenEarthMap	LoveDA	iSAID	Potsdam	Vaihingen	UAVid	UDD5	VDD	Average
	Satellite Images					UAV Images
CLIP [ 29 ]	9.16	9.59	6.53	11.62	7.84	8.94	7.91	11.27	9.11
MaskCLIP [ 58 ]	22.30	25.25	14.32	31.41	22.78	30.09	32.74	29.12	26.13
SCLIP [ 35 ]	23.85	24.60	15.08	31.84	21.96	29.69	33.25	34.26	26.82
GEM [ 4 ]	29.96	26.95	20.01	38.24	21.59	33.49	36.02	38.46	30.59
ClearCLIP [ 16 ]	29.94	28.06	16.62	40.20	20.55	34.35	38.07	40.24	31.00

Table 1: Quantitative comparison of open-vocabulary semantic segmentation performance (mIoU, %) across eight remote sensing datasets. These datasets comprise high-resolution satellite imagery (first five) and UAV-based urban scenes (latter three). Bold and underlined values indicate the best and second-best performance, respectively.

Figure 3: Examples from the UDD5 dataset illustrating cases where ConInfer encounters difficulties, including weak boundaries on small objects (vehicles) and category confusion (roof tiles).

Methods	OpenEarthMap	WHU Aerial	WBS-SI
MaskCLIP [ 58 ]	22.30	32.00	40.30
+SegEarth-OV	25.94	38.18	47.75
+ConInfer	29.28 $\uparrow$ 6.98	54.98 $\uparrow$ 22.98	52.43 $\uparrow$ 12.13
SCLIP [ 35 ]	23.85	37.89	42.91
+SegEarth-OV	27.98	44.61	48.51
+ConInfer	33.52 $\uparrow$ 9.67	59.37 $\uparrow$ 21.48	49.38 $\uparrow$ 6.47

Table 2: Proposed method is a plug-and-play module that can be seamlessly integrated into existing OVS methods.

Figure 4: Qualitative comparison of different OVSS methods on the OpenEarthMap (land cover mapping), VDD (UAV aerial scene), and WBI-SI (water body segmentation) datasets. GT denotes the ground truth.

Method	WHU Aerial	WHU Sat.II	Inria	xBD pre	CHN6-CUG	DeepGlobe	Massachusetts	SpaceNet	WBS-SI	Average
	Buiilding Extraction				Road Extraction				Flood Detection
CLIP [ 29 ]	6.62	1.45	15.12	8.88	3.74	1.13	3.50	3.20	8.98	5.85
MaskCLIP [ 58 ]	32.00	11.77	33.15	26.92	16.93	11.53	8.74	17.39	40.30	22.08
SCLIP [ 35 ]	37.89	7.82	33.92	29.42	13.86	7.28	5.56	8.13	42.91	20.75
GEM [ 4 ]	41.48	15.98	40.74	33.48	22.40	13.43	7.51	17.92	50.64	27.06
ClearCLIP [ 16 ]	43.59	13.72	37.08	29.83	25.20	12.86	7.18	17.02	46.55	25.89

Table 3: Quantitative comparison on the open-vocabulary single-class remote sensing extraction task, measured by Foreground IoU. Bold and underlined values indicate the best and second-best performance, respectively.

	Multi-class segmentation				Single-class extraction
Methods	OpenEarthMap	LoveDA	iSAID	Vaihingen	WHU Sat.II	Inria	xBD pre	WBS-SI
Baseline	32.46	31.58	18.17	19.94	15.17	37.26	31.40	52.57
Context ${}_{\text{CLIP}}$	35.12 $\uparrow$ 2.66	33.21 $\uparrow$ 1.63	19.05 $\uparrow$ 0.88	25.10 $\uparrow$ 5.16	17.38 $\uparrow$ 2.21	38.27 $\uparrow$ 1.01	34.98 $\uparrow$ 3.58	54.48 $\uparrow$ 1.91
No joint-learning	36.23 $\uparrow$ 3.77	36.27 $\uparrow$ 4.69	18.86 $\uparrow$ 0.69	32.80 $\uparrow$ 12.86	28.56 $\uparrow$ 13.39	54.47 $\uparrow$ 17.21	36.30 $\uparrow$ 4.90	56.78 $\uparrow$ 4.21
ConInfer	41.95 $\uparrow$ 9.49	39.33 $\uparrow$ 7.75	20.08 $\uparrow$ 1.91	31.37 $\uparrow$ 11.43	39.35 $\uparrow$ 24.18	55.65 $\uparrow$ 18.39	41.34 $\uparrow$ 9.94	61.38 $\uparrow$ 8.81

Table 4: Ablation study of ConInfer . We analyze the impact of alternative context information extraction (via CLIP features) and the removal of joint latent distribution optimization (applying prior knowledge post-EM).

Figure 5: Convergence analysis was conducted on 8 datasets. During the iterative process, the

loss

rapidly converged and subsequently remained stable.

Dataset	Class Name
OpenEarthMap	background, {bareland, barren}, grass, pavement, road, {tree, forest}, {water, river}, cropland, {building, roof, house}
LoveDA	background, roof, road, river, barren, forest, agricultural
iSAID	{background, grass, agriculture, water, forest, road, building, barren, urban}, boat, industrial cylindrical storage tank, baseball diamond, tennis court, basketball court, ground track field, bridge, large vehicle, small vehicle, helicopter, swimming pool, roundabout, soccer ball field, plane, harbor
Potsdam	{road, parking lot}, building, low vegetation, tree, car, {clutter, background}
Vaihingen	impervious surface, building, low vegetation, tree, car, clutter
UAVid	background, building, road, car, tree, vegetation, human

Table 5: The prompt class name of the evaluation datasets. indicates multiple prompt vocabularies for one class.

Method	OpenEarthMap	LoveDA	iSAID	Potsdam	Vaihingen	UAVid_img	UDD5	VDD	Average
Baseline (w/o $\mathrm{Attn}_{M}$ )	31.87	30.09	18.43	43.15	21.60	37.66	42.36	42.20	33.42
Baseline (w/ $\mathrm{Attn}_{M}$ )	32.46	31.58	18.17	44.49	19.94	37.20	42.17	42.54	33.57
Ours (w/o $\mathrm{Attn}_{M}$ )	41.35	37.89	19.29	49.20	31.36	45.92	46.54	47.95	39.94
Ours (w/ $\mathrm{Attn}_{M}$ )	41.95	39.33	20.08	49.99	31.37	46.40	46.86	50.29	40.78

Table 6: Comparisons of mIoU (448) on different datasets.

Method	WHU_Aerial	WHU_Sat.II	Inria	xBD_pre	CHN6-CUG	DeepGlobe	Massachusetts	SpaceNet	WBS-SI	Average
Baseline (w/o $\mathrm{Attn}_{M}$ )	45.09	17.00	40.52	34.27	27.30	14.13	8.04	19.07	48.98	28.27
Baseline (w/ $\mathrm{Attn}_{M}$ )	45.76	15.17	37.26	31.40	29.12	14.68	8.42	19.83	52.57	28.25
Ours (w/o $\mathrm{Attn}_{M}$ )	55.07	31.85	53.66	41.01	35.67	18.02	11.84	23.47	59.40	36.67
Ours (w/ $\mathrm{Attn}_{M}$ )	58.54	39.35	55.65	41.34	40.00	19.85	12.16	24.02	61.38	39.14

Table 7: Comparison of Foreground IoU (448) on different datasets.

Setting	OpenEarthMap	LoveDA	iSAID	Potsdam	Vaihingen	UAVid_img	UDD5	VDD
$\lambda=0.0$	41.35	37.89	19.29	49.20	31.36	45.92	46.54	47.95
$\lambda=0.2$	41.82	38.19	19.73	49.78	31.15	46.12	46.71	48.23
$\lambda=0.4$	42.09	38.90	19.97	49.93	30.62	46.14	46.82	49.31
$\lambda=0.6$	41.95	39.33	20.08	49.99	29.73	45.60	46.86	50.29
$\lambda=0.8$	40.06	39.61	19.86	49.83	27.74	44.59	46.93	48.43
$\lambda=1.0$	39.92	39.31	19.35	49.35	25.92	43.43	44.58	46.39

Table 8: Performance comparison with different

\lambda

values across multiple datasets.

Setting	WHU_Aerial	WHU_Sat.II	Inria	xBD_pre	CHN6-CUG	DeepGlobe	Massachusetts	SpaceNet	WBS-SI
$\lambda=0.0$	55.07	31.85	53.66	41.01	35.67	18.02	11.84	23.47	59.40
$\lambda=0.2$	56.23	34.49	54.58	41.62	36.28	18.63	12.16	23.67	59.84
$\lambda=0.4$	57.47	36.92	55.45	41.86	36.33	19.37	12.16	23.81	60.65
$\lambda=0.6$	58.54	39.35	55.65	41.82	40.00	19.85	12.16	24.02	61.38
$\lambda=0.8$	58.97	41.84	55.36	41.13	37.00	20.07	12.13	24.14	61.79
$\lambda=1.0$	57.41	41.64	54.22	39.28	33.73	19.89	12.14	24.34	62.25

Table 9: Performance comparison on multiple datasets with varying

\lambda

Figure 6: Qualitative comparison of different OVSS methods on the OpenEarthMap (land cover mapping) datasets.

Figure 7: Qualitative comparison of different OVSS methods on the VDD (UAV aerial scene) datasets.

Figure 8: Qualitative comparison of different OVSS methods on the WBI-SI (water body segmentation) datasets.

Image Segmentation

Image segmentation assigns pixels or voxels to objects and regions, supporting measurement, scene understanding, and decisions across image domains.

See all 390 papers

SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery
Sep 19, 2026Changhao Zhao, Linglin Zeng, Hai LiuOpen-Vocabulary Semantic Segmentation Remote Sensing Image Segmentation
College of Resources and Environment, Huazhong Agricultural University, Wuhan 430070, China
Enabling Training-Free Text-Based Remote Sensing Segmentation
Feb 19, 2026Jose Sosa, Danila Rukhovich, Anis Kacem +1Text-Guided Image Segmentation Remote Sensing Image Segmentation
SnT, University of Luxembourg
Few-Shot Open-Vocabulary Remote Sensing Segmentation via Textual Inversion
Jul 28, 2026Junhyuk Heo, Junghwan ParkRemote Sensing Image Segmentation Few-Shot Semantic Segmentation
TelePIX, Seoul, Republic of Korea

Sep 19, 2026J/K move · Enter open · S save

SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery

Changhao Zhao, Linglin Zeng, Hai Liu

College of Resources and Environment, Huazhong Agricultural University, Wuhan 430070, China

Open-Vocabulary Semantic Segmentation Remote Sensing Image Segmentation Semantic Segmentation

Open paper AskPDF arXiv

Open-vocabulary semantic segmentation (OVS) of remote sensing imagery is a challenging pixel-level task requiring strong generalization and adaptation to the spatial characteristics of remote sensing data. Although existing vision-language foundation models perform well in general domains, their image-level classification design weakens the spatial priors needed for high-resolution remote sensing segmentation: structural spatial relations are degraded during deep feature transformation, and fine-grained spatial details are lost during downsampling. To address these complementary deficiencies, we propose SatOV, a training-free framework for open-vocabulary remote sensing segmentation that restores spatial priors at two stages of the representation pipeline. Specifically, Residual QQ Attention (ResQQ) extracts Query-Key self-attention from an intermediate CLIP layer and fuses it with final-layer Query-Query attention via a residual combination, restoring structural spatial priors suppressed by the final-layer representation. Spatially Modulated Upsampling (SatUp) uses the original high-resolution RGB image as spatial guidance, combining spatial feature modulation with guided cross-attention to reconstruct pixel-level textures and boundaries. Extensive experiments on DOTA, UDD, LoveDA, and Vaihingen show that SatOV consistently improves training-free OVS and achieves competitive quantitative and qualitative results against state-of-the-art methods. These results validate the effectiveness of restoring spatial priors at both the representation and spatial-resolution stages for remote sensing open-vocabulary segmentation.

Abstract

Image Segmentation

Explore similar work

SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery

Enabling Training-Free Text-Based Remote Sensing Segmentation

Few-Shot Open-Vocabulary Remote Sensing Segmentation via Textual Inversion