Open-Vocabulary Domain Generalization in Urban-Scene Segmentation

Dong Zhao; Qi Zang; Nan Pu; Wenjing Li; Nicu Sebe; Zhun Zhong

cs.CVFeb 21, 2026

Open-Vocabulary Domain Generalization in Urban-Scene Segmentation

Authors: Dong Zhao, Qi Zang, Nan Pu, Wenjing Li, Nicu Sebe, Zhun Zhong

Organizations: Department of Information Engineering and Computer Science, University of Trento, Italy · School of Computer Science and Information Engineering, Hefei University of Technology, China

Abstract

Domain Generalization in Semantic Segmentation (DG-SS) aims to enable segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that mitigates domain-induced distortions and produces more consistent text-image correlations under distribution changes. Extensive experiments on our constructed benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.

Figures & tables

Figure 1: Concept of Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS). (a) OVDG-SS aims to generalize across both unseen domains (e.g., lighting, weather, geographic region, construction context) and unseen classes (e.g., barrier, cone, railway), moving beyond conventional domain generalization (DG-SS) and open-vocabulary segmentation (OV-SS). (b) Comparison of segmentation results. Traditional DG-SS fails to recognize unseen categories, and OV-SS struggles under domain shifts, while our OVDG-SS effectively handles both unseen domains and classes simultaneously.

OV-SS Method	Training Data	Seen	Num.	Shared classes		Performance
OV-SS Method	Training Data	class	Num.	Dv-19	Dv-58	Dv-19	Dv-58
CAT-Seg [ 7 ]	COCO-Stuff [ 27 ]	171	118K	17	34	31.6	32.4
CAT-Seg	SYNTHIA [ 43 ]	7	9.4K	7	7	43.6	45.3
CAT-Seg	GTA [ 41 ]	7	25K	7	7	47.5	48.2
CAT-Seg	Cityscapes [ 8 ]	7	3K	7	7	49.3	50.0
MaskAdapter [ 24 ]	COCO-Stuff	171	118K	17	34	30.1	29.6

Table 1: Effect of training domains on training-based OV-SS performance. Colored rows indicate models trained on driving datasets. Dv-19 and Dv-58 refer to cross-domain driving datasets with 19 and 58 categories.

Figure 2: Efficiency and performance comparison on OVDG-SS tasks using EVA02 ViT-B/16 as backbone. FPS is tested on images with a short edge of 480 and a long edge of 960. Our method achieves the best trade-off among generalization ability, speed, and parameter efficiency.

Figure 3: Overview of the proposed S 2 -Corr. The upper part shows the CLIP-based encoding and correlation aggregation pipeline. The lower part illustrates our S 2 -Corr, which refines text–image correlations using a specially designed chunked State-Space Models (SSM) aggregation scheme.

Figure 4: Effect of domain shift on text–image correlations of the class “ sky ” from EVA02 model [ 11 ] . Color map ranges from blue (low correlation) to red (high correlation) . As the domain shift increases from left to right, the initial correlation maps become progressively noisier, with incorrect activations spreading across irrelevant regions.

Figure 5: Comparison of scanning strategies in state-space correlation aggregation between VMamba [ 30 ] and ours. Our method introduces a learnable geometric decay ( $*\gamma$ ) to suppress long-range noise within each chunk and uses a snake-shaped scanning strategy that preserves spatial continuity by passing end states between adjacent chunks.

Method	Backbone	Training Data	Dv-19			Ave.	Dv-58				Ave.
Method	Backbone	Training Data	ACDC-19	BDD-19	Mapi-19	Ave.	ACDC-41	BDD-41	Mapi-30	RW-10	Ave.
CLIP-DINOiser(ECCV2024) [ 55 ]	EVA02+DINOV2 ViT-L	Training-Free	25.3	28.5	27.7	27.2	31.9	34.0	26.9	21.6	28.6
ClearCLIP(ECCV2024) [ 19 ]	EVA02 ViT-L	Training-Free	26.7	28.3	27.0	27.3	33.5	34.0	26.6	21.6	28.9
ProxyCLIP (ECCV’24) [ 20 ]	EVA02+SAM ViT-L	Training-Free	31.0	38.2	40.0	36.4	38.5	52.2	33.5	27.2	37.9
CAT-Seg (CVPR’24) [ 7 ]	EVA02 ViT-L/14	COCO	30.3	31.9	36.1	32.8	33.2	42.8	28.6	27.6	33.1
DeCLIP (CVPR’25) [ 49 ]	EVA02+DINOV2 ViT-L	COCO	32.1	31.8	38.1	34.0	34.2	47.0	32.0	28.3	35.4

Table 2: Comparison of different OV-SS methods across different backbones under the Real-to-Real OVDG-SS setting. Dv-19 groups ACDC-19, BDD-19, and Mapi-19, while Dv-58 groups ACDC-41, BDD-41, Mapi-30, and RW-10.

Method	Backbone	Training Data	Dv-19				Ave.	Dv-58				Ave.
Method	Backbone	Training Data	CS-19	ACDC-19	BDD-19	Mapi-19	Ave.	ACDC-41	BDD-41	Mapi-30	RW-10	Ave.
OVSeg (CVPR’23) [ 26 ]	EVA02 ViT-B/16	GTA-7	41.2	34.9	42.5	45.1	40.9	45.1	52.6	35.5	32.9	41.5
SAN (CVPR’23) [ 56 ]		GTA-7	43.3	36.7	42.0	48.1	42.5	47.7	49.0	37.8	30.3	41.2
CAT-Seg (CVPR’24) [ 7 ]		GTA-7	43.7	37.6	45.4	48.8	43.9	51.2	56.2	39.3	35.7	45.6
MAFT+(ECCV’24) [ 16 ]		GTA-7	45.7	38.5	43.7	50.2	44.5	49.1	48.3	39.9	31.8	42.3
ESC-Net (CVPR’25) [ 21 ]		GTA-7	42.2	35.9	44.6	48.1	42.7	49.4	52.4	37.9	33.3	43.3

Table 3: Comparison of different OV-SS methods across various backbones under the synthetic-to-real OVDG-SS setting.

Note	Methods	ViT-B/16		ViT-L/14		Ave.
Note	Methods	Dv-19	Dv-58	Dv-19	Dv-58	Ave.
Base	Cross-attention	43.5	43.5	49.3	50.0	46.6
#1	Seletive SSM	45.6	44.1	50.7	50.5	47.7
#2	+Modulation	47.6	45.3	52.1	50.9	49.0
#3	+Geometric decay	48.3	46.4	53.2	51.8	49.9
#4	+Chunk	49.6	47.3	55.3	52.7	51.2

Table 4: Ablation study of the proposed method under different design components on CS7 → Dv-19 and Dv-58 across backbones. Method #1 replaces cross-attention by selective SSM. Methods of #2–#5 are progressively added on top of #1.

Figure 6: Comparison of text–image correlation aggregation on seen and unseen classes from unseen domains. Our method yields clearer and more localized text–image correlations than CAT-Seg [ 7 ] and Corelation Map (initially obtained by Eq. 1 ), improving both seen and unseen predictions under domain shifts.

Figure 7: Qualitative comparisons on unseen domains under the OVDG-SS setting, with DG-SS method [ 61 ] and OV-SS method [ 24 ] .

Model	FPS	FPS	FPS	GPU	Training
Model	vocab19	vocab58	vocab150	Mem. (GB)	Time (min)
CAT-Seg [ 7 ]	15.4	10.6	5.7	13.8	180
ESC-Net [ 21 ]	15.0	9.9	5.1	15.7	220
Seletive SSM	12.3	9.4	6.4	16.8	240
S 2 -Corr (Ours)	26.1	22.2	18.3	9.2	140

Table 5: Inference and training efficiency of competing methods using a ViT-B/16 backbone. FPS is evaluated under different test vocabulary sizes.

dataset	From	Size	Resolution	Classes
CS-7	Cityscapes Train [ 8 ]	2,975	2048×1024	Rd, Sw, Bldg, Veg, Sky, Per, Car
GTA-7	GTA Train [ 41 ]	24999	2048×1024	Rd, Sw, Bldg, Veg, Sky, Per, Car
ACDC-19	ACDC Train [ 44 ]	1,600	1920×1080	Rd, Sw, Bldg, Wal, Fen, Pol, TL, TS
BDD-19	BDD100K Validation [ 60 ]	1000	1280×720	Veg, Ter, Sky, Per, Rid, Car, Trk, Bus, Trn, Mot, Bic
Mapi-19	Mapillary Validation [ 60 ]	2000	2k-4k
ACDC-41	SD 2.1 + ACDC Train	1000	1920×1080	Rd, Sw, Bldg, Wal, Fen, Pol, TL, TS, Veg, Ter, Sky, Per,

Table 6: Summary of datasets used in the OVDG-SS benchmark.

Figure 8: Impact of hyperparameters: chunk size, prior decay, text prompt templates, and training resolution

VLM	ACDC-19	BDD-19	Mapi-19	ACDC-41	BDD-41	Mapi-30	RW-10	Ave.
EVA-02-CLIP-L/14 [ 11 ]	54.3	53.1	60	62	61.7	47.4	41.9	44.7
SigLIP-ViT-L/16 [ 48 ]	52.7	54.1	59.2	60.5	60.4	48.3	42.5	54.0
CLIP-ViT-L/14 [ 39 ]	53.7	52.8	59.0	61.6	60.9	46.7	40.9	53.7
SelfCLIP-ViT-L/14 [ 52 ]	56.3	53.6	60.9	63.8	63.5	48.1	41.9	59.6

Table 7: Comparison of different Vision-Language Models (VLMs) under the OVDG-SS setting across multiple datasets. The best results are underlined.

Variant	Row	Col	mIoU Dv-19	mIoU Dv-58	Training Time (Min)
Row-only (ours)	$\checkmark$	$\times$	50.3	47.9	140
Col	$\times$	$\checkmark$	48.6	45.7	140
Row+Col	$\checkmark$	$\checkmark$	50.4	48.1	165

Table 8: Ablation on different scanning strategies used in the spatial aggregation module.

Training Data: GTA-19
	Type	Pretrain	CS-19	BDD-19	Mapi-19	Ave.	Mapi-30	RW-10	ACDC-41	BDD-41	Ave.
Rein (CVPR’24) [ 50 ]	DG	EVA02-L	65.3	60.5	64.9	63.6	-	-	-	-	-
tqdm (ECCV’24) [ 36 ]	DG	EVA02-L	68.9	59.2	70.1	66.1	-	-	-	-	-
SoMA (CVPR’25) [ 61 ]	DG	EVA02-L	68.1	60.8	68.3	65.7	-	-	-	-	-
MaskAdapter (CVPR’25) [ 24 ]	OVSS	EVA02-L	61.5	55.5	65.7	60.9	48.5	41.0	63.3	61.8	53.7
MAFT+(ECCV24) [ 16 ]	OVSS	EVA02-L	61.2	55.0	65.0	61.2	48.2	40.1	63.9	62.4	53.7

Table 9: Extended results under a larger 19-class training vocabulary. The main paper reports OVDG performance using a compact 7-class training set, this table presents results obtained with the full 19-class vocabulary (GTA-19).

Figure 9: More comparison of initial text–image correlation aggregation on seen and unseen classes from unseen domains.

Methods	FS Static		FS Lost&Found		SMIYC-Anomaly		SMIYC-Obstacle		RoadAnomaly
Methods	IoU	mean F1	IoU	mean F1	IoU	mean F1	IoU	mean F1	IoU	mean F1
Synboost [ 10 ]	32.8	25.7	18.4	10.9	42.0	39.5	14.0	9.0	27.2	29.3
PEBAL [ 47 ]	26.9	13.3	6.4	2.6	42.4	35.1	6.7	1.1	33.8	23.9
RPL+CoroCL [ 31 ]	36.5	13.2	15.8	3.9	68.8	31.6	28.7	5.7	51.0	24.6
S2M [ 70 ]	70.0	70.2	30.5	35.3	77.5	60.4	67.6	65.0	58.5	61.7
Ours	74.6	76.7	36.3	42.2	82.9	69.7	76.2	73.8	66.5	66.9

Table 10: Comparison of open-set semantic segmentation on five anomaly benchmarks. Existing OOD segmentation methods (SynBoost, PEBAL, RPL+CoroCL, S2M) rely on COCO-Stuff segmentation supervision or additional anomaly masks for training. In contrast, our open-vocabulary segmentation model performs OOD detection purely through a text-extended vocabulary (Cityscapes 19 ID classes + 150 curated COCO-Stuff OOD concepts vocabulary) without using any COCO-Stuff pixel annotations.

Figure 10: Qualitative comparison with S2M [ 70 ] on SMIYC-Anomaly and SMIYC-Obstacle.

Figure 11: More qualitative comparisons on unseen domains under the OVDG-SS setting on ACDC-41 dataset.

Figure 12: More qualitative comparisons on unseen domains under the OVDG-SS setting on Mapi-30 datasets.

Figure 13: More qualitative comparisons on unseen domains under the OVDG-SS setting on RW-10 datasets.

Method	Backbone	Training Data	Dv-19			Ave.	Dv-58				Ave.
Method	Backbone	Training Data	ACDC-19	BDD-19	Mapi-19	Ave.	ACDC-41	BDD-41	Mapi-30	RW-10	Ave.
CLIP-DINOiser(ECCV2024) [ 55 ]	CLIP+DinoV2 ViT-L/14	Training-Free	25.3	28.5	27.7	27.2	31.9	34.0	26.9	21.6	28.6
ClearCLIP(ECCV2024) [ 19 ]	CLIP ViT-L/14	Training-Free	26.7	28.3	27.0	27.3	33.5	34.0	26.6	21.6	28.9
ProxyCLIP (ECCV’24)	CLIP+SAM ViT-L/14	Training-Free	31.0	38.2	40.0	36.4	38.5	52.2	33.5	27.2	37.9
OVSeg (CVPR’23) [ 26 ]	EVA02 ViT-L/14	CS-7	41.9	42.6	43.9	42.8	51.2	49.3	33.7	35.7	42.5
SAN (CVPR’23) [ 56 ]		CS-7	46.6	47.0	48.3	47.3	57.5	56.9	38.0	34.8	46.8

Table 11: Comparison of different OV-SS methods using ViT-L/14 backbone under OVDG-SS setting.

CS-7→ACDC-19 ViT-B/16
Methods	Rd	Sw	Bldg	Veg	Sky	Per	Car	Wal	Fen	Pol	TL	TS	Ter	Rid	Trk	Bus	Trn	Mot	Bic	mIoU
OVSeg (CVPR’23) [ 26 ]	83.3	37.5	85.5	69.1	80.5	43.8	68.2	29.3	21.0	8.9	8.6	7.8	13.0	2.3	42.6	35.7	9.4	16.5	32.9	36.6
SAN (CVPR’23) [ 56 ]	86.4	34.5	82.6	73.6	96.5	49.8	71.2	31.0	18.2	7.5	7.0	5.7	14.3	0.1	46.1	32.7	9.0	17.2	30.9	37.6
CAT-Seg (CVPR’24) [ 7 ]	82.2	39.8	84.8	73.5	92.0	49.3	78.9	34.9	20.4	8.8	8.5	7.2	15.4	2.3	44.6	34.6	11.1	18.4	32.0	38.9
CLIPSelf (ICLR’24) [ 52 ]	84.9	38.9	73.1	72.0	89.1	45.4	74.7	27.9	31.6	24.9	24.5	32.8	18.3	14.5	40.3	35.5	12.1	31.8	36.5	42.6
RSC-CLIPSelf (ICLR’25) [ 38 ]	74.4	39.1	83.0	74.4	73.9	44.9	69.7	26.7	27.5	23.4	27.5	28.6	16.5	12.5	38.2	30.2	11.6	29.9	36.2	40.4

Table 12: Per-class results for OVDG-SS. Green entries denote seen classes, while blue entries correspond to unseen classes.

CS-7→ACDC-41 ViT-B/16
	Rd	Sw	Bldg	Veg	Sky	Per	Car	Wal	Fen	Pol	TL	TS	Ter	Rid	Trk	Bus	Trn	Mot	Bic	Bag	Bal	Bar	Bnc	Brd	Btl	Crt	Cat	Chr	Cow	Der	Dog	Drn	Elp	Hat	Hrs	Rbt	Shp	Tbl	Toy	Umb	Zbr	mIoU
SAN (CVPR’23) [ 56 ]	82.5	35.4	83.6	65.2	76.2	37.9	73.9	25.0	12.1	7.5	6.6	4.0	13.9	0.0	41.2	32.1	6.7	17.9	17.1	62.1	79.3	56.8	4.0	41.4	40.1	41.1	62.2	44.7	68.3	52.9	60.1	44.5	80.4	80.4	68.9	60.9	62.9	6.6	28.3	67.5	66.3	44.4
CAT-Seg (CVPR’24) [ 7 ]	83.5	42.4	81.4	65.3	85.4	39.6	71.9	29.4	14.3	8.9	7.9	5.6	15.0	1.4	46.5	33.6	10.1	20.7	17.6	69.0	76.2	60.4	7.1	49.4	44.1	51.3	75.3	53.3	66.9	51.7	68.3	49.3	76.6	83.3	67.1	62.2	71.2	8.1	30.3	74.6	76.2	47.6
CLIPSelf (ICLR’24) [ 52 ]	88.2	42.0	80.6	73.9	92.0	41.3	72.0	29.5	14.7	9.2	9.0	6.3	15.1	1.5	50.2	33.7	10.5	20.0	17.1	72.5	78.9	68.8	7.6	45.4	46.7	53.3	72.1	55.4	63.8	51.2	73.0	46.5	84.0	86.9	63.3	69.9	80.6	7.8	30.0	73.7	86.0	49.4
RSC-CLIPSelf (ICLR’25) [ 38 ]	88.2	44.0	88.5	74.5	92.6	40.1	69.3	28.8	16.1	8.7	8.8	6.1	15.1	1.6	50.1	37.0	10.5	21.7	18.2	75.1	74.8	76.3	8.1	48.1	45.1	57.4	72.8	58.2	63.3	54.7	74.9	49.7	79.1	82.1	60.3	73.1	80.6	7.7	32.2	70.9	84.2	50.0
MAFT+(ECCV’24) [ 16 ]	80.0	40.9	74.0	66.6	81.6	36.3	68.2	29.8	13.5	8.7	8.4	5.7	13.1	1.5	46.8	29.8	10.4	20.2	15.8	68.3	78.8	68.4	6.9	45.4	46.7	48.1	64.9	48.6	61.7	45.8	72.1	47.7	76.9	87.9	59.7	66.1	80.7	6.8	27.1	67.8	75.9	46.4

Table 13: Per-class results for OVDG-SS in CS-7→ACDC-41 and BDD-41.

CS-7→Map-30 ViT-B/16
	Rd	Sw	Bldg	Veg	Sky	Per	Car	Wal	Fen	Pol	TL	TS	Ter	Rid	Trk	Bus	Trn	Mot	Bic	Chr	Brg	Tun	Wtr	Snw	Snd	Anm	Sbd	Rwy	Bot	Tc	mIoU
SAN (CVPR’23) [ 56 ]	69.3	44.7	67.6	87.1	101.5	61.2	84.6	21.2	18.2	10.8	4.7	6.0	13.2	0.4	36.4	52.5	15.5	51.0	43.6	29.4	27.1	20.2	31.0	42.0	18.0	38.6	1.0	16.0	10.5	39.8	35.4
CAT-Seg (CVPR’24) [ 7 ]	78.0	43.0	81.6	82.5	95.8	62.7	83.9	30.7	20.4	13.3	6.9	8.1	17.8	2.5	45.0	54.9	18.8	48.5	42.3	31.2	32.0	44.2	32.3	43.9	19.3	37.1	3.1	15.7	11.1	40.4	38.2
CLIPSelf (ICLR’24) [ 52 ]	81.1	45.8	82.4	81.4	108.2	70.0	88.6	30.3	23.1	14.0	6.7	7.9	17.9	2.7	44.8	52.5	20.7	51.4	46.3	29.7	32.2	32.9	32.8	46.6	18.8	41.3	3.1	16.5	11.7	45.1	39.6
RSC-CLIPSelf (ICLR’25) [ 38 ]	80.1	45.6	88.4	86.1	98.0	65.4	82.8	29.9	25.6	13.7	6.2	7.7	16.8	3.1	49.0	48.5	18.9	55.1	49.5	29.5	34.5	44.2	37.0	51.8	17.6	43.3	2.9	15.4	13.3	42.8	40.1
MAFT+(ECCV’24) [ 16 ]	82.6	43.1	79.3	76.9	110.2	71.9	87.9	30.1	22.6	12.5	5.9	7.8	16.1	2.6	45.4	49.7	18.1	46.7	44.4	27.7	29.6	33.3	32.3	41.9	16.8	38.5	3.0	15.8	12.0	39.5	38.1

Table 14: Per-class results for OVDG-SS in CS-7→Mapi-30.

CS-7→RW-10 ViT-B/16
	Rd	Sw	Bar	PV	WV	PO	Wkr	Con	ABd	TTC	mIoU
SAN (CVPR’23) [ 56 ]	80.3	37.9	58.7	3.8	41.6	28.7	32.4	6.8	3.7	12.7	30.7
CAT-Seg (CVPR’24) [ 7 ]	89.9	62.8	57.3	20.9	48.1	22.9	24.7	10.0	4.5	24.2	36.5
CLIPSelf (ICLR’24) [ 52 ]	88.1	62.9	58.8	20.0	49.5	23.5	24.8	9.5	4.5	23.8	36.5
RSC-CLIPSelf (ICLR’25) [ 38 ]	85.6	61.8	54.9	20.9	46.7	23.5	23.4	9.8	4.3	23.4	35.4
MAFT+(ECCV’24) [ 16 ]	89.4	55.9	51.4	20.9	44.8	20.8	23.8	9.7	4.3	23.8	34.5

Table 15: Per-class results for OVDG-SS in CS-7→RW10.

Domain Generalization

Domain generalization develops models that perform reliably in new environments without training on target-domain data.

See all 375 papers

dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
Mar 19, 2026Saikat Dutta, Biplab Banerjee, Hamid RezatofighiOpen-Vocabulary Semantic Segmentation Vision-Language Models
IITB-Monash Research Academy · IIT Bombay · Monash University
Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection
Apr 6, 2026Weihao Cao, Runqi Wang, Xiaoyue Duan +3Open-Vocabulary Object Detection Vision Foundation Model Adaptation
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1
Open Vocabulary Domain Unlearning
Sep 25, 2026Sumanth Udupa, Mehrtash Harandi, Yadan Luo +1VLM Unlearning Machine Unlearning
The University of Queensland · Monash University

Mar 19, 2026J/K move · Enter open · S save

dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3

Saikat Dutta, Biplab Banerjee, Hamid Rezatofighi

IITB-Monash Research Academy · IIT Bombay · Monash University

Open-Vocabulary Semantic Segmentation Vision-Language Models

Open paper AskPDF arXiv

Open-Vocabulary Semantic Segmentation (OVSS) assigns pixel-level labels from an open set of text-defined categories, demanding reliable generalization to unseen classes at inference. Although modern vision-language models (VLMs) support strong open-vocabulary recognition, their representations learned through global contrastive objectives remain suboptimal for dense prediction, prompting many OVSS methods to depend on limited adaptation or refinement of image-text similarity maps. This, in turn, restricts spatial precision and robustness in complex, cluttered scenes. We introduce dinov3.seg, extending dinov3.txt into a dedicated framework for OVSS. Our contributions are four-fold. First, we design a task-specific architecture tailored to this backbone, systematically adapting established design principles from prior open-vocabulary segmentation work. Second, we jointly leverage text embeddings aligned with both the global [CLS] token and local patch-level visual features from ViT-based encoder, effectively combining semantic discrimination with fine-grained spatial locality. Third, unlike prior approaches that rely primarily on post hoc similarity refinement, we perform early refinement of visual representations prior to image-text interaction, followed by late refinement of the resulting image-text correlation features, enabling more accurate and robust dense predictions in cluttered scenes. Finally, we propose a high-resolution local-global inference strategy based on sliding-window aggregation, which preserves spatial detail while maintaining global context. We conduct extensive experiments on five widely adopted OVSS benchmarks to evaluate our approach. The results demonstrate its effectiveness and robustness, consistently outperforming current state-of-the-art methods.

Abstract

Domain Generalization

Explore similar work

dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3

Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection

Open Vocabulary Domain Unlearning