Open-Vocabulary Domain Generalization in Urban-Scene Segmentation
Authors: Dong Zhao, Qi Zang, Nan Pu, Wenjing Li, Nicu Sebe, Zhun Zhong
Organizations: Department of Information Engineering and Computer Science, University of Trento, Italy · School of Computer Science and Information Engineering, Hefei University of Technology, China
Domain Generalization in Semantic Segmentation (DG-SS) aims to enable segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that mitigates domain-induced distortions and produces more consistent text-image correlations under distribution changes. Extensive experiments on our constructed benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.
Figures & tables
Figure 1: Concept of Open-Vocabulary Domain Generalization in Semantic Segmentation (OVDG-SS). (a) OVDG-SS aims to generalize across both unseen domains (e.g., lighting, weather, geographic region, construction context) and unseen classes (e.g., barrier, cone, railway), moving beyond conventional domain generalization (DG-SS) and open-vocabulary segmentation (OV-SS). (b) Comparison of segmentation results. Traditional DG-SS fails to recognize unseen categories, and OV-SS struggles under domain shifts, while our OVDG-SS effectively handles both unseen domains and classes simultaneously.
OV-SS Method
Training Data
Seen
Num.
Shared classes
Performance
class
Dv-19
Dv-58
Dv-19
Dv-58
CAT-Seg [ 7 ]
COCO-Stuff [ 27 ]
171
118K
17
34
31.6
32.4
CAT-Seg
SYNTHIA [ 43 ]
7
9.4K
7
7
43.6
45.3
CAT-Seg
GTA [ 41 ]
7
25K
7
7
47.5
48.2
CAT-Seg
Cityscapes [ 8 ]
7
3K
7
7
49.3
50.0
MaskAdapter [ 24 ]
COCO-Stuff
171
118K
17
34
30.1
29.6
Table 1: Effect of training domains on training-based OV-SS performance. Colored rows indicate models trained on driving datasets. Dv-19 and Dv-58 refer to cross-domain driving datasets with 19 and 58 categories.
Figure 2: Efficiency and performance comparison on OVDG-SS tasks using EVA02 ViT-B/16 as backbone. FPS is tested on images with a short edge of 480 and a long edge of 960. Our method achieves the best trade-off among generalization ability, speed, and parameter efficiency.
Figure 3: Overview of the proposed S 2 -Corr. The upper part shows the CLIP-based encoding and correlation aggregation pipeline. The lower part illustrates our S 2 -Corr, which refines text–image correlations using a specially designed chunked State-Space Models (SSM) aggregation scheme.
Figure 4: Effect of domain shift on text–image correlations of the class “ sky ” from EVA02 model [ 11 ] . Color map ranges from blue (low correlation) to red (high correlation) . As the domain shift increases from left to right, the initial correlation maps become progressively noisier, with incorrect activations spreading across irrelevant regions.
Figure 5: Comparison of scanning strategies in state-space correlation aggregation between VMamba [ 30 ] and ours. Our method introduces a learnable geometric decay ( ∗γ ) to suppress long-range noise within each chunk and uses a snake-shaped scanning strategy that preserves spatial continuity by passing end states between adjacent chunks.
Method
Backbone
Training Data
Dv-19
Ave.
Dv-58
Ave.
ACDC-19
BDD-19
Mapi-19
ACDC-41
BDD-41
Mapi-30
RW-10
CLIP-DINOiser(ECCV2024) [ 55 ]
EVA02+DINOV2 ViT-L
Training-Free
25.3
28.5
27.7
27.2
31.9
34.0
26.9
21.6
28.6
ClearCLIP(ECCV2024) [ 19 ]
EVA02 ViT-L
Training-Free
26.7
28.3
27.0
27.3
33.5
34.0
26.6
21.6
28.9
ProxyCLIP (ECCV’24) [ 20 ]
EVA02+SAM ViT-L
Training-Free
31.0
38.2
40.0
36.4
38.5
52.2
33.5
27.2
37.9
CAT-Seg (CVPR’24) [ 7 ]
EVA02 ViT-L/14
COCO
30.3
31.9
36.1
32.8
33.2
42.8
28.6
27.6
33.1
DeCLIP (CVPR’25) [ 49 ]
EVA02+DINOV2 ViT-L
COCO
32.1
31.8
38.1
34.0
34.2
47.0
32.0
28.3
35.4
Table 2: Comparison of different OV-SS methods across different backbones under the Real-to-Real OVDG-SS setting. Dv-19 groups ACDC-19, BDD-19, and Mapi-19, while Dv-58 groups ACDC-41, BDD-41, Mapi-30, and RW-10.
Method
Backbone
Training Data
Dv-19
Ave.
Dv-58
Ave.
CS-19
ACDC-19
BDD-19
Mapi-19
ACDC-41
BDD-41
Mapi-30
RW-10
OVSeg (CVPR’23) [ 26 ]
EVA02 ViT-B/16
GTA-7
41.2
34.9
42.5
45.1
40.9
45.1
52.6
35.5
32.9
41.5
SAN (CVPR’23) [ 56 ]
GTA-7
43.3
36.7
42.0
48.1
42.5
47.7
49.0
37.8
30.3
41.2
CAT-Seg (CVPR’24) [ 7 ]
GTA-7
43.7
37.6
45.4
48.8
43.9
51.2
56.2
39.3
35.7
45.6
MAFT+(ECCV’24) [ 16 ]
GTA-7
45.7
38.5
43.7
50.2
44.5
49.1
48.3
39.9
31.8
42.3
ESC-Net (CVPR’25) [ 21 ]
GTA-7
42.2
35.9
44.6
48.1
42.7
49.4
52.4
37.9
33.3
43.3
Table 3: Comparison of different OV-SS methods across various backbones under the synthetic-to-real OVDG-SS setting.
Note
Methods
ViT-B/16
ViT-L/14
Ave.
Dv-19
Dv-58
Dv-19
Dv-58
Base
Cross-attention
43.5
43.5
49.3
50.0
46.6
#1
Seletive SSM
45.6
44.1
50.7
50.5
47.7
#2
+Modulation
47.6
45.3
52.1
50.9
49.0
#3
+Geometric decay
48.3
46.4
53.2
51.8
49.9
#4
+Chunk
49.6
47.3
55.3
52.7
51.2
Table 4: Ablation study of the proposed method under different design components on CS7 → Dv-19 and Dv-58 across backbones. Method #1 replaces cross-attention by selective SSM. Methods of #2–#5 are progressively added on top of #1.
Figure 6: Comparison of text–image correlation aggregation on seen and unseen classes from unseen domains. Our method yields clearer and more localized text–image correlations than CAT-Seg [ 7 ] and Corelation Map (initially obtained by Eq. 1 ), improving both seen and unseen predictions under domain shifts.
Figure 7: Qualitative comparisons on unseen domains under the OVDG-SS setting, with DG-SS method [ 61 ] and OV-SS method [ 24 ] .
Model
FPS
FPS
FPS
GPU
Training
vocab19
vocab58
vocab150
Mem. (GB)
Time (min)
CAT-Seg [ 7 ]
15.4
10.6
5.7
13.8
180
ESC-Net [ 21 ]
15.0
9.9
5.1
15.7
220
Seletive SSM
12.3
9.4
6.4
16.8
240
S 2 -Corr (Ours)
26.1
22.2
18.3
9.2
140
Table 5: Inference and training efficiency of competing methods using a ViT-B/16 backbone. FPS is evaluated under different test vocabulary sizes.
Table 6: Summary of datasets used in the OVDG-SS benchmark.
Figure 8: Impact of hyperparameters: chunk size, prior decay, text prompt templates, and training resolution
VLM
ACDC-19
BDD-19
Mapi-19
ACDC-41
BDD-41
Mapi-30
RW-10
Ave.
EVA-02-CLIP-L/14 [ 11 ]
54.3
53.1
60
62
61.7
47.4
41.9
44.7
SigLIP-ViT-L/16 [ 48 ]
52.7
54.1
59.2
60.5
60.4
48.3
42.5
54.0
CLIP-ViT-L/14 [ 39 ]
53.7
52.8
59.0
61.6
60.9
46.7
40.9
53.7
SelfCLIP-ViT-L/14 [ 52 ]
56.3
53.6
60.9
63.8
63.5
48.1
41.9
59.6
Table 7: Comparison of different Vision-Language Models (VLMs) under the OVDG-SS setting across multiple datasets. The best results are underlined.
Variant
Row
Col
mIoU Dv-19
mIoU Dv-58
Training Time (Min)
Row-only (ours)
✓
×
50.3
47.9
140
Col
×
✓
48.6
45.7
140
Row+Col
✓
✓
50.4
48.1
165
Table 8: Ablation on different scanning strategies used in the spatial aggregation module.
Training Data: GTA-19
Type
Pretrain
CS-19
BDD-19
Mapi-19
Ave.
Mapi-30
RW-10
ACDC-41
BDD-41
Ave.
Rein (CVPR’24) [ 50 ]
DG
EVA02-L
65.3
60.5
64.9
63.6
-
-
-
-
-
tqdm (ECCV’24) [ 36 ]
DG
EVA02-L
68.9
59.2
70.1
66.1
-
-
-
-
-
SoMA (CVPR’25) [ 61 ]
DG
EVA02-L
68.1
60.8
68.3
65.7
-
-
-
-
-
MaskAdapter (CVPR’25) [ 24 ]
OVSS
EVA02-L
61.5
55.5
65.7
60.9
48.5
41.0
63.3
61.8
53.7
MAFT+(ECCV24) [ 16 ]
OVSS
EVA02-L
61.2
55.0
65.0
61.2
48.2
40.1
63.9
62.4
53.7
Table 9: Extended results under a larger 19-class training vocabulary. The main paper reports OVDG performance using a compact 7-class training set, this table presents results obtained with the full 19-class vocabulary (GTA-19).
Figure 9: More comparison of initial text–image correlation aggregation on seen and unseen classes from unseen domains.
Methods
FS Static
FS Lost&Found
SMIYC-Anomaly
SMIYC-Obstacle
RoadAnomaly
IoU
mean F1
IoU
mean F1
IoU
mean F1
IoU
mean F1
IoU
mean F1
Synboost [ 10 ]
32.8
25.7
18.4
10.9
42.0
39.5
14.0
9.0
27.2
29.3
PEBAL [ 47 ]
26.9
13.3
6.4
2.6
42.4
35.1
6.7
1.1
33.8
23.9
RPL+CoroCL [ 31 ]
36.5
13.2
15.8
3.9
68.8
31.6
28.7
5.7
51.0
24.6
S2M [ 70 ]
70.0
70.2
30.5
35.3
77.5
60.4
67.6
65.0
58.5
61.7
Ours
74.6
76.7
36.3
42.2
82.9
69.7
76.2
73.8
66.5
66.9
Table 10: Comparison of open-set semantic segmentation on five anomaly benchmarks. Existing OOD segmentation methods (SynBoost, PEBAL, RPL+CoroCL, S2M) rely on COCO-Stuff segmentation supervision or additional anomaly masks for training. In contrast, our open-vocabulary segmentation model performs OOD detection purely through a text-extended vocabulary (Cityscapes 19 ID classes + 150 curated COCO-Stuff OOD concepts vocabulary) without using any COCO-Stuff pixel annotations.
Figure 10: Qualitative comparison with S2M [ 70 ] on SMIYC-Anomaly and SMIYC-Obstacle.
Figure 11: More qualitative comparisons on unseen domains under the OVDG-SS setting on ACDC-41 dataset.
Figure 12: More qualitative comparisons on unseen domains under the OVDG-SS setting on Mapi-30 datasets.
Figure 13: More qualitative comparisons on unseen domains under the OVDG-SS setting on RW-10 datasets.
Method
Backbone
Training Data
Dv-19
Ave.
Dv-58
Ave.
ACDC-19
BDD-19
Mapi-19
ACDC-41
BDD-41
Mapi-30
RW-10
CLIP-DINOiser(ECCV2024) [ 55 ]
CLIP+DinoV2 ViT-L/14
Training-Free
25.3
28.5
27.7
27.2
31.9
34.0
26.9
21.6
28.6
ClearCLIP(ECCV2024) [ 19 ]
CLIP ViT-L/14
Training-Free
26.7
28.3
27.0
27.3
33.5
34.0
26.6
21.6
28.9
ProxyCLIP (ECCV’24)
CLIP+SAM ViT-L/14
Training-Free
31.0
38.2
40.0
36.4
38.5
52.2
33.5
27.2
37.9
OVSeg (CVPR’23) [ 26 ]
EVA02 ViT-L/14
CS-7
41.9
42.6
43.9
42.8
51.2
49.3
33.7
35.7
42.5
SAN (CVPR’23) [ 56 ]
CS-7
46.6
47.0
48.3
47.3
57.5
56.9
38.0
34.8
46.8
Table 11: Comparison of different OV-SS methods using ViT-L/14 backbone under OVDG-SS setting.
CS-7→ACDC-19 ViT-B/16
Methods
Rd
Sw
Bldg
Veg
Sky
Per
Car
Wal
Fen
Pol
TL
TS
Ter
Rid
Trk
Bus
Trn
Mot
Bic
mIoU
OVSeg (CVPR’23) [ 26 ]
83.3
37.5
85.5
69.1
80.5
43.8
68.2
29.3
21.0
8.9
8.6
7.8
13.0
2.3
42.6
35.7
9.4
16.5
32.9
36.6
SAN (CVPR’23) [ 56 ]
86.4
34.5
82.6
73.6
96.5
49.8
71.2
31.0
18.2
7.5
7.0
5.7
14.3
0.1
46.1
32.7
9.0
17.2
30.9
37.6
CAT-Seg (CVPR’24) [ 7 ]
82.2
39.8
84.8
73.5
92.0
49.3
78.9
34.9
20.4
8.8
8.5
7.2
15.4
2.3
44.6
34.6
11.1
18.4
32.0
38.9
CLIPSelf (ICLR’24) [ 52 ]
84.9
38.9
73.1
72.0
89.1
45.4
74.7
27.9
31.6
24.9
24.5
32.8
18.3
14.5
40.3
35.5
12.1
31.8
36.5
42.6
RSC-CLIPSelf (ICLR’25) [ 38 ]
74.4
39.1
83.0
74.4
73.9
44.9
69.7
26.7
27.5
23.4
27.5
28.6
16.5
12.5
38.2
30.2
11.6
29.9
36.2
40.4
Table 12: Per-class results for OVDG-SS. Green entries denote seen classes, while blue entries correspond to unseen classes.
CS-7→ACDC-41 ViT-B/16
Rd
Sw
Bldg
Veg
Sky
Per
Car
Wal
Fen
Pol
TL
TS
Ter
Rid
Trk
Bus
Trn
Mot
Bic
Bag
Bal
Bar
Bnc
Brd
Btl
Crt
Cat
Chr
Cow
Der
Dog
Drn
Elp
Hat
Hrs
Rbt
Shp
Tbl
Toy
Umb
Zbr
mIoU
SAN (CVPR’23) [ 56 ]
82.5
35.4
83.6
65.2
76.2
37.9
73.9
25.0
12.1
7.5
6.6
4.0
13.9
0.0
41.2
32.1
6.7
17.9
17.1
62.1
79.3
56.8
4.0
41.4
40.1
41.1
62.2
44.7
68.3
52.9
60.1
44.5
80.4
80.4
68.9
60.9
62.9
6.6
28.3
67.5
66.3
44.4
CAT-Seg (CVPR’24) [ 7 ]
83.5
42.4
81.4
65.3
85.4
39.6
71.9
29.4
14.3
8.9
7.9
5.6
15.0
1.4
46.5
33.6
10.1
20.7
17.6
69.0
76.2
60.4
7.1
49.4
44.1
51.3
75.3
53.3
66.9
51.7
68.3
49.3
76.6
83.3
67.1
62.2
71.2
8.1
30.3
74.6
76.2
47.6
CLIPSelf (ICLR’24) [ 52 ]
88.2
42.0
80.6
73.9
92.0
41.3
72.0
29.5
14.7
9.2
9.0
6.3
15.1
1.5
50.2
33.7
10.5
20.0
17.1
72.5
78.9
68.8
7.6
45.4
46.7
53.3
72.1
55.4
63.8
51.2
73.0
46.5
84.0
86.9
63.3
69.9
80.6
7.8
30.0
73.7
86.0
49.4
RSC-CLIPSelf (ICLR’25) [ 38 ]
88.2
44.0
88.5
74.5
92.6
40.1
69.3
28.8
16.1
8.7
8.8
6.1
15.1
1.6
50.1
37.0
10.5
21.7
18.2
75.1
74.8
76.3
8.1
48.1
45.1
57.4
72.8
58.2
63.3
54.7
74.9
49.7
79.1
82.1
60.3
73.1
80.6
7.7
32.2
70.9
84.2
50.0
MAFT+(ECCV’24) [ 16 ]
80.0
40.9
74.0
66.6
81.6
36.3
68.2
29.8
13.5
8.7
8.4
5.7
13.1
1.5
46.8
29.8
10.4
20.2
15.8
68.3
78.8
68.4
6.9
45.4
46.7
48.1
64.9
48.6
61.7
45.8
72.1
47.7
76.9
87.9
59.7
66.1
80.7
6.8
27.1
67.8
75.9
46.4
Table 13: Per-class results for OVDG-SS in CS-7→ACDC-41 and BDD-41.
CS-7→Map-30 ViT-B/16
Rd
Sw
Bldg
Veg
Sky
Per
Car
Wal
Fen
Pol
TL
TS
Ter
Rid
Trk
Bus
Trn
Mot
Bic
Chr
Brg
Tun
Wtr
Snw
Snd
Anm
Sbd
Rwy
Bot
Tc
mIoU
SAN (CVPR’23) [ 56 ]
69.3
44.7
67.6
87.1
101.5
61.2
84.6
21.2
18.2
10.8
4.7
6.0
13.2
0.4
36.4
52.5
15.5
51.0
43.6
29.4
27.1
20.2
31.0
42.0
18.0
38.6
1.0
16.0
10.5
39.8
35.4
CAT-Seg (CVPR’24) [ 7 ]
78.0
43.0
81.6
82.5
95.8
62.7
83.9
30.7
20.4
13.3
6.9
8.1
17.8
2.5
45.0
54.9
18.8
48.5
42.3
31.2
32.0
44.2
32.3
43.9
19.3
37.1
3.1
15.7
11.1
40.4
38.2
CLIPSelf (ICLR’24) [ 52 ]
81.1
45.8
82.4
81.4
108.2
70.0
88.6
30.3
23.1
14.0
6.7
7.9
17.9
2.7
44.8
52.5
20.7
51.4
46.3
29.7
32.2
32.9
32.8
46.6
18.8
41.3
3.1
16.5
11.7
45.1
39.6
RSC-CLIPSelf (ICLR’25) [ 38 ]
80.1
45.6
88.4
86.1
98.0
65.4
82.8
29.9
25.6
13.7
6.2
7.7
16.8
3.1
49.0
48.5
18.9
55.1
49.5
29.5
34.5
44.2
37.0
51.8
17.6
43.3
2.9
15.4
13.3
42.8
40.1
MAFT+(ECCV’24) [ 16 ]
82.6
43.1
79.3
76.9
110.2
71.9
87.9
30.1
22.6
12.5
5.9
7.8
16.1
2.6
45.4
49.7
18.1
46.7
44.4
27.7
29.6
33.3
32.3
41.9
16.8
38.5
3.0
15.8
12.0
39.5
38.1
Table 14: Per-class results for OVDG-SS in CS-7→Mapi-30.
CS-7→RW-10 ViT-B/16
Rd
Sw
Bar
PV
WV
PO
Wkr
Con
ABd
TTC
mIoU
SAN (CVPR’23) [ 56 ]
80.3
37.9
58.7
3.8
41.6
28.7
32.4
6.8
3.7
12.7
30.7
CAT-Seg (CVPR’24) [ 7 ]
89.9
62.8
57.3
20.9
48.1
22.9
24.7
10.0
4.5
24.2
36.5
CLIPSelf (ICLR’24) [ 52 ]
88.1
62.9
58.8
20.0
49.5
23.5
24.8
9.5
4.5
23.8
36.5
RSC-CLIPSelf (ICLR’25) [ 38 ]
85.6
61.8
54.9
20.9
46.7
23.5
23.4
9.8
4.3
23.4
35.4
MAFT+(ECCV’24) [ 16 ]
89.4
55.9
51.4
20.9
44.8
20.8
23.8
9.7
4.3
23.8
34.5
Table 15: Per-class results for OVDG-SS in CS-7→RW10.
Open-Vocabulary Semantic Segmentation (OVSS) assigns pixel-level labels from an open set of text-defined categories, demanding reliable generalization to unseen classes at inference. Although modern vision-language models (VLMs) support strong open-vocabulary recognition, their representations learned through global contrastive objectives remain suboptimal for dense prediction, prompting many OVSS methods to depend on limited adaptation or refinement of image-text similarity maps. This, in turn, restricts spatial precision and robustness in complex, cluttered scenes. We introduce dinov3.seg, extending dinov3.txt into a dedicated framework for OVSS. Our contributions are four-fold. First, we design a task-specific architecture tailored to this backbone, systematically adapting established design principles from prior open-vocabulary segmentation work. Second, we jointly leverage text embeddings aligned with both the global [CLS] token and local patch-level visual features from ViT-based encoder, effectively combining semantic discrimination with fine-grained spatial locality. Third, unlike prior approaches that rely primarily on post hoc similarity refinement, we perform early refinement of visual representations prior to image-text interaction, followed by late refinement of the resulting image-text correlation features, enabling more accurate and robust dense predictions in cluttered scenes. Finally, we propose a high-resolution local-global inference strategy based on sliding-window aggregation, which preserves spatial detail while maintaining global context. We conduct extensive experiments on five widely adopted OVSS benchmarks to evaluate our approach. The results demonstrate its effectiveness and robustness, consistently outperforming current state-of-the-art methods.
Saikat Dutta, Biplab Banerjee, Hamid Rezatofighi
IITB-Monash Research Academy · IIT Bombay · Monash University
Open-vocabulary object detection (OVOD) enables models to detect any object category, including unseen ones. Benefiting from large-scale pre-training, existing OVOD methods achieve strong detection performance on general scenarios (e.g., OV-COCO) but suffer severe performance drops when transferred to downstream tasks with substantial domain shifts. This degradation stems from the scarcity and weak semantics of category labels in domain-specific task, as well as the inability of existing models to capture auxiliary semantics beyond coarse-grained category label. To address these issues, we propose HSA-DINO, a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection. Specifically, we propose a multi-scale prompt bank that leverages image feature pyramids to capture hierarchical semantics and select domain-specific local semantic prompts, progressively enriching textual representations from coarse to fine-grained levels. Furthermore, we introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference, thereby preventing parameter updates from degrading the generalization ability of the pre-trained OVOD model. We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings. The results show that HSA-DINO performs favorably against previous state-of-the-art methods, achieving a superior trade-off between domain adaptability and open-vocabulary generalization.
Weihao Cao, Runqi Wang, Xiaoyue Duan +3
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1
Vision-Language Models (VLMs) exhibit remarkable zero-shot generalization, yet they often encode unwanted or hazardous stylistic domains such as idealized textbook diagrams in medical AI or cartoon vehicles in autonomous driving. Approximate Domain Unlearning (ADU) aims to selectively erase a model's recognition of a target visual domain while preserving accuracy on the remaining domains. However, existing ADU methods operate under a flawed closed-vocabulary assumption: they evaluate unlearning solely on the specific object classes seen during the unlearning fine-tuning phase. Consequently, these methods do not unlearn the domain itself; they merely overfit to seen class-domain pairs, leaving the domain easily recognizable for unseen classes and providing a false sense of removal. We argue that true domain erasure must be class-agnostic. To address this, we formalize Open-Vocabulary Domain Unlearning (OVDU), a rigorous protocol that mandates domain forgetting must transfer to held-out classes. To solve the OVDU challenge, we propose a surgical parameter-editing framework. First, a Fisher Information mask isolates domain-sensitive weights, mathematically protecting foundational zero-shot generalization. Second, our Targeted Manifold Scattering (TMS) objective uses preference-based mining to locally scatter the forget domain's stylistic geometry. Evaluated across PACS, OfficeHome, and DomainNet, our method vastly improves open-vocabulary generalization over existing baselines. Crucially, it delivers exceptional sample efficiency, outperforming peak 8-shot baseline results with only 4 shots.