OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation

Seungjae Moon; Seunghyun Oh; Youngmin Ro

cs.CVApr 9, 2026

OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation

Authors: Seungjae Moon, Seunghyun Oh, Youngmin Ro

Organizations: Machine Intelligence Laboratory, University of Seoul, Korea

Abstract

Training-free open-vocabulary semantic segmentation(TF-OVSS) has recently attracted attention for its ability to perform dense prediction by leveraging the pretrained knowledge of large vision and vision-language models, without requiring additional training. However, due to the limited input resolution of these pretrained encoders, existing TF-OVSS methods commonly adopt a sliding-window strategy that processes cropped sub-images independently. While effective for managing high-resolution inputs, this approach prevents global attention over the full image, leading to fragmented feature representations and limited contextual reasoning. We propose OV-Stitcher, a training-free framework that addresses this limitation by stitching fragmented sub-image features directly within the final encoder block. By reconstructing attention representations from fragmented sub-image features, OV-Stitcher enables global attention within the final encoder block, producing coherent context aggregation and spatially consistent, semantically aligned segmentation maps. Extensive evaluations across eight benchmarks demonstrate that OV-Stitcher establishes a scalable and effective solution for open-vocabulary segmentation, achieving a notable improvement in mean Intersection over Union(mIoU) from 48.7 to 50.7 compared with prior training-free baselines.

Figures & tables

Figure 1: Top : Prior works process cropped sub-images independently, preventing attention across different sub-image features. Bottom : We introduce a Stitch Attention mechanism that enables global attention across all cropped regions, yielding more coherent and contextually consistent feature integration.

Figure 2: Illustration of the attention maps and patch-interactions for prior methods and our Stitch Attention. (a) presents prior methods, and (c) presents our approach.

Figure 3: Visualization of feature representations and segmentation results. (a) and (b) show the image feature maps and segmentation results obtained from the baseline and the proposed Stitch Attention, respectively. The top row shows the feature maps after applying PCA, and the bottom row presents the corresponding segmentation results.

Figure 4: Overview of our method OV-Stitcher . Our core framework starts from processing each sub-image using a sliding window approach. From the final layer of each sub-image, we extract $\tilde{Q}$ , $\tilde{K}$ , and $\tilde{V}$ features, and stitch each type separately across all sub-images to form the global $Q$ , $K$ , and $V$ . Self-attention on these stitched features produces a feature map capturing global correlations. The features resulting from Stitch Attention provide CLIP with global spatial information, enabling coherent reasoning across the full image. Additionally, we add Class-Biased Prompts to the existing prompts to generate text embeddings reducing ambiguity among similar categories.

OpenAI CLIP ViT-B/16
Method		With a background category			Without background category					Avg.
Method		VOC21	Context60	Object	VOC20	City	Context59	ADE20K	Stuff	Avg.
CLIP [ 39 ]	ICML’21	18.6	7.8	6.5	49.1	6.7	11.2	3.2	5.7	13.6
MaskCLIP [ 66 ]	ECCV’22	43.4	23.2	20.6	74.9	24.9	26.4	11.9	16.7	30.3
CLIPtrase [ 43 ]	ECCV’24	50.9	29.9	43.6	81.0	21.3	33.8	16.4	22.8	32.7
ClearCLIP [ 30 ]	ECCV’24	51.8	32.6	33.0	80.9	30.0	35.9	16.7	23.9	38.1

Table 1: Quantitative Comparison of Prior Open Vocabulary Segmentation Works. The highest-performing result is highlighted in bold , and the second highest in underline for clarity. The “w/o post-processing” rows show the performance without the post-processing step, where each SAM-generated mask is assigned the label corresponding to the most frequent raw logit prediction within that mask.

Figure 5: Qualitative comparison with previous training-free open vocabulary segmentation methods.

Figure 6: Qualitative comparison without post-processing.

StitchAttn	CBP	V21	C60	Obj.	Stf.	City	ADE
		72.2	41.6	40.7	30.7	44.6	23.7
	✓	73.2	42.7	41.6	31.0	45.9	24.3
✓		74.7	42.7	42.3	31.2	46.7	23.9
✓	✓	75.7	43.7	42.7	31.8	48.1	24.7

Table 2: Ablation study evaluating the impact of each component in our proposed method.

Figure 7: Ablation on resolution robustness. Post-processing is excluded to clearly show the effect of the proposed framework. The x-axis represents the settings in the format shorter side – window size – stride .

ProxyCLIP	V21	C60	Obj.	Stf.	City	ADE
X	61.3	35.3	37.5	26.5	38.1	20.2
✓	62.9	36.3	38.1	26.7	39.8	20.9

Table 3: Effectiveness of Stitch Attention on Other Method. “X” indicates the original ProxyCLIP; “✓” indicates ProxyCLIP with Stitch Attention.

Figure 8: Ablation on resolution robustness. Post-processing is excluded to clearly show the effect of the proposed framework. The x-axis represents the settings in the format shorter side – window size – stride .

OV-Stitcher
Resolution	V21	C60	Obj.	Stf.	ADE	avg.
336 ${}^{(336\text{–}112)}$	71.74	41.81	40.80	30.14	22.69	41.43
448 ${}^{(336\text{–}112)}$	73.13	42.42	41.42	30.76	23.61	42.27
560 ${}^{(336\text{–}112)}$	73.15	42.40	40.79	30.40	23.66	42.08
672 ${}^{(336\text{–}112)}$	72.19	41.75	39.06	29.50	23.67	41.23
CorrCLIP

Table 4: Ablation on resolution robustness. Comparison between CorrCLIP and WeaveCLIP under varying input resolutions without post-processing to clearly show the effect of the proposed framework. Each resolution is denoted as shorter side

{}^{(\textbf{window size - stride})}

Input Res.	# Crops	# Params. (M)	Mem. (MB)	Thru. (img/sec)
Precomputed Masks
336 $\times$ 336	1	235	1435	6.98
448 $\times$ 448	4	235	1450	4.72
560 $\times$ 560	9	235	2040	3.12
672 $\times$ 672	16	235	3198	2.12
Masks Generated On-the-Fly

Table 5: Computational costs on RTX 4090 with FP16. We separate cases where SAM2 masks for highlighting the attention map and post-processing are precomputed from those where they are generated on-the-fly.

Input Res.	Naive Ver. Stitch Attention		Flash Ver. Stitch Attention
Input Res.	Latency (ms)	Memory (MB)	Latency (ms)	Memory (MB)
336 $\times$ 336	0.25	293	0.21 ( ${\color[rgb]{0,0.293,0.7031}16.0\%\downarrow}$ )	220 ( ${\color[rgb]{0,0.293,0.7031}24.9\%\downarrow}$ )
448 $\times$ 448	0.72	454	0.44 ( ${\color[rgb]{0,0.293,0.7031}38.9\%\downarrow}$ )	241 ( ${\color[rgb]{0,0.293,0.7031}46.9\%\downarrow}$ )
560 $\times$ 560	1.65	799	0.81 ( ${\color[rgb]{0,0.293,0.7031}50.9\%\downarrow}$ )	273 ( ${\color[rgb]{0,0.293,0.7031}65.8\%\downarrow}$ )
672 $\times$ 672	3.08	1423	1.48 ( ${\color[rgb]{0,0.293,0.7031}52.0\%\downarrow}$ )	317 ( ${\color[rgb]{0,0.293,0.7031}77.7\%\downarrow}$ )

Table 6: Computational costs on RTX 4090 with FP16. Latency and peak CUDA memory of StitchAttention with naive attention and Flash Attention at different resolutions.

Type	Size	Acc.	V21	C60	Obj.	Stf.	City	ADE
OpenCLIP	ViT-B/16	70.2%	72.92	40.54	42.27	31.20	51.07	28.01
MetaCLIP		72.1%	76.37	43.92	44.59	32.10	52.26	27.80
DFNCLIP		76.2%	72.26	41.82	43.93	32.16	52.09	27.78
OpenCLIP	ViT-L/14	75.3%	73.10	41.55	42.16	31.02	52.82	28.10
MetaCLIP		79.2%	76.47	45.50	49.75	34.47	53.01	30.59
DFNCLIP		81.4%	74.58	43.39	43.13	33.77	51.77	28.63

Table 7: Comparison of different CLIP variants used as vision–language backbones. Acc. denotes the zero-shot classification accuracy of each CLIP model on ImageNet-1K [ 15 ]

Type	Size	V21	C60	Obj.	Stf.	City	ADE
DINO	ViT-S/8	75.84	43.72	42.63	31.86	47.05	24.61
	ViT-B/8	75.72	43.85	42.55	31.83	48.06	24.72
	ViT-B/16	75.47	43.68	41.77	31.69	46.58	24.26
DINOv2	ViT-B/14	75.42	43.41	42.14	31.53	45.23	24.31
DINOv2	ViT-L/14	75.28	43.21	42.54	31.45	44.14	24.36

Table 8: Evaluation of various feature extractors.

Figure 9: Qualitative comparison showing the effect of CBP. Qualitative comparison showing the effect of CBP. To enable a more explicit comparison, post-processing is removed; while higher feature coherence can cause larger regions to be assigned to the wrong class, CBP reduces class ambiguity and helps maintain correct labeling.

Class

Class-Biased Prompts Examples

goldfish

“a small, rounded body covered in shimmering scales” “a fish with a flat tail and vertical fins” “a small, orange fish with a white belly” “a gold-colored fish with a transparent tail” “a small, slender fish with a rounded head” “a fish with a long, flowing fins” “a small, yellow-gold fish with a black spot” “a fish with a flat, broad head” “a small, streamlined fish for swimming” “a fish with a bright orange dorsal fin” “a small, rounded fish with a horizontal stripe” “a fish with transparent scales reflecting light” “a small, gold-colored fish with a pointed snout” “a fish with a long, pointed fin on its back” “a small, slender fish with a distinctive pattern”

salad

“a mix of leafy greens and colorful vegetables”, “a bowl of fresh greens and vegetables arranged”, “a tossed salad with mixed vegetables and greens”, “a colorful salad with edible flowers”, “a crunchy salad with crispy vegetables and nuts”, “a fresh mix of lettuce and other leafy greens”, “a salad with a variety of textures and colors”, “a salad bowl filled with mixed greens and toppings”, “a simple salad of mixed greens and cherry tomatoes”, “a large salad bowl with multiple layers”, “a colorful salad with fruits and vegetables”, “a green salad with croutons and cheese”, “a mixed salad with crunchy and soft ingredients”, “a salad with a variety of leafy greens and vegetables”, “a refreshing salad with lettuce, tomatoes, and cucumbers”

Table 9: Representative examples of class-biased prompts generated for each category.

Figure 10: Qualitative comparison without post-processing. By removing post-processing, it becomes clear that our method produces more spatially and semantically feature-coherent results than the baseline CorrCLIP.

Figure 11: Additional qualitative comparison on VOC21 [ 16 ] .

Figure 12: Additional qualitative comparison on COCO Object [ 5 ] .

Figure 13: Additional qualitative comparison on Context60 [ 37 ] .

Figure 14: Additional qualitative comparison on Cityscapes [ 12 ]

Figure 15: Additional qualitative comparison on ADE20K [ 65 ]

Figure 16: Additional qualitative comparison on COCO Stuff [ 5 ]

Explore similar work

Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation
Mar 24, 2026ByeongCheol Lee, Hyun Seok Seong, Sangeek Hyun +3Open-Vocabulary Semantic Segmentation Sliding-Window Attention
Sungkyunkwan University
Direct Segmentation without Logits Optimization for Training-Free Open-Vocabulary Semantic Segmentation
Apr 9, 2026Jiahao Li, Yang Lu, Yachao Zhang +3Open-Vocabulary Semantic Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University. · Hanjiang National Laboratory. · East China Normal University.
dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
Mar 19, 2026Saikat Dutta, Biplab Banerjee, Hamid RezatofighiOpen-Vocabulary Semantic Segmentation Vision-Language Models
IITB-Monash Research Academy · IIT Bombay · Monash University

Mar 24, 2026J/K move · Enter open · S save

Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation

ByeongCheol Lee, Hyun Seok Seong, Sangeek Hyun, Gilhan Park+2

Sungkyunkwan University

Open-Vocabulary Semantic Segmentation Sliding-Window Attention Attention Mechanisms VLM Adaptation Vision-Language Alignment

Open paper AskPDF arXiv

A sliding-window inference strategy is commonly adopted in recent training-free open-vocabulary semantic segmentation methods to overcome limitation of the CLIP in processing high-resolution images. However, this approach introduces a new challenge: each window is processed independently, leading to semantic discrepancy across windows. To address this issue, we propose Global-Local Aligned CLIP~(GLA-CLIP), a framework that facilitates comprehensive information exchange across windows. Rather than limiting attention to tokens within individual windows, GLA-CLIP extends key-value tokens to incorporate contextual cues from all windows. Nevertheless, we observe a window bias: outer-window tokens are less likely to be attended, since query features are produced through interactions within the inner window patches, thereby lacking semantic grounding beyond their local context. To mitigate this, we introduce a proxy anchor, constructed by aggregating tokens highly similar to the given query from all windows, which provides a unified semantic reference for measuring similarity across both inner- and outer-window patches. Furthermore, we propose a dynamic normalization scheme that adjusts attention strength according to object scale by dynamically scaling and thresholding the attention map to cope with small-object scenarios. Moreover, GLA-CLIP can be equipped on existing methods and broad their receptive field. Extensive experiments validate the effectiveness of GLA-CLIP in enhancing training-free open-vocabulary semantic segmentation performance. Code is available at https://github.com/2btlFe/GLA-CLIP.

Abstract

Explore similar work

Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation

Direct Segmentation without Logits Optimization for Training-Free Open-Vocabulary Semantic Segmentation

dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3