SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentation

Naomi Kombol; Ivan Martinović; Siniša Šegvić; Giorgos Tolias

cs.CVApr 2, 2026

SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentation

Authors: Naomi Kombol, Ivan Martinović, Siniša Šegvić, Giorgos Tolias

Organizations: Faculty of Electrical Engineering and Computing, University of Zagreb · VRG, Faculty of Electrical Engineering, Czech Technical University in Prague

Abstract

Foundational Vision Transformers (ViTs) have limited effectiveness in tasks requiring fine-grained spatial understanding, due to their fixed pre-training resolution and inherently coarse patch-level representations. These challenges are especially pronounced in dense prediction scenarios, such as open-vocabulary segmentation with ViT-based vision-language models, where high-resolution inputs are essential for accurate pixel-level reasoning. Existing approaches typically process large-resolution images using a sliding-window strategy at the pre-training resolution. While this improves accuracy through finer strides, it comes at a significant computational cost. We introduce SPAR: Single-Pass Any-Resolution ViT, a resolution-agnostic dense feature extractor designed for efficient high-resolution inference. We distill the spatial reasoning capabilities of a finely-strided, sliding-window teacher into a single-pass student using a feature regression loss, without requiring architectural changes or pixel-level supervision. Applied to open-vocabulary segmentation, SPAR improves single-pass baselines by up to 10.5 mIoU and even surpasses the teacher, demonstrating effectiveness in efficient, high-resolution reasoning. Code: https://github.com/naomikombol/SPAR

Figures & tables

Panel of Figure 1 — Figure 1: Performance vs . inference time trade-off. Comparison between the pre-trained SigLIP2 – ViT-B-16 with single-pass or sliding-window (stride value reported in text labels) inference, and our single-pass SPAR-distilled model. The teacher has sliding windows of size $512\times 512$ with stride 24. We report average performance across six datasets along with average inference time for an 1024 $\times$ 2048 image. *ND: stride not divisible by patch size.

Figure 2: Overview of SPAR. During training, the teacher branch uses a frozen foundational vision encoder to generate feature maps via a sliding-window process followed by stitching. Stitching refers to merging the feature maps of overlapping windows into a unified representation aligned with the original image layout. The student branch, initialized from the same pre-trained weights, trains to match the teacher’s output using efficient single-pass inference. At inference time, the student model enables fast and accurate segmentation at diverse resolutions and aspect ratios using a single forward pass.

Models	Inference	Upsampler	Voc21	Voc20	CS	ADE	C60	C59	$\text{Mean}_{6}$
SigLIP2 [ 41 ] – ViT-B-16
NaFlex [ 41 ]	single-pass	bilinear	35.8	66.6	22.8	16.2	23.7	25.2	31.7
Pre-trained	sliding-window	bilinear	45.0	77.5	38.4	21.8	30.7	34.0	41.2		$\Rsh$
Pre-trained	single-pass	bilinear	36.1	71.3	23.5	16.8	24.5	26.1	33.1	$\Rsh$
SPAR	single-pass	bilinear	47.3	81.5	38.4	23.4	33.8	37.2	43.6	+10.5	+2.4
LPOSS [ 36 ] +Pre-tr.	single-pass	bilinear	46.1	89.6	34.5	19.9	30.9	35.2	42.7	$\Rsh$

Table 1: Performance evaluation via mIoU on six datasets. We report results for 3 backbones comparing SPAR to the pre-trained model and NaFlex [ 41 ] . Sliding-window inference uses stride

s=24

, which corresponds to the teacher model. We combine SPAR with LPOSS [ 36 ] to show complementarity with current open-vocabulary semantic segmentation methods. Prediction upsampling is either with bilinear interpolation or AnyUp [ 45 ] . Labels as follows: CS: Cityscapes, C60: Context60, C59: Context59. Best results per section are bolded .

Panel of Figure 3 — Figure 3: Performance (left) and inference time (right) vs . image resolution. We compare SPAR to the pre-trained single-pass and sliding-window models, and single-pass NaFlex. SigLIP2 – ViT-B-16 with native resolution $512\times 512$ , and $K=512$ used. Results on Cityscapes with resolution corresponding to the denoted area. Resolution 512 does not qualify for $K=512$ . Single-pass models, including SPAR, have the same inference time. $\dagger$ : larger resolution used in training (short side up to 2560 pixels instead of 2048). ALL: training the whole network, otherwise only the last 2 blocks.

Panel of Figure 4 — Figure 4: Performance vs. resolution for DINOv3.txt .

Training Configuration	#params (M)	CS	ADE	$\text{Mean}_{6}$
Pre-trained model
Single-pass		23.5	16.8	33.1
Sliding-windows $s=24$		38.4	21.8	41.2
SPAR model
ALL params	93.5	38.5	22.8	42.5
ALL params $\dagger$	93.5	40.5	22.7	42.0

Table 2: Effect of fine-tuning different network elements . We show mIoU on Cityscapes (CS), ADE20K, and the mean over six datasets for SPAR-SigLIP2 – ViT-B-16 . Fine-tuning only the last two transformer blocks, bolded for emphasis, yields the best overall performance. #params: number of fine-tuned parameters.

\dagger

: larger resolution used in training. Best result is bolded , second best is underlined .

Training Set	CS	ADE	$\text{Mean}_{6}$
Pre-trained model
Sliding-window $s=24$	38.4	21.8	41.2
SPAR model
ADE20k+CS+VOC	39.3	23.6	43.4
ADE20k	37.4	23.5	43.0
SA-1B 1.25k (5%)	35.4	21.9	41.1

Table 3: Impact of distillation data composition and scale . We show mIoU on Cityscapes (CS), ADE20K, and the mean over six datasets for SPAR-SigLIP2 – ViT-B-16 when training on different unlabeled sources. Comparable results across all settings show that SPAR does not rely on in-domain data, and performance saturates with 25k distillation images. Best result is bolded , second best is underlined .

Panel of Figure 5 — Figure 5: Qualitative segmentation and PCA analysis results. We show that SPAR yields less noisy and spatially smoother predictions than the teacher, further improved by LPOSS [ 36 ] . Images are from VOC21 [ 12 ] , Context60 [ 30 ] , ADE20K [ 53 ] and Cityscapes [ 8 ] .

SigLIP2 – ViT-B-16	VOC21	CS	ADE
Linear Probe
Pre-trained single-pass	67.1	54.1	36.0
SPAR (Last 2 blocks)	70.2	56.2	38.1
SPAR (ALL)	68.9	66.7	36.5
Hummingbird
Pre-trained single-pass	31.9	26.7	13.5

Table 4: SPAR on vision-only dense prediction tasks . We show mIoU for linear probing and Hummingbird [ 2 ] segmentation on VOC21, Cityscapes (CS) and ADE20K. Oracle panoptic segmentation [ 29 , 33 ] shows PQ on the latter two datasets. Improvement across all benchmarks attests to SPAR’s benefit to visual representation quality. Best result is bolded .

Training Configuration	$\mathrm{Reported\>Mean}_{6}$	$\overline{\mathrm{Mean}_{6}}$ $\pm$ $\sigma_{\mathrm{Mean}_{6}}$
SPAR model
All params	42.5	42.3 $\pm$ 0.27
Last block	43.3	43.2 $\pm$ 0.14
Last 2 blocks (default)	43.6	43.6 $\pm$ 0.04
Last 3 blocks	42.9	43.1 $\pm$ 0.15
Patch projection	38.0	38.1 $\pm$ 0.11

Table 5: Effect of fine-tuning different network elements (mean and std performance across three seeds) . We show mean mIoU over six datasets, along with the mean and standard deviation across three independent runs, for various training configurations of SPAR-SigLIP2 – ViT-B-16 . Fine-tuning only the last two transformer blocks, bolded for emphasis, achieves the best overall performance and maintains the second smallest standard deviation.

Training Set	$\mathrm{Reported\>Mean}_{6}$	$\overline{\mathrm{Mean}_{6}}$ $\pm$ $\sigma_{\mathrm{Mean}_{6}}$
SPAR model
ADE20k+CS+VOC	43.4	43.6 $\pm$ 0.27
ADE20k	43.0	43.1 $\pm$ 0.12
SA-1B 1.25k (5%)	41.1	41.0 $\pm$ 0.09
SA-1B 2.5k (10%)	42.1	41.9 $\pm$ 0.31
SA-1B 12.5k (50%)	43.2	43.0 $\pm$ 0.15

Table 6: Impact of distillation data composition and scale (mean and std performance across three seeds) . We show mean mIoU over six datasets, along with the mean and standard deviation across three independent runs, for SPAR-SigLIP2 – ViT-B-16 when training on different unlabeled sources. Comparable results across all settings show SPAR does not rely on in-domain data, and performance saturates at 25k distillation images.

SigLIP2 – ViT-B-16	VOC21	CS	ADE
Linear Probe - native resolution
Pre-trained single-pass	67.1	54.1	36.0
SPAR (Last 2 blocks)	70.2	56.2	38.1
SPAR (All)	68.9	66.7	36.5
Linear Probe - repository resolution
Pre-trained single-pass	71.2	57.0	37.7

Table 7: SPAR linear probing performance using different resolutions . We show mIoU for linear probing on VOC21, Cityscapes (CS) and ADE20K when utilizing native image resolution vs default resizing of [ 18 ] for training and evaluation. Improved results across all benchmarks show SPAR’s benefit to visual representation quality. Best result is bolded .

GT Panoptic	ADE				CS
	$512^{2}$	$640^{2}$	$800^{2}$	$1024^{2}$	$1024\times 2048$ (native)
Pre-trained single pass	33.9	34.3	33.6	31.0	31.5
SPAR Last 2 blocks	34.6	35.2	34.0	30.7	28.6
SPAR All parameters	37.9	39.7	41.1	39.0	52.4

Table 8: SPAR mask-oracle panoptic segmentation results for different resolutions . We show PQ for panoptic segmentation on Cityscapes (CS) and ADE20K when using ground truth masks for pooling features [ 29 , 33 ] . Improved results across resolution show SPAR-All’s benefit to visual representation quality. Best result is bolded .

SigLIP2 – ViT-B-16	Voc21	Voc20	CS	ADE	C60	C59	$\text{Mean}_{6}$
Pre-trained single-pass	36.1	71.3	23.5	16.8	24.5	26.1	33.1
Multi-resolution distill.	38.0	76.8	26.3	19.0	26.8	29.0	36.0
SPAR w/ Teach. $s=16$	43.8	77.6	36.2	21.3	31.0	34.0	40.6
SPAR w/ Teach. $s=24$	47.3	81.5	38.4	23.4	33.8	37.2	43.6
Class sim. distill. ADE	48.0	75.4	37.1	23.0	32.8	36.4	42.1
Class sim. distill. CS	44.0	64.8	38.3	19.8	26.0	31.8	37.4

Table 9: SPAR compared to other distillation schemes . We show mIoU over six datasets, along with the mean. The results underscore the importance of SPAR using a sliding-window teacher with a stride allowing for non-overlapping patches between windows as well as aligning at the visual feature level. Distilling class similarities overfits to the utilized classes. Best results are bolded .

Panel of Figure 6 — Figure 6: Qualitative segmentation results. We show SPAR yields more coherent and smoother predictions than the teacher. Images are from ADE20K [ 53 ] (first three columns) and Cityscapes [ 8 ] (last two columns).

Panel of Figure 7 — Figure 7: Feature quality visualization via PCA. We project features onto the same PCA basis, computed from the teacher’s sliding-window representations, to enable a consistent and semantically aligned comparison across models. SPAR improves inter-object separability and smooths intra-object consistency without losing fine details. The former is visible in the clearer distinction between people and the wall behind them (1st column) or pedestrians on the street (4th column), while improved intra-object consistency is most apparent in the interior of the camper (3rd column). Images are from ADE20K [ 53 ] (first three columns) and Cityscapes [ 8 ] (last two columns).

Explore similar work

Vision Transformers Need Better Token Interaction
May 22, 2026Linxiang SuVisual Representation Learning Vision Transformer
University of Szeged
ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
Mar 27, 2026David Hagerman, Roman Naeem, Erik Brorsson +2Efficient ViTs Vision Transformer
Chalmers University of TechnologyChalmers University of technology, 412 96 Gothenburg, Sweden
Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?
Feb 26, 2026Tilemachos Aravanis, Vladan Stojnić, Bill Psomas +2Open-Vocabulary Semantic Segmentation Test-Time Adaptation of VLMs
VRG, FEE, Czech Technical University in Prague · University of Crete · Archimedes, Athena RC +1

May 22, 2026J/K move · Enter open · S save

Vision Transformers Need Better Token Interaction

Linxiang Su

University of Szeged

Visual Representation Learning Vision Transformer Sparse Attention Semantic Segmentation

Open paper AskPDF arXiv

Vision Transformers (ViTs) can learn strong image-level representations while their patch representations become less effective for dense prediction during prolonged training. We revisit this dense degradation phenomenon and argue that it is not fully explained by high-norm artifacts alone. Instead, we characterize semantic diffusion: an optimization shortcut in which global semantic information spreads through patch tokens beyond what is locally justified. Our analysis shows that dense representation quality is not captured by locality alone: shallow features can remain better aligned with foreground regions yet underperform deeper features, and [CLS] features remain complementary for dense prediction. These observations suggest that the goal should not be to remove global context, but to make token interactions more selective. We therefore study sparse attention as a minimal intervention, replacing softmax attention with entmax-1.5 while preserving global token connectivity. On DINOv1 ViT-S/16 trained for 200 epochs on ImageNet-1K, this change preserves ImageNet linear probing accuracy and substantially improves semantic segmentation performance: VOC mIoU increases from 42.80 to 48.78, ADE20K from 19.85 to 21.97, and Cityscapes from 36.79 to 37.87. These results suggest that selective token mixing is a simple and effective bias for improving dense ViT representations.

Abstract

Explore similar work

Vision Transformers Need Better Token Interaction

ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction

Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?