While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate at the patch or region level, where annotation requirements remain high due to extensive pixel labeling. Extending ViT-based AL to the pixel-level, low-budget regime is uniquely challenging: with as little as one pixel query per image, a method must simultaneously identify the most informative locations and train a reliable decoder head from an extremely sparse signal. We introduce TEAL (Token-space Efficient Active Learning), the first pixel-level AL framework built around ViT representations. TEAL uses a frozen ViT backbone and performs diversity-based selection directly on its native token lattice, avoiding the artifacts introduced by interpolating token embeddings to dense pixel resolutions. Our framework applies a two-level MaxHerding strategy over multi-layer token descriptors to select representative candidates, followed by margin-based uncertainty refinement on a finer decoder grid. Across CamVid, Cityscapes, ADE20K, and Pascal Context, TEAL consistently outperforms previous baselines under extreme label scarcity, improving over the strongest baseline by up to +24.36 mIoU with a single labeled pixel (px) per image, and reaching up to 97% of fully supervised performance with just 10 px. To rigorously establish the source of the gains, we further show that the improvements persist when all baselines are equipped with the same DINOv3 backbone, demonstrating that coupling token-space selection with uncertainty refinement on the decoder grid makes ViT token geometry an effective foundation for extremely low-budget active segmentation.
Figures & tables
Figure 1: Token-space acquisition avoids pixel-space interpolation artifacts. (a) Compares query locations from TEAL and the two-stage eDALD baseline Kim et al. (2026) , with both methods using the same frozen DINOv3 backbone for a controlled comparison. Pixel-space selection often concentrates near boundaries, where upsampled token features mix neighboring descriptors, while TEAL selects more representative regions by operating on the native token lattice. As (c) shows that this leads to cleaner segmentation maps under the same sparse pixel-label budget.
Figure 2: One round of TEAL active learning. A Mini-DPT head is trained on the current labeled pixel set Lt using frozen multi-layer DINOv3 token features. Stage 1 performs two-level MaxHerding in the native token space: local herding selects k representative unlabeled tokens per image, and global herding compresses their union into the dataset-level candidate set Ct . Stage 2 refines these token candidates on the full 64×64 Mini-DPT decoder grid; in the figure, this is visualized locally by subdividing each selected candidate token into a 4×4 block of decoder cells. Margin uncertainty is evaluated only on valid decoder cells that overlap Ct and have unlabeled center pixels. The center pixels of the most uncertain valid cells are queried and added to Lt+1 for the next round.
Dataset
Method
Budget – average pixels per image
Ceiling
1
10
2562
CamVid
PixelPick
30.56 ± 1.58
37.51 ± 0.66
42.85
2-Stage eDALD
34.15 ± 0.72
40.20 ± 0.65
58.27
Teal
49.54 ± 0.37 (+15.39)
55.93 ± 0.11 (+15.73)
62.02
Cityscapes
PixelPick
25.96 ± 0.16
31.03 ± 0.33
32.38
2-Stage eDALD
30.40 ± 0.41
34.60 ± 0.47
43.89
Table 1: Performance across datasets and labeling budgets . Green values indicate absolute improvement over the strongest baseline. The Ceiling column reports each method’s fully supervised mIoU when trained with dense labels using its own backbone. PixelPick uses a CNN-based DeepLabV3 backbone ( Chen et al., 2017 ) , 2-Stage eDALD uses DDPM ( Ho et al., 2020 ) features, and TEAL uses DINOv3-base ( Siméoni et al., 2025 ) features with our token/grid-based segmentation head. Results are reported as mean ± standard deviation.
Dataset
Ceiling
Method
Budget – average pixels per image
% of Sup.
2562
1
10
@10 px
CamVid
62.02
PixelPick
47.07 ± 0.28
53.79 ± 0.27
86.7
2-Stage eDALD
45.88 ± 1.68
53.98 ± 0.57
87.0
Teal
49.54 ± 0.37 (+3.66)
55.93 ± 0.11 (+1.95)
90.2
Cityscapes
49.91
PixelPick
40.21 ± 0.27
43.28 ± 0.17
86.7
2-Stage eDALD
40.55 ± 0.41
41.68 ± 0.20
83.5
Table 2: Matched-backbone comparison across datasets and labeling budgets . Green values indicate absolute improvement over the previous two-stage method ( Kim et al., 2026 ) . Ceiling reports the fully supervised mIoU obtained using dense labels and the DINOv3 backbone, shared by all methods. The final column reports the fraction of the fully supervised performance recovered at the 10-pixel budget. All methods use DINOv3 features for a matched-backbone comparison.
Figure 3: Decomposing the gap from matched-backbone eDALD to Teal on CamVid. Solid segments show the change from the previous configuration ( → replaces, + adds; red marks a loss). Pixel herding helps at one pixel per image but hurts at ten, while token-to-cell refinement and token-space herding help at both.
Figure 4: Results at 512×512 resolution. Matched-backbone comparison of eDALD DINO, pixel herding (eDALD DINO with margin sampling), and Teal at 1 and 10 pixels per image. Shaded bands show the standard deviation over three seeds.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Local budget k
mIoU
5
46.15 ±1.53
10
49.02 ±2.10
20 (default)
49.54 ±0.37
40
49.43 ±0.43
80
48.40 ±0.75
Appendix
Table 3: Sensitivity to the local budget k .
Global retention ratio
mIoU
0.25
49.29 ±0.78
0.50 (default)
49.54 ±0.37
0.75
50.43 ±0.11
1.00
48.80 ±1.21
Appendix
Table 4: Sensitivity to the global retention ratio.
Scale factor s
mIoU
1.0
48.48 ±0.78
2.0 (default)
49.54 ±0.37
3.0
49.45 ±0.95
Appendix
Table 5: Sensitivity to the RBF bandwidth scale factor s , where σ=s⋅ median pairwise token distance.
Herding layers
mIoU
Shallow [0,1,2,3]
30.30 ±3.85
Balanced - Mini-DPT default [2,5,8,11] [ Yang et al., 2025 ]
48.80 ±0.58
Late [5,7,9,11]
50.21 ±1.29
Deep [8,9,10,11] (ours)
49.54 ±0.37
Appendix
Table 6: Sensitivity to the DINOv3 layers used for herding.
Dataset
Method
Budget – average pixels per image
1
10
CamVid
Pixel herding
47.34 ± 0.29
53.38 ± 0.50
Teal
49.54 ± 0.37 (+2.20)
55.93 ± 0.11 (+2.55)
Cityscapes
Pixel herding
40.61 ± 0.34
41.80 ± 0.31
Teal
42.25 ± 0.37 (+1.64)
43.79 ± 0.26 (+1.99)
ADE20K
Pixel herding
32.67 ± 0.39
38.31 ± 0.54
Appendix
Table 7: Pixel herding vs. Teal under matched backbones. mIoU at 1 and 10 pixels per image, mean ± std over three seeds. Green values indicate absolute improvement over pixel herding.
Dataset
Method
Budget – average pixels per image
1
10
CamVid
eDALD DINO
49.86 ± 1.15
57.82 ± 1.28
Pixel herding
51.37 ± 1.70
58.07 ± 1.02
Teal
55.49 ± 1.40
61.38 ± 0.96
Cityscapes
eDALD DINO
53.53 ± 0.89
56.76 ± 0.10
Pixel herding
53.07 ± 0.44
54.93 ± 1.07
Appendix
Table 8: Results at 512×512 resolution. Matched-backbone comparison at 1 and 10 pixels per image, mean ± std over three seeds. Pixel herding is eDALD DINO with margin sampling in place of eBALD.
Method
On token edge (0 px)
Within 1 px
Within 2 px
Pixel herding
60.1%
87.8%
97.0%
Teal
0.0%
40.7%
69.6%
Uniform random
23.4%
43.8%
60.9%
Appendix
Table 9: Fraction of selected pixels lying on or near token boundaries.
Label set (matched backbone)
Rare-class share
Multi-label tokens
Final mIoU
Teal , 1 px
14.5%
2.0%
49.54 ± 0.37
eDALD DINO, 1 px
10.1%
0.0%
45.88 ± 1.68
Teal , 10 px
12.7%
29.8%
55.93 ± 0.11
eDALD DINO, 10 px
7.5%
1.8%
53.98 ± 0.57
Appendix
Table 10: Query allocation under matched backbones on CamVid. Rare-class share is the fraction of queries placed on rare classes; multi-label tokens is the fraction of labeled tokens that received more than one label.
Figure 5: Per-class IoU and selection mass at 10 pixels per image. Comparison of token-level and pixel-level herding using DINOv3 features on CamVid. The top panel reports the per-class IoU difference between token and pixel herding (positive values favor token herding), and the bottom panel shows the fraction of selected queries assigned to each class. Token herding improves IoU for most classes, with the largest gains on lower-frequency, spatially localized categories such as sign symbol, fence, pedestrian, and bicyclist. Pixel herding is slightly better only on large surface classes such as road and pavement.
Per-image latency (ms)
Wall-Clock Runtime
Method
Stage 1
Train step
Stage 2
10-round AL run
eDALD-DDPM
184.3
112.7
555.7
1h33min
TEAL (ours)
7.2
6.5
5.6
0h23min
Appendix
Table 11: Compute cost of CamVid run across different methods. Per-image latency for Stage 1 herding, head training step, Stage 2 uncertainty, and the total wall-clock for a full 10-round run.
Component
Original Mini-DPT
Mini-DPT (ours)
Branch resampling
Bilinear resize, then learned 3×3 conv
Transposed conv (kernel = stride) for ×4 , ×2 ; identity for ×1 ; strided 3×3 conv for ×21
Resampling parameters
589,824
475,136
Pyramid scales
[8,4,2,1]
[4,2,1,21]
Decoder grid
128×128 (2-px cells)
64×64 (4-px cells)
Appendix
Table 12: Architectural differences between the original Mini-DPT decoder and our Mini-DPT head. Grid sizes are for a 256×256 input.
Figure 6: Qualitative segmentation maps at the final active-learning round. We compare Teal against eDALD with DDPM features and a matched-backbone eDALD using DINOv3 features. Teal produces cleaner object regions and fewer boundary artifacts, consistent with the quantitative gains in the low-budget setting.
Foundational Vision Transformers (ViTs) have limited effectiveness in tasks requiring fine-grained spatial understanding, due to their fixed pre-training resolution and inherently coarse patch-level representations. These challenges are especially pronounced in dense prediction scenarios, such as open-vocabulary segmentation with ViT-based vision-language models, where high-resolution inputs are essential for accurate pixel-level reasoning. Existing approaches typically process large-resolution images using a sliding-window strategy at the pre-training resolution. While this improves accuracy through finer strides, it comes at a significant computational cost. We introduce SPAR: Single-Pass Any-Resolution ViT, a resolution-agnostic dense feature extractor designed for efficient high-resolution inference. We distill the spatial reasoning capabilities of a finely-strided, sliding-window teacher into a single-pass student using a feature regression loss, without requiring architectural changes or pixel-level supervision. Applied to open-vocabulary segmentation, SPAR improves single-pass baselines by up to 10.5 mIoU and even surpasses the teacher, demonstrating effectiveness in efficient, high-resolution reasoning. Code: https://github.com/naomikombol/SPAR
Naomi Kombol, Ivan Martinović, Siniša Šegvić +1
Faculty of Electrical Engineering and Computing, University of Zagreb · VRG, Faculty of Electrical Engineering, Czech Technical University in Prague
We present ARTA, a mixed-resolution coarse-to-fine vision transformer for efficient dense feature extraction. Unlike models that begin with dense high-resolution (fine) tokens, ARTA starts with low-resolution (coarse) tokens and uses a lightweight allocator to predict which regions require more fine tokens. The allocator iteratively predicts a semantic (class) boundary score and allocates additional tokens to patches above a low threshold, concentrating token density near boundaries while maintaining high sensitivity to weak boundary evidence. This targeted allocation encourages tokens to represent a single semantic class rather than a mixture of classes. Mixed-resolution attention enables interaction between coarse and fine tokens, focusing computation on semantically complex areas while avoiding redundant processing in homogeneous regions. Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs, and delivers competitive performance on Cityscapes at markedly lower compute. For example, ARTA-Base attains 54.6 mIoU on ADE20K in the ~100M-parameter class while using fewer FLOPs and less memory than comparable backbones.
David Hagerman, Roman Naeem, Erik Brorsson +2
Chalmers University of TechnologyChalmers University of technology, 412 96 Gothenburg, Sweden
Vision Transformers (ViTs) are strong backbones for semantic segmentation, but their computational cost limits deployment. Recent token compression methods for efficient transformer-based segmentation reduce this cost by decreasing the number of tokens. However, existing evaluations primarily focus on low-to-moderate compression, leaving their behavior under aggressive compression and corrupted inputs unclear. Meanwhile, structural pruning provides an orthogonal route to efficiency by removing redundant components in the ViT architecture, but is rarely compared to token compression under a unified protocol. To bridge this gap, we benchmark representative token compression and structural pruning methods for ViT-based semantic segmentation under matched FLOPs on ADE20K and Cityscapes, together with their common-corruption variants ADE20K-C and Cityscapes-C. Our results reveal a consistent trend on both clean and corrupted inputs: token compression is highly effective at mild reductions but degrades sharply when compression becomes severe, consistent with substantial information loss from overly aggressive token reduction. In contrast, structural pruning exhibits a smoother degradation curve and is more stable at high compression. Motivated by these findings, we study a prune-then-merge pipeline that applies moderate token compression on top of a moderately pruned backbone. At comparable FLOPs, this combined strategy consistently achieves a better accuracy-robustness trade-off at high compression, offering a practical recipe for deployment-oriented ViT segmentation. Code is available at https://github.com/phatnguyencs/vit-seg-compression.
Tien-Phat Nguyen, Ngai-Man Cheung
Temasek Laboratories, Singapore University of Technology and Design, Singapore