In-context segmentation (ICS) aims to segment arbitrary concepts, e.g., objects, parts, or personalized instances, given one annotated visual examples. Existing work relies on (i) fine-tuning vision foundation models (VFMs), which improves in-domain results but harms generalization, or (ii) combines multiple frozen VFMs, which preserves generalization but yields architectural complexity and fixed segmentation granularities. We revisit ICS from a minimalist perspective and ask: Can a single self-supervised backbone support both semantic matching and segmentation, without any supervision or auxiliary models? We show that scaled-up dense self-supervised features from DINOv3 exhibit strong spatial structure and semantic correspondence. We introduce INSID3, a training-free approach that segments concepts at varying granularities only from frozen DINOv3 features, given an in-context example. INSID3 achieves state-of-the-art results across one-shot semantic, part, and personalized segmentation, outperforming previous work by +7.5 % mIoU, while using 3x fewer parameters and without any mask or category-level supervision. Code is available at https://github.com/visinf/INSID3 .
Figures & tables
Figure 2: Region-level grouping from DINOv3. Each pair shows an input image (left) and the corresponding clustering map (right) obtained by applying agglomerative clustering to dense DINOv3 features. The resulting clusters delineate coherent object- and part-level regions, providing a structured decomposition of the scene.
Figure 3: Overview of INSID3. We leverage the semantic and spatial structure of DINOv3 to perform in-context segmentation without training or model composition. Dense features from the reference and target images are first debiased to suppress positional bias, improving cross-image matching. The target is then decomposed into coherent regions through agglomerative clustering, providing a structured representation. We retain candidate clusters that match the reference through backward correspondence in the debiased space; a reference prototype derived from the annotated region anchors the seed cluster via cross-image similarity. Finally, we combine cross-image similarity, capturing semantic alignment, with self-similarity, measuring the affinity of each cluster to the seed, to form the final mask from the seed.
Figure 4: Positional bias in DINOv3 features. For both region 4 and keypoint 4 prompts, similarity maps computed with the original DINOv3 features show structured activations aligned with the reference coordinates, independent of semantics. Our debiased features mitigate this behavior. 4 PCA of features from images with low semantic complexity ( e.g . , noise, flat textures) reveals a stable low-dimensional positional subspace underlying this bias.
Table 1: Comparison of INSID3 (mIoU in %, ↑ ) on one-shot semantic, part, and personalized segmentation. State-of-the-art methods are grouped into task-specific fine-tuning and training-free approaches. Previous training-free methods rely on SAM, pre-trained with mask-level supervision, whereas INSID3 uses only frozen self-supervised DINOv3 features. Gray indicates the model was trained on the corresponding train split of the dataset; best results bold , 2 nd best underlined . † denotes a GF-SAM variant using DINOv3 features.
Figure 5: Comparison of INSID3 with GF-SAM [ 69 ] and SegIC [ 41 ] on one-shot semantic (left) , part (top right) , and personalized (bottom right) segmentation. SegIC performs well in-domain but struggles to generalize across domains and part granularity, reflecting its limited flexibility beyond the training distribution. GF-SAM , relying on the strong segmentation priors of SAM [ 33 ] , produces high-quality masks; however, the decoupled mechanism between correspondence and segmentation often leads to over- or under-segmentation. INSID3 , despite relying solely on self-supervised features, achieves precise localization and competitive mask quality.
Small
Base
Large
T
original
debias
original
debias
original
debias
0.05
26.8
27.9
29.2
32.3
32.7
33.6
0.10
43.8
45.7
45.0
50.0
50.6
52.0
0.15
53.2
55.6
54.0
59.9
60.3
62.1
0.20
59.8
62.6
59.8
66.4
66.4
68.6
Table 2: Semantic correspondence on SPair-71k (PCK@ T in %, ↑ ). Comparison across DINOv3 backbones, w/ and w/o debiasing.
Figure 6: Qualitative examples on SPair-71k with DINOv3-L.
Figure 7: INSID3 hyperparameters. We study the effect of the debiasing rank s (left) and the clustering granularity τ (right) on semantic (COCO-20 i ) and part (PASCAL-Part) segmentation.
Table 3: Effect of clustering and aggregation. We compare our approach with tuned baselines on COCO-20 i and PASCAL-Part (mIoU %, ↑ ). All thresholds (0.55) and clustering granularities ( \tau{=}\text{0.5} , \tau{=}\text{0.6} ) are tuned to optimal values for fair comparison.
Figure 8: Positional bias in DINOv2 vs . DINOv3. We visualize the similarity map between a reference keypoint and all target patches using Eq. 16 . Darker purple regions indicate low similarity, while brighter yellow regions indicate high similarity. DINOv3 exhibits strong coordinate-aligned artifacts, with bright responses appearing at the same absolute spatial location as the reference keypoint. Our debiasing projection suppresses these spurious activations. We observe that DINOv2 shows much weaker positional bias.
Table 4: Semantic correspondence on SPair-71k. PCK@ T (%, ↑ ) for DINOv2 and DINOv3 using original and debiased features. Debiasing produces marginal improvements for DINOv2, but yields substantial gains for DINOv3.
Method
mIoU
No debiasing
54.5
Augmentations, 4 views
55.7
Augmentations, 12 views
56.5
Ours (SVD from 1 noise image)
57.6
SVD from 5 noise images
57.7
SVD from 10 noise images
57.7
Table 5: Different debiasing strategies. Comparison of training-free strategies for feature debiasing on COCO-20 i using mIoU (in %, ↑ ). Augmentations are a viable alternative but require multiple forward passes . In contrast our method adds no overhead, except for a single matrix multiplication, since the debiasing projection is pre-computed offline and stored.
Figure 9: Debiased features preserve semantic structure. We compute PCA bases on COCO training images for (i) the original DINOv3 features and (ii) our debiased features. We then project both representations to d principal components and evaluate semantic correspondence ( PCK@0.10 ) on SPair-71k. Across all d , Debias + PCA consistently outperforms PCA-original, indicating that the removed directions primarily encode positional bias. After debiasing, the remaining variance concentrates around semantic structure, yielding better results when compressed via PCA.
Hyperparameter
Value
τ
(clustering sensitivity)
0.6
α
(aggregation threshold)
0.2
s
(debias rank)
500
Table 6: Hyperparameter overview. INSID3 uses only three hyperparameters, all fixed across datasets, for both semantic, part, and personalized segmentation.
Table 7: Comparison of INSID3 (mIoU in %, ↑ ) on 5-shot semantic and part segmentation. Models are provided with 5 contextual examples and tasked with segmenting the annotated concept in the target image. INSID3 scales effectively to multiple references, achieving robust performance across domains. All hyperparameters are reused from the 1-shot setting without any tuning, highlighting the versatility of our approach. Gray indicates the model was trained on the corresponding train split of the dataset; best results bold , 2 nd best underlined . † denotes a GF-SAM variant using DINOv3 features.
DINOv3
DINOv2
Franca
Perception Enc.
Stable Diff.
57.6
45.1
39.2
48.4
33.2
Table 8: Backbone analysis for INSID3. We report mIoU (in %, ↑ ) on COCO-20 i of INSID3 with different backbone features. DINOv3 features lead to the best segmentation accuracy.
Method
mIoU
Training free : Mask-supervised pre-training
Segment Anything 3 (video-style propagation)
26.7
Segment Anything 3 (image concatenation)
52.9
Training free : Unsupervised pre-training
INSID3 (ours)
57.6
Table 9: Comparison with SAM3. We compare INSID3 to Segment Anything 3 (SAM 3) on COCO-20 i using mIoU (in %, ↑ ). We evaluate two adaptations of SAM 3: (i) a video-style formulation, where reference and target are treated as consecutive frames and the mask is propagated, and (ii) an image concatenation strategy, where images are combined horizontally and prompted.
Table 10: Absence of the reference concept. Percentage of correct empty-mask predictions (%, ↑ ) when the target image does not contain the prompted object (COCO, 4 000 pairs). n.a. : methods that always return a non-empty mask ( i.e . , 0% in practice).
Table 11: Computational analysis of INSID3 vs . previous methods in milliseconds (ms, ↓ ). Inference time is measured for a single in-context example ( i.e . , example and target image). Runtime is measured on a single RTX 4090.
Component
Runtime ↓
Encoder forward + debiasing
78ms
Compute similarity + cluster scoring
3ms
Clustering
166 ms
CRF refinement
55 ms
Total inference time
302ms
Table 12: Detailed computational analysis of INSID3. We report inference time of each component in milliseconds (ms, ↓ ). Inference time is measured for a single in-context example. Runtime measured on a single RTX 4090.
Low resolution ← Runtime of INSID3 → High resolution
512px
720px
896px
1024px
1440px
1600px
1760px
93ms
180ms
230ms
302ms
689ms
930ms
1102ms
Table 13: Computational analysis for different resolutions. We report inference time of INSID3 for different resolution in milliseconds (ms, ↓ ). Inference time is measured for a single in-context example. Runtime measured on a single RTX 3090.
Figure 10: Qualitatives results with INSID3 . Each pair shows the reference image with its annotated region ( light blue mask ) and the predicted mask on the target ( purple ) . INSID3 handles a wide range of semantic granularities, from full objects to fine-grained parts, and generalizes robustly across diverse domains, including aerial, marine, medical, and everyday scenes.
In-Context Segmentation (ICS) aims to precisely segment arbitrary semantic concepts, such as objects or parts, given one or a few annotated visual exemplars. In this paper, we revisit ICS from a more classical segmentation perspective, viewing it as a coarse-to-fine progressive refinement process. Rather than directly predicting the final mask through reference-query matching, we progressively refine the segmentation from coarse and ambiguous foreground responses to precise and complete foreground structures. Building upon this perspective, we propose a training-free in-context segmentation framework, termed FoRIS. Specifically, FoRIS consists of three key stages: Foreground Purification, Foreground Localization, and Foreground Consolidation, which progressively suppress background distractions, localize discriminative target regions, and recover complete foreground structures through semantic aggregation. Experimental results demonstrate that FoRIS achieves SOTA performance across semantic and part segmentation tasks, with average improvements of 4.5 and 4.8 mIoU points over existing approaches in the 1-shot and 5-shot settings, respectively. Code: https://github.com/Xi-Mu-Yu/FoRIS.
Ming Hu, Jianfu Yin, Mingyu Dou +5
Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xi’an Jiaotong University +1
Accurately segmenting objects without any manual annotations remains one of the core challenges in computer vision. In this work, we introduce Selfment, a fully self-supervised framework that segments foreground objects directly from raw images without human labels, pretrained segmentation models, or any post-processing. Selfment first constructs patch-level affinity graphs from self-supervised features and applies NCut to obtain an initial coarse foreground--background separation. We then introduce Iterative Patch Optimization (IPO), a feature-space refinement procedure that progressively enforces spatial coherence and semantic consistency through iterative patch clustering. The refined masks are subsequently used as supervisory signals to train a lightweight segmentation head with contrastive and region-consistency objectives, allowing the model to learn stable and transferable object representations. Despite its simplicity and complete absence of manual supervision, Selfment sets new state-of-the-art (SoTA) results across multiple benchmarks. It achieves substantial improvements on Fmax over previous unsupervised saliency detection methods on ECSSD (+4.0%), HKUIS (+4.6%), and PASCAL-S (+5.7%). Moreover, without any additional fine-tuning, Selfment demonstrates remarkable zero-shot generalization to camouflaged object detection tasks (e.g., 0.910Sm on CHAMELEON and 0.792Fβω on CAMO), outperforming all existing unsupervised approaches and even rivaling the SoTA fully supervised methods.
Remote sensing semantic segmentation is hindered by costly pixel-level annotations, motivating training-free open-vocabulary methods. Recently, the recent release of DINOv3 brings DINO.txt, which equips the standalone DINO backbone with image-text contrastive learning and thus opens up the possibility of open-vocabulary segmentation. We propose DinoSplat-OV, a training-free framework that adapts DINOv3 to remote sensing without fine-tuning or additional pretraining. Targeting the dense distribution, multi-scale nature, and large size of remote sensing imagery, we design two core modules. Its Text-aware Laplacian Propagation module de-noises patch-level predictions by combining textual semantic affinities with local visual similarity, improving regional consistency while preserving boundaries. Its Gaussian Splatting Upsampling module reconstructs pixel-level features through RGB-guided anisotropic aggregation and test-time optimization. A global-anchor sliding-window strategy further supports large-scale imagery. Experiments on UDD5, DOTA, and LoveDA demonstrate competitive or superior performance over existing training-free methods, effectively filling the gap of DINO-series models in training-free open-vocabulary segmentation and providing a viable new path for further advances in this direction.
Changhao Zhao, Haoxiang Li, Yuke Li +2
College of Resources and Environment, Huazhong Agricultural University, Wuhan 430070, China