Combining Foundation Model Confidence and Monocular Depth for Training-Free Out-of-Distribution Segmentation

Serin Varghese; Fabian Hüger; Kira Maag

cs.CVSep 19, 2026

Combining Foundation Model Confidence and Monocular Depth for Training-Free Out-of-Distribution Segmentation

Authors: Serin Varghese, Fabian Hüger, Kira Maag

Organizations: CARIAD SE Wolfsburg, Germany

Abstract

Autonomous vehicles operating in open-world scenarios are inevitably confronted with previously unknown objects, such as exotic animals or loose cargo. The reliable detection and segmentation of these out-of-distribution (OOD) objects is therefore crucial for a safe understanding of the environment and decision-making. Most existing approaches require access to OOD training samples, retraining of the segmentation backbone, or dedicated auxiliary architectures, limiting their practical applicability. We propose a training-free method that derives dense OOD scores directly from the confidence predictions of a foundation segmentation model, without any task-specific fine-tuning or access to anomalous data. To improve the robustness of our OOD segmentation, geometric information from monocular depth estimation is incorporated into the decision process, providing complementary cues to uncertainty-based predictions. We evaluate the proposed method on the SegmentMeIfYouCan benchmark and additionally assess its performance on OOD tracking in video sequences, reflecting the temporal nature of real-world perception systems. The method performs strongly on road-centered benchmarks.

Figures & tables

Fig. 1: (a) Input image from the SOS dataset, (b) road confidence by the foundation segmentation model, (c) monocular depth estimation (the blue boxes show the smooth areas that are not OODs and can be suppressed), and (d) our obtained OOD score.

Fig. 2: Block diagram of the proposed depth-ring OOD detection pipeline. The input image $\mathbf{x}_{t}$ is processed by two branches: SAM 2 provides dense anomaly scores $\mathbf{s}^{\mathrm{SAM}}\in\mathbb{I}^{H\times W}$ via inverted road segmentation logits and the road mask $\mathcal{M}_{\mathrm{road}}$ ; Depth Anything V2 estimates monocular depth $\hat{\mathbf{d}}$ . Connected components $\mathcal{C}_{n}$ from thresholded OOD scores are scored by depth disparity $\Delta_{n}$ against a surrounding road ring $\mathcal{R}_{n}$ , retaining only components satisfying $\Delta_{n}\geq\delta_{\min}$ .

Fig. 3: Examples of systematic false positives. Each column highlights a characteristic failure mode of the raw OOD predictions: (a) inaccurate road boundaries, (b) small isolated artifacts, (c) large connected components of ego-vehicle hood, and (d) flat road structures such as manhole covers. The first row shows the RGB image overlaid with the dense OOD score map. The second row visualizes the resulting OOD detections, where true positive OOD pixels are shown in green and false positive detections in red.

		SOS		LostAndFound
Row	Configuration	AuPRC $\uparrow$	FPR 95 $\downarrow$	AuPRC $\uparrow$	FPR 95 $\downarrow$
1	SAM 2, 1 point	7.05	100.00	4.48	11.30
2	SAM 2, 3P+2N	18.89	4.47	24.46	4.88
3	SAM 2, 3P+2N+BB (70%)	93.34	0.22	71.13	3.20

TABLE I: Left: Ablation study on the prompting strategy for SAM 2 on SOS and LostAndFound datasets, evaluated using pixel-level metrics AuPRC and FPR 95 . Right: Ablation study on the cumulative effect of each post-processing stage on SOS and LostAndFound datasets, evaluated using segment-level

\overline{F_{1}}

-score.

	LostAndFound
	AuPRC $\uparrow$	FPR 95 $\downarrow$	$\overline{\text{sIoU}}$ $\uparrow$	$\overline{\text{PPV}}$ $\uparrow$	$\overline{F_{1}}$ $\uparrow$
Maximum Softmax	30.1	33.2	14.2	62.2	10.3
MC Dropout	36.8	35.6	17.4	34.7	13.0
Ensemble	2.9	82.0	6.7	7.6	2.7
PGN	69.3	9.8	50.0	44.8	45.4
W-EDL	50.4	25.8	21.2	38.0	19.2

TABLE II: OOD segmentation benchmark results on the test split of the LostAndFound dataset.

	RoadAnomaly					RoadObstacle
	AuPRC $\uparrow$	FPR 95 $\downarrow$	$\overline{\text{sIoU}}$ $\uparrow$	$\overline{\text{PPV}}$ $\uparrow$	$\overline{F_{1}}$ $\uparrow$	AuPRC $\uparrow$	FPR 95 $\downarrow$	$\overline{\text{sIoU}}$ $\uparrow$	$\overline{\text{PPV}}$ $\uparrow$	$\overline{F_{1}}$ $\uparrow$
Maximum Softmax	28.0	72.1	15.5	15.3	5.4	15.7	16.6	19.7	15.9	6.3
MC Dropout	28.9	69.5	20.5	17.3	4.3	4.9	50.3	5.5	5.8	1.1
Ensemble	17.7	91.1	16.4	20.8	3.4	1.1	77.2	8.6	4.7	1.3
PGN	36.7	61.4	21.6	17.5	6.2	16.5	19.7	19.5	14.9	7.4
W-EDL	54.3	50.4	26.3	21.5	7.8	30.7	42.3	11.6	26.4	8.7

TABLE III: OOD segmentation benchmark results for RoadAnomaly and RoadObstacle datasets.

	SOS					WOS
	AuPRC $\uparrow$	FPR 95 $\downarrow$	$\overline{F_{1}}$ $\uparrow$	MOTA $\uparrow$	MOTP $\downarrow$	AuPRC $\uparrow$	FPR 95 $\downarrow$	$\overline{F_{1}}$ $\uparrow$	MOTA $\uparrow$	MOTP $\downarrow$
Baseline	85.20	1.30	50.40	0.32	12.45	94.92	0.59	30.13	0.13	51.17
PixOOD	94.76	0.18	47.15	0.41	8.24	97.57	0.21	43.36	0.00	14.05
JSR-Net	87.01	1.59	9.98	-4.59	97.67	59.66	19.49	2.11	-19.30	301.52
DaCUP	93.57	0.17	20.25	-1.34	89.79	83.13	3.77	12.93	-2.26	280.74
UNO-SAM2	90.21	0.18	15.45	0.44	2.15	88.10	0.22	26.57	0.37	3.98

TABLE IV: OOD segmentation and tracking benchmark results for SOS and WOS datasets.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 222 papers

Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving
Jul 5, 2026Zhiran Yan, Gordon ElgerImage Segmentation Autonomous Driving Perception
Institute of Innovative Mobility (IIMo), Technical University Ingolstadt of Applied Sciences, Ingolstadt, 85049, Germany. · Fraunhofer Institute for Transportation and Infrastructure Systems IVI, Ingolstadt, 85051, Germany.
CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring
Mar 18, 2026Jin Mo Yang, Hyung-Sin Kim, Saewoong BahkOOD Detection
Seoul National University, Seoul, South Korea
A Probabilistic Perspective on Wasserstein-Based Evidential Uncertainty for Out-of-Distribution Segmentation
Oct 7, 2026Arnold Brosch, Abdelrahman Eldesokey, Michael Felsberg +1Segmentation Uncertainty Estimation Evidential Deep Learning
Heinrich-Heine-University Düsseldorf, Universitätsstraße 1, 40225, Düsseldorf, Germany · Hamad Bin Khalifa University (HBKU), Education City, Gate 8, P.O. Box 5825, Doha, Qatar · Linköping University, SE-581 83, Linköping, Sweden

Jul 5, 2026J/K move · Enter open · S save

Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving

Zhiran Yan, Gordon Elger

Institute of Innovative Mobility (IIMo), Technical University Ingolstadt of Applied Sciences, Ingolstadt, 85049, Germany. · Fraunhofer Institute for Transportation and Infrastructure Systems IVI, Ingolstadt, 85051, Germany.

Image Segmentation Autonomous Driving Perception OOD Detection Zero-Shot Anomaly Detection

Open paper AskPDF arXiv

Traditional semantic segmentation models operate under a closed-set assumption and struggle to recognize unknown or unexpected objects-an essential capability for autonomous driving. As a result, such models often misclassify or overlook out-of-distribution (OOD) road anomalies, posing safety risks in open-world environments. We present a lightweight, postprocessing, road-aware anomaly segmentation framework that requires no retraining, no OOD data, and no auxiliary supervision. Our approach builds on a mask transformer-based segmentation network by exploiting query-level mask confidence and deriving a polygonal road prior to detect gap regions that may correspond to anomalies. To further suppress false positives, we introduce a CLIP-based zero-shot semantic filtering module using in-distribution prompts, with optional generalized OOD prompts. By jointly leveraging spatial priors and semantic verification, our framework produces robust and interpretable anomaly predictions. Evaluation on three public benchmarks-Fishyscapes, SMIYC, and RoadAnomaly-shows consistently strong performance. In particular, our method outperforms the training-free baseline Maskomaly on most metrics and achieves the highest AP on Fishyscapes LostAndFound. These results demonstrate the practicality and deployability of our approach for real-world autonomous driving systems.

Abstract

OOD Detection

Explore similar work

Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

A Probabilistic Perspective on Wasserstein-Based Evidential Uncertainty for Out-of-Distribution Segmentation