The Invisible Gorilla Effect in Out-of-distribution Detection

Harry Anthony; Ziyun Liang; Hermione Warr; Konstantinos Kamnitsas

cs.CVFeb 23, 2026

The Invisible Gorilla Effect in Out-of-distribution Detection

Authors: Harry Anthony, Ziyun Liang, Hermione Warr, Konstantinos Kamnitsas

Organizations: Department of Engineering Science, University of Oxford, Oxford, UK

Abstract

Deep Neural Networks achieve high performance in vision tasks by learning features from regions of interest (ROI) within images, but their performance degrades when deployed on out-of-distribution (OOD) data that differs from training data. This challenge has led to OOD detection methods that aim to identify and reject unreliable predictions. Although prior work shows that OOD detection performance varies by artefact type, the underlying causes remain underexplored. To this end, we identify a previously unreported bias in OOD detection: for hard-to-detect artefacts (near-OOD), detection performance typically improves when the artefact shares visual similarity (e.g. colour) with the model's ROI and drops when it does not - a phenomenon we term the Invisible Gorilla Effect. For example, in a skin lesion classifier with red lesion ROI, we show the method Mahalanobis Score achieves a 31.5% higher AUROC when detecting OOD red ink (similar to ROI) compared to black ink (dissimilar) annotations. We annotated artefacts by colour in 11,355 images from three public datasets (e.g. ISIC) and generated colour-swapped counterfactuals to rule out dataset bias. We then evaluated 40 OOD methods across 7 benchmarks and found significant performance drops for most methods when artefacts differed from the ROI. Our findings highlight an overlooked failure mode in OOD detection and provide guidance for more robust detectors. Code and annotations are available at: https://github.com/HarryAnthony/Invisible_Gorilla_Effect.

Figures & tables

Figure 1: Invisible Gorilla Effect. (Left) A DNN (e.g. a ResNet-18 skin lesion classifier) is trained for a primary task, where the region of interest (ROI) is the skin lesion, with a mean RGB of $(176,116,77)$ across the dataset. Once trained, the model is deployed on held-out data, including OOD data containing an unseen ink artefact. (Centre) Visualisation of the OOD detection methods analysed in this study, categorised as Internal Post-hoc, Internal Ad-hoc and External. Circle area reflects the number of method hyperparameters evaluated. (Right) We evaluated OOD detection AUROC on artefacts with colours similar (red) and dissimilar (green, purple, black) to the model’s ROI. A statistically significant AUROC drop ( $p<10^{-5}$ , Wilcoxon signed-rank) is observed for Mahalanobis Score and RealNVP methods on dissimilar-colour artefacts - an instance of the Invisible Gorilla Effect. Error bars denote 95% confidence intervals over 25 random seeds.

Dataset	Training Data Desc.	# Training Images	# ID Test Images	OOD Data Desc.	# OOD Images	# OOD Type
CheXpert	No Support Devices	5 213	1 282	Synthetic squares	1 282	Synthetic Covariate
ISIC	Planar images with no rulers, ink or colour charts	25 266	6 316	Colour charts	8 964	Real Covariate
ISIC	Planar images with no rulers, ink or colour charts	25 266	6 316	Ink annotations	2 358	Real Covariate
MVTec	Metal nuts with no ink	286	71	Ink artefacts	16	Semantic (new defect)
MVTec	Pills with no ink	233	58	Ink artefacts	18	Semantic (new defect)

Table 1: Summary of Training data, ID test data and OOD test data used for OOD detection evaluation.

Figure 2: (Left) Visualisation of annotated images from MVTec and ISIC, where red borders indicate artefacts visually similar to the model’s region of interest (ROI), and blue borders indicate dissimilar artefacts. (Right) Examples of colour-swapped counterfactuals for colour charts for both dissimilar and similar artefacts, showing the original image, artefact mask and the resulting counterfactual image.

Figure 3: Panel (a) shows training data with hyperintense hearts, (b) shows counterfactuals with hypointense hearts. Top rows show training examples and middle rows show OOD samples with synthetic squares of varying intensities. Bottom plots display AUROC for Mahalanobis Score across square intensities, with each point averaged over 25 runs on ResNet18 primary models and connected by a cubic spline. Results show that when models are trained on hyperintense hearts, this improves the OOD detection performance for hyperintense artefacts, and conversely for hypointense hearts.

OOD Method	Ink Artefacts		Colour Chart Artefacts
OOD Method	Similar	Dissimilar	Similar	Dissimilar	Synth Similar	Synth Dissimilar	$\Delta$ (sim. - diss.)
Feature-based Methods
CoP	71.74 (0.02)	65.76 (0.01)	87.37 (0.01)	84.35 (0.01)	88.58 (0.01)	83.97 (0.01)	4.50
CoRP	71.16 (0.02)	64.98 (0.01)	85.75 (0.01)	82.53 (0.01)	84.75 (0.02)	83.97 (0.01)	4.70
FeatureNorm	75.12 (0.01)	52.91 (0.02)	62.40 (0.08)	58.14 (0.12)	61.47 (0.02)	46.50 (0.05)	13.24
GRAM	80.32 (0.03)	72.71 (0.02)	59.08 (0.01)	56.79 (0.01)	61.08 (0.04)	57.45 (0.02)	4.95

Table 2: OOD detection AUROC (%) results for the ISIC benchmark, using a ResNet18 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to skin lesions (red for ink artefacts; red, orange, yellow for colour charts) or visually dissimilar (green, purple, black for ink artefacts; green, blue, black, grey for colour charts). Results on colour-swapped counterfactuals for colour chart artefacts are also reported. Mean AUROC drop between similar and dissimilar OOD (pp) on the two real datasets are reported. Methods are grouped into four groups: feature-based, confidence-based, ad-hoc and external. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets (in AUROC scale, not %).

Figure 4: a) Classification accuracy drop between ID and OOD datasets versus Mahalanobis OOD AUROC for each colour (marker colour = artefact colour) for the ISIC ink benchmark averaged over 25 ResNet18s, showing a weak correlation ( $r=0.39$ ). b) OOD AUROC drop between similar and dissimilar colours versus OOD AUROC on similar colours across 40 methods (averaged across ISIC benchmarks), showing a weak positive correlation.

Method	Pill		Metal Nut
Method	Similar	Diss.	Similar	Diss.	$\Delta$ (S-D)
Feature-based Methods
CoP	88.27	81.15	53.07	39.20	10.50
KDE	81.35	73.59	63.52	43.98	13.65
KNN	93.33	86.15	71.02	36.93	20.64
Mahal.	71.86	68.72	69.77	58.30	7.31

Table 3: OOD AUROC (%) on the MVTec benchmarks using ResNet18 as the primary model. The table reports the mean AUROC over 25 seeds for each method’s best-performing hyperparameters, covering both feature and confidence-based methods.

Method	Light aug. (ID acc -0.1 )		Heavy aug. (ID acc -5.5 )
Method	Similar	Diss.	Similar	Diss.
Feature-based Methods
CoP	77.0 (+5.3)	70.3 (+4.6)	79.6 (+7.8)	69.8 (+4.1)
KNN	90.1 (+4.5)	77.3 (+7.2)	87.9 (+2.2)	77.6 (+7.5)
NAN	76.0 (+0.4)	58.9 (+10)	73.6 (-2.0)	64.4 (+16)
Confidence-based Methods

Table 4: OOD AUROC (%) on the ISIC ink benchmark using 10 ResNet18 primary models. Brackets show the OOD AUROC and ID accuracy difference relative to the primary model’s trained without colour jitter augmentations (Relative to Table 2 ).

Panel of Figure 5 — Figure 5: (a) PCA variance vs. colour sensitivity for features from layer1.1 in a ResNet18 trained on ISIC, showing positive Spearman correlation ( $\rho=0.47,p<1.5\times 10^{-4}$ ). (b) OOD performance on ISIC ink benchmark with and without nuisance subspace projection (equation 3), averaged over 25 seeds.

Figure 6: OOD detection AUROC for Mahalanobis score across ResNet18 modules (excluding downsampling layers) for red, purple and black ink annotations in ISIC. Light (hyperparameter configuration A) and dark grey (hyperparameter configuration B) bands highlight selected modules for comparison. The results show that the observed OOD detection performance varies significantly across layers, demonstrating that conclusions drawn using Mahalanobis score are dependent on the choice of hyperparameter. The Invisible Gorilla Effect would not be observed if only the dark grey hyperparameter is evaluated, highlighting the benefit of evaluating a wide range of hyperparameter settings. Each point represents the mean over 25 seeds, with coloured shaded regions indicating the 95% confidence interval.

Method	Method Hyperparameter(s)	Number of Settings per Model
		ResNet18	VGG16	ViT-b32
CoP	$\bullet$ Dimensionality, $D_{\text{CoP}}\in[2,10,20]$	3	3	3
CoRP	$\bullet$ Dimensionality, $D_{\text{CoRP}}\in[2,10,20]$ $\bullet$ Gaussian Kernel, $\gamma_{\text{CoRP}}=[0.1]$ $\bullet$ Number of random Fourier features, $n_{rff}=[20]$	3	3	3
FeatureNorm	$\bullet$ Layer selection	66	42	139
GRAM	$\bullet$ Powers, $p_{GRAM}\in[[1],[10],[2,4,6]]$	3	3	3
KDE (Gaussian)	$\bullet$ Layer selection	66	42	139

Table 5: List of hyperparameters for feature-based OOD detection methods. The table presents the hyperparameter configurations evaluated in our study, along with the total number of settings used per primary network architecture for each method.

Method	Method Hyperparameter(s)	Number of Settings
Bayes By Backprop	$\bullet$ Monte Carlo Samples, $n_{MC}=[30]$ $\bullet$ Scoring function, $\mathcal{S}_{\text{BNN}}=$ [Mutual Information]	1
CIDER	$\bullet$ Distance function, $d_{\text{CIDER}}=$ [Cosine distance] $\bullet$ Feature extractor layer, $l_{\text{CIDER}}=$ [Output layer]	1
Outlier Exposure	$\bullet$ Scoring Function, $\mathcal{S}_{\text{OE}}=$ [MCP]	1
Reject Class		1
Rotation Prediction	$\bullet$ Scoring Function, $\mathcal{S}_{\text{RP}}=$ [Cross Entropy Loss]	1

Table 6: List of hyperparameters for internal ad-hoc OOD detection methods. The table presents the hyperparameter configurations evaluated in our study.

Method	Method Hyperparameter(s)	Number of Settings per Model
		ResNet18	VGG16	ViT-b32
ASH	$\bullet$ ASH function, $g_{\text{ASH}}\in$ [ASH-b, ASH-p, ASH-s] $\bullet$ Percentile, $p_{\text{ASH}}\in[0.6,0.7,0.8,0.9]$	12	12	12
Deep Ensemble	$\bullet$ Number of Ensemble members, $n_{\text{DE}}=[5]$ $\bullet$ Scoring function, $S_{\text{DE}}=$ [MCP]	1	1	1
DICE	$\bullet$ Sparsity parameter, $p_{\text{DICE}}\in[0.6,0.7,0.8,0.9]$ $\bullet$ Truncation layer, $l_{\text{DICE}}=$ [Penultimate layer]	4	4	4
GAIA-A		1	1	1
GradNorm	$\bullet$ Parameter selection $\bullet$ Summation method, $\ell_{n}\in[\ell_{1},\ell_{2}]$	132	64	304

Table 7: The table presents the hyperparameter configurations evaluated in our study for confidence-based methods , along with the total number of settings used per primary network architecture for each method. The table presents the hyperparameter configurations evaluated in our study, along with the total number of settings used per primary network architecture for each method.

Method	Method Hyperparameter(s)	Number of Settings
DeepSVDD		1
DDPM-MSE	$\bullet$ Number of steps, $n_{\text{DDPM-step}}=[1000]$ $\bullet$ Number of reconstructions, $n_{\text{recon}}=[100]$ $\bullet$ ID comparator, $\mathcal{D}_{\text{DDPM}}=$ [Training Data]	1
DDPM-LPIPS	$\bullet$ Number of steps, $n_{\text{DDPM-step}}=[1000]$ $\bullet$ Number of reconstructions, $n_{\text{recon}}=[100]$ $\bullet$ ID comparator, $\mathcal{D}_{\text{DDPM}}=$ [Training Data] $\bullet$ Feature extractor, $\phi=$ [Pretrained AlexNet]	1
Foreign Patch Interpolation	$\bullet$ Threshold, $\tau_{\text{FPI}}\in[0.4,0.5,0.6,0.7,0.8,0.9]$	6
RealNVP	$\bullet$ Layer selection	$\bullet$ 66 for ResNet18 $\bullet$ 42 for VGG16 $\bullet$ 139 for Vit-b32

Table 8: List of hyperparameters for external OOD detection methods. The table presents the hyperparameter configurations evaluated in our study, along with the total number of settings used per primary network architecture for each method.

Colour of chart	Number of Images
Blue	4 203
Yellow	2 502
Green	2 315
Orange	2 193
Red	1 462
Grey / white	380

Table 9: Number of images containing a colour chart in ISIC where the specified colour is present. For images with multiple colours, each colour is counted independently (i.e. counts are not mutually exclusive).

Colour of chart	Number of Images
Blue	2 334
Yellow	1 202
Green	648
Orange	571
Red	805
Grey / white	365

Table 10: Number of images containing a colour chart in ISIC with a single, uniquely assigned colour. Each image is associated with exactly one colour category.

Colour of chart	Number of Images
Blue	904
Yellow	419
Green	185
Orange	207
Red	321
Grey / white	92

Table 11: Number of images containing a colour chart in ISIC with a uniquely assigned colour, where the size of the colour chart is less than 10% of the image area. Each image is associated with exactly one colour category.

Colour of Ink	Number of Images
Purple	1 558
Black	887
Green	26
Red	25

Table 12: Number of images containing an ink annotation in ISIC with a uniquely assigned colour where the specified colour is present. For images with multiple colours, each colour is counted independently (i.e. counts are not mutually exclusive).

Colour of Ink	Number of Images
Purple	1 481
Black	839
Green	22
Red	22

Table 13: Number of images containing an ink annotation in ISIC with a single, uniquely assigned colour. Each image is associated with exactly one colour category.

Colour of Ink	Number of Images
a) Metal Nut
Black	8
Blue	8
b) Pill
Red	12
Yellow	6

Table 14: Number of images containing an ink annotation with a single, uniquely assigned colour in the MVTec benchmarks, separated by a) Metal Nuts and b) Pills.

Figure 7: Distribution of AUROC scores across 25 random seeds for Mahalanobis Score on red (visually similar) and green (visually dissimilar) ink artefacts in the ISIC benchmark. Results are shown for the best-performing feature layer over the OOD dataset. A statistically significant difference was observed (

p=3.28\times 10^{-6}

, Wilcoxon signed-rank test).

Figure 8: Explainability analysis of model focus on ISIC lesion regions. Three dermoscopic images from ISIC (top left) and their corresponding saliency maps generated using seven explainable AI methods (Expected Gradients [ 32 ] , LayerCAM [ 52 ] , HiResCAM [ 27 ] , LRP [ 6 ] , CRP [ 1 ] , Integrated Gradients [ 86 ] and GradCAM [ 80 ] ) for a VGG16 primary model trained for classifying between malignant versus benign lesions. cross all methods, the highlighted regions consistently localise on the lesion, providing evidence that the primary models’s region of interest is the lesion.

Dataset	Model ROI	Mask	Mean RGB
ISIC	Skin Lesion	SAM	(176, 116, 77)
MVTec-AD	Metal Nut	SAM	(77, 86, 83)

Table 15: Mean RGB for the model’s ROI across the training/ID dataset (or a sample of 100 images for ISIC), using segmentation masks created with SAM using human annotator-guided prompts.

Figure 9: Examples of segmentation masks used to calculate the mean RGB of regions of interest (ROIs) for in-distribution data and OOD artefacts. A human annotator iteratively prompted the Segment Anything Model (SAM) until the produced mask closely matched the target region. Panel (a) shows segmentations of the model’s ROI and panel (b) shows segmentations the OOD artefacts for the ISIC and MVTec benchmarks.

Colour	Mask	Mean RGB	$\ell_{2}$ dist	Label
Red	SAM	(222, 52, 57)	81.3	Sim.
Orange	SAM	(207, 123, 48)	43.0	Sim.
Yellow	SAM	(207, 191, 48)	86.2	Sim.
Blue	SAM	(65, 52, 57)	129.7	Diss.
Green	SAM	(53, 152, 69)	128.4	Diss.
White	SAM	(144, 158, 162)	100.1	Diss.

Table 16: Mean RGB values of the colour-chart artefact colours used to make the OOD benchmark for ISIC. Segmentation masks were created using SAM with human annotator-guided prompts. The Euclidean distance between the mean artefact and model’s ROI is given (

\ell_{2}

), along with the categorisation of similar and dissimilar colours under a threshold of

\ell_{2}=90

Colour	Mask	Mean RGB	$\ell_{2}$ dist	Label
Blue	GT	(57, 59, 59)	40.9	Sim.
Black	GT	(49, 53, 68)	45.1	Diss.

Table 17: Mean RGB values of the ink artefact colours used to make the OOD benchmark for ISIC. Segmentation masks were created using the ground truth (GT) masks provided by MVTec-AD. The Euclidean distance between the mean artefact and model’s ROI is given (

\ell_{2}

), along with the categorisation of similar and dissimilar colours under a threshold of

\ell_{2}=42

OOD Method	Ink Artefacts		Colour Chart Artefacts
	AURC ( $\downarrow$ )		AURC ( $\downarrow$ )
	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	0.11 (0.01)	17.34 (0.01)	2.73 (0.01)	4.28 (0.01)
CoRP	0.14 (0.03)	17.39 (0.01)	3.03 (0.01)	4.71 (0.01)
FeatureNorm	0.15 (0.01)	26.01 (0.03)	1.53 (0.01)	2.11 (0.02)

Table 18: OOD detection AURC (%) results for the ISIC benchmark, using a ResNet18 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to skin lesions (red for ink artefacts; red, orange, yellow for colour charts) or visually dissimilar (green, purple, black for ink artefacts; green, blue, black, grey for colour charts). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AURC over 25 seeds, with 95% confidence intervals in brackets. For AURC, lower values indicate better performance.

OOD Method	Ink Artefacts		Colour Chart Artefacts
	FPR@TPR80 ( $\downarrow$ )		FPR@TPR80 ( $\downarrow$ )
	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	55.30 (0.01)	61.85 (0.01)	22.06 (0.02)	28.34 (0.03)
CoRP	56.91 (0.03)	61.91 (0.01)	26.16 (0.03)	32.39 (0.03)
FeatureNorm	34.91 (0.02)	77.85 (0.03)	60.44 (0.02)	64.73 (0.04)

Table 19: OOD detection FPR@TPR80 (%) results for the ISIC benchmark, using a ResNet18 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to skin lesions (red for ink artefacts; red, orange, yellow for colour charts) or visually dissimilar (green, purple, black for ink artefacts; green, blue, black, grey for colour charts). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean FPR@TPR80 over 25 seeds, with 95% confidence intervals in brackets. For FPR@TPR80, lower values indicate better performance.

OOD Method	Ink Artefacts		Colour Chart Artefacts
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	77.43 (0.02)	72.08 (0.01)	91.09 (0.01)	87.99 (0.02)
CoRP	77.51 (0.03)	70.05 (0.01)	89.35 (0.01)	85.52 (0.02)
FeatureNorm	68.37 (0.03)	52.33 (0.03)	86.06 (0.01)	76.39 (0.04)
GRAM	63.54 (0.03)	52.68 (0.01)	78.90 (0.03)	77.92 (0.03)

Table 20: OOD detection AUROC (%) results for the ISIC benchmark, using a ViT-B/32 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to skin lesions (red for ink artefacts; red, orange, yellow for colour charts) or visually dissimilar (green, purple, black for ink artefacts; green, blue, black, grey for colour charts). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets.

OOD Method	Ink Artefacts		Colour Chart Artefacts
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	64.07 (0.01)	57.30 (0.01)	54.70 (0.02)	54.38 (0.02)
CoRP	63.63 (0.01)	56.07 (0.01)	54.63 (0.02)	54.16 (0.02)
FeatureNorm	82.48 (0.01)	56.18 (0.01)	60.49 (0.02)	59.88 (0.03)
GRAM	68.25 (0.04)	55.25 (0.03)	59.32 (0.02)	59.27 (0.02)

Table 21: OOD detection AUROC (%) results for the ISIC benchmark, using a VGG16 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to skin lesions (red for ink artefacts; red, orange, yellow for colour charts) or visually dissimilar (green, purple, black for ink artefacts; green, blue, black, grey for colour charts). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets.

OOD Method	Pill		Metal Nut
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	88.27 (0.01)	81.15 (0.02)	53.07 (0.01)	39.20 (0.03)
CoRP	81.92 (0.01)	71.41 (0.02)	48.86 (0.01)	39.09 (0.03)
FeatureNorm	71.67 (0.02)	56.41 (0.02)	55.00 (0.01)	52.69 (0.02)
GRAM	74.76 (0.01)	69.83 (0.01)	68.94 (0.01)	64.70 (0.01)

Table 22: OOD detection AUROC (%) results for the MVTec benchmark, using a ResNet18 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to the model’s ROI (red for pill; black for metal nut) or visually dissimilar (yellow for ink artefacts; blue for metal nut). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets.

OOD Method	Pill		Metal Nut
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	72.76 (0.01)	66.92 (0.01)	78.79 (0.02)	42.33 (0.02)
CoRP	69.87 (0.01)	64.38 (0.01)	75.19 (0.03)	41.86 (0.02)
FeatureNorm	66.03 (0.02)	62.49 (0.02)	85.23 (0.02)	68.75 (0.01)
GRAM	57.55 (0.03)	54.83 (0.02)	78.23 (0.02)	52.54 (0.02)

Table 23: OOD detection AUROC (%) results for the MVTec benchmark, using a ViT-B/32 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to the model’s ROI (red for pill; black for metal nut) or visually dissimilar (yellow for ink artefacts; blue for metal nut). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets.

OOD Method	Pill		Metal Nut
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	76.68 (0.04)	69.07 (0.03)	68.18 (0.02)	53.52 (0.02)
CoRP	69.42 (0.02)	65.77 (0.02)	63.63 (0.02)	50.90 (0.04)
FeatureNorm	52.28 (0.02)	51.15 (0.01)	57.95 (0.02)	45.45 (0.04)
GRAM	64.90 (0.03)	60.19 (0.03)	72.46 (0.04)	65.32 (0.03)

Table 24: OOD detection AUROC (%) results for the MVTec benchmark, using a VGG16 primary model for internal methods. Detection was evaluated on artefacts that are either visually similar to the model’s ROI (red for pill; black for metal nut) or visually dissimilar (yellow for ink artefacts; blue for metal nut). Methods are grouped into two groups: feature-based and confidence-based. Each entry shows the best-performing hyperparameter setting, reported as the mean AUROC over 25 seeds, with 95% confidence intervals in brackets.

OOD Method	Light Augmentation		Heavy Augmentation
OOD Method	Similar	Dissimilar	Similar	Dissimilar
Feature-based Methods
CoP	77.04 (+5.30)	70.34 (+4.58)	79.58 (+7.84)	69.81 (+4.05)
CoRP	74.60 (+3.44)	68.72 (+3.74)	74.33 (+3.17)	67.00 (+2.02)
FeatureNorm	75.92 (+0.80)	64.24 (+11.33)	75.66 (+0.54)	68.09 (+15.18)
GRAM	67.25 (-13.07)	59.24 (-13.47)	66.83 (-13.49)	61.21 (-11.50)

Table 25: OOD AUROC (%) on the ISIC ink benchmark using 10 ResNet18 primary models, trained with either light or heavy colour jitter augmentations. Brackets show the OOD AUROC difference compared to the primary model’s trained without colour jitter augmentations (Table 2, main paper).

OOD Method	Light Augmentation
OOD Method	Similar	Dissimilar
Feature-based Methods
CoP	87.82 (+10.39)	72.23 (+0.15)
CoRP	80.95 (+3.44)	69.28 (-0.77)
FeatureNorm	82.09 (+13.72)	73.27 (+20.94)
GRAM	44.24 (-19.30)	44.65 (-8.03)

Table 26: OOD AUROC (%) on the ISIC ink benchmark using 10 ViT-B/32 primary models, trained with light colour jitter augmentations. Brackets show the OOD AUROC difference compared to the primary model’s trained without colour jitter augmentations (Table 20 ).

Method	Inference Latency (ms)
Mahalanobis	12.15
+ proj.	14.23
Featurenorm	8.63
+ proj.	10.32
NaN	11.04
+ proj.	13.09

Table 27: Average inference latency (in milliseconds) for a single image on the ISIC Ink benchmark on a ResNet18 primary model. Results are shown for three feature-based OOD detection methods with and without the proposed subspace projection, showing small increases in inference latency. The forward pass through the model is included in this value. The inference latency of the external method baseline (DDPM-MSE) is included to highlight that some OOD detection methods have significantly higher computational cost.

OOD Artefact	k=2	k=5	k=10	k=20
Similar	77.2	77.5	53.7	48.2
Dissimilar	67.0	75.8	59.4	47.0

Table 28: Average OOD detection AUROC (%) for the Mahalanobis method as a function of the nuisance subspace dimensionality

k

. Removing a small number of nuisance directions (up to

k=5

) improves performance for both similar and dissimilar artefacts, whereas removing larger subspaces (

k\geq 10

) removes task-relevant structure and substantially degrades AUROC.

Method	Ink Artefacts		Colour Charts
Method	Sim.	Diss.	Sim.	Diss.
Mahalanobis	77.0	63.6	96.7	95.4
+Proj.	77.5	75.8	95.8	97.7
MBM	77.4	63.7	97.0	95.5
+Proj.	77.6	75.3	96.1	97.2
FeatureNorm	75.1	52.9	62.4	58.1

Table 29: OOD performance on ISIC ink and colour chart benchmarks with and without nuisance subspace projection (equation 3), averaged over 25 ResNet18 models with random seeds. Using projected features can reduce the Invisible Gorilla Effect across several feature based OOD detection methods, although not for all methods (e.g. XOOD-M).

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 222 papers

Mar 18, 2026

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitation: logit-based methods see only the classifier's confidence signal, while feature-based methods attempt to measure membership in the training distribution but do so in the full feature space where confidence and membership are entangled, inheriting architecture-sensitive failure modes. We observe that penultimate features naturally decompose into two orthogonal subspaces: a classifier-aligned component encoding confidence, and a residual the classifier discards. We discover that this residual carries a class-specific directional signature for in-distribution data -- a membership signal invisible to logit-based methods and entangled with noise in feature-based methods. We propose CORE (COnfidence + REsidual), which disentangles the two signals by scoring each subspace independently and combines them via normalized summation. Because the two signals are orthogonal by construction, their failure modes are approximately independent, producing robust detection where either view alone is unreliable. CORE achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations, ranking first in three of five settings and achieving the highest grand average AUROC with negligible computational overhead.

Jin Mo Yang, Hyung-Sin Kim, Saewoong Bahk

Seoul National University, Seoul, South Korea

OOD Detection

Ask PDF

May 5, 2026

A Robust Out-of-Distribution Detection Framework via Synergistic Smoothing

Reliable out-of-distribution (OOD) detection is a critical requirement for the safe deployment of machine learning systems. Despite recent progress, state-of-the-art OOD detectors are highly susceptible to adversarial attacks, which undermines their trustworthiness in automated systems. To address this vulnerability, we apply median smoothing to baseline OOD detection scores, balancing clean and adversarial accuracies. Our key insight is that the noisy samples generated for median smoothing can be repurposed to quantify the local instability of the base score. We observe that OOD samples exhibit higher instability under perturbation. Based on this, we propose ROSS, a novel and robust post-hoc OOD detector that leverages the instability of baseline scores to further distinguish between in-distribution (ID) and OOD samples. ROSS achieves symmetric robustness, performing strongly against both score-minimising and score-maximising attacks, unlike prior work. This symmetric defence leads to state-of-the-art robustness, outperforming prior methods by up to 40 AUROC points. We demonstrate ROSS's effectiveness on extensive experiments across CIFAR-10, CIFAR-100, and ImageNet. Code is available at: https://github.com/Abdu-Hekal/ROSS.

Maria Stoica, Abdelrahman Hekal, Alessio Lomuscio

Imperial College London · Zeroth Research

Adversarial Robustness OOD Detection

Ask PDF

Mar 8, 2026

A Systematic Comparison of Training Objectives for Out-of-Distribution Detection in Image Classification

Out-of-distribution (OOD) detection is critical in safety-sensitive applications. While this challenge has been addressed from various perspectives, the influence of training objectives on OOD behavior remains comparatively underexplored. In this paper, we present a systematic comparison of four widely used training objectives: Cross-Entropy Loss, Prototype Loss, Triplet Loss, and Average Precision (AP) Loss, spanning probabilistic, prototype-based, metric-learning, and ranking-based supervision, for OOD detection in image classification under standardized OpenOOD protocols. Within the evaluated ResNet-18/OpenOOD setting and objective-specific OOD scoring rules, Cross-Entropy Loss, Prototype Loss, and AP Loss achieve comparable in-distribution accuracy, while Cross-Entropy Loss provides the most consistent near- and far-OOD AUROC overall; the other objectives can be competitive in specific settings.

Furkan Genç, Onat Özdemir, Emre Akbaş

Dept. of Electrical and Electronics Engineering, Bilkent University · Dept. of Computer Engineering, Middle East Technical University

OOD Detection Image Classification

Ask PDF

Abstract

OOD Detection

Explore similar work

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

A Robust Out-of-Distribution Detection Framework via Synergistic Smoothing

A Systematic Comparison of Training Objectives for Out-of-Distribution Detection in Image Classification