Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gain

Donghoon Lee; Shinjin Kang

cs.LGOct 1, 2026

Dataset Identity, Not Novelty: The Source of an Inflated OOD Detection Gain

Authors: Donghoon Lee, Shinjin Kang

Organizations: Hongik University

Abstract

A post-hoc out-of-distribution (OOD) detector reads the activations of a trained classifier and returns a score. It fits that score on in-distribution data, and the benchmarks that evaluate it supply a second piece of OOD data for the fitting itself. Some detectors tune a constant on it. Others fit a direction in feature space or train a flexible combiner and report the number that it reaches as the gain that is still available. Every such fit is validated on held-out samples of the same OOD dataset. That check rules out memorizing individual images. It says nothing about a fit that has instead learned which dataset it is looking at, and a direction that recognizes one OOD dataset rather than novelty passes it perfectly. The detector that a practitioner installs meets OOD data from a source that nobody fitted it on, so the difference decides what the reported number is worth. We measure it by holding out the whole OOD dataset rather than a sample of it, and we call that gap the inflation. We read it across a range of combiners on ImageNet and CIFAR-100 backbones. Most of the gain that the usual protocol reports turns out to be dataset identity rather than novelty. The size of the fit does not move what survives, so the effect is not ordinary overfitting. The share depends instead on whether the input exposes class identity, and two controls that vary that property alone separate the inflation on every backbone of both benchmarks. A closed form accounts for the effect and computes it from the fitting rows, so a practitioner can tell which fits will inflate without running the hold-out protocol. One of these fits survives, namely the single constant that the field already picks on a designated validation dataset. Anything above it reports a gain that the hold-out protocol does not return, and on one benchmark what survives falls while what is reported climbs.

Figures & tables

Figure 1: Where the OOD datasets point. The usual protocol gives the fit the direction of the dataset that it will be scored on, while deployment gives it the direction that the other datasets share. The pole is that shared direction and the dome is the unit sphere around it. Each dot is one OOD dataset at its angle to the pole, for the two profiles at full dimension over 11 ImageNet backbones. Each cone opens to its family mean. The bars give the AUROC each family reaches under the two protocols. Rotation around the pole carries no data.

Figure 2: What a fitted score buys and what survives. The solid line gives the AUROC that the usual protocol reports and the dashed line gives the AUROC that survives once we hold the whole dataset out. The distance between them is the part of the reported gain that comes from dataset identity. The dotted line marks the zero capacity reference, which fits nothing. Both panels run on the 13 shared channels, and the widest point adds the whitened feature space.

		ImageNet-1k			CIFAR-100
Input	params	reported	held out	infl.	reported	held out	infl.
none (VarRatio)	$0$	89.13	89.13	$+0.00$	80.84	80.84	$+0.00$
$\{d_{(1)},\operatorname{Var}\}$ , target tuned	$1$	89.91	89.39	$+0.52$	82.28	80.75	$+1.53$
2 channels	$2$	89.22	88.47	$+0.75$	81.30	79.32	$+1.98$
4 channels	$4$	89.51	88.30	$+1.21$	82.16	79.64	$+2.52$
8 channels	$8$	91.69	90.04	$+1.66$	84.93	80.46	$+4.47$

Table 1: What each fit reads, how many parameters it holds, and what survives. Columns give the AUROC that the usual protocol reports, the AUROC that survives holding the OOD dataset out, and the difference. The parameter column counts the ImageNet fit at

D=1024

, and counts that depend on

D

vary with it. Every row scores the held-out halves of Section 4.1 , so the reference row differs from Table 5 , and Appendix I gives the spread behind each average.

Figure 3: The inflation is the product of a length and an angle. Left, the prediction of equation 6 against the inflation measured under the two protocols, one dot per representation per backbone. The prediction reads the fitting rows only. Right, the angular factor against the width of the fit. The factor separates the two materials by a wide margin at every width, which is why sorting the class distances flattens the width curve.

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Architecture	$D$	Params	seed 0	seed 1	seed 2	mean $\pm$ sd	minutes
ResNet-18	512	11.2M	78.40	78.15	77.60	78.05 $\pm$ 0.41	8
ResNet-34	512	21.3M	78.24	78.44	79.11	78.60 $\pm$ 0.46	15
WRN-28-10	640	36.5M	81.64	81.58	81.71	81.64 $\pm$ 0.07	57
DenseNet-BC	342	0.80M	77.92	77.05	77.13	77.37 $\pm$ 0.48	38
VGG16-BN	512	14.8M	74.85	74.45	74.46	74.59 $\pm$ 0.23	6
MobileNetV2	1280	2.35M	72.74	71.64	71.98	72.12 $\pm$ 0.56	14

Appendix

Table 2: CIFAR-100 backbones. Top-1 accuracy for each seed, feature width

D

, parameter count and training time on one RTX 4090. The recipe is identical for every row and only the seed changes.

Method	ours	OpenOOD	$\Delta$	ours	OpenOOD	$\Delta$
	near			far
MSP	76.01	76.02	$-0.01$	84.56	85.23	$-0.67$
MaxLogit	76.44	76.46	$-0.02$	88.89	89.57	$-0.68$
Energy	75.88	75.89	$-0.01$	88.79	89.47	$-0.68$
ViM	72.07	72.08	$-0.01$	92.55	92.68	$-0.13$
KNN	71.48	71.10	$+0.38$	89.71	90.18	$-0.47$

Appendix

Table 3: Our implementation against OpenOOD v1.5 on ImageNet-1k with torchvision ResNet-50 V1. AUROC in percent, grouped as OpenOOD groups them.

#	Channel	Definition	Both
1	nearest distance	$\log d_{(1)}$	✓
2	whitened norm	$\lVert z\rVert^{2}$	✓
3	two nearest	$\log d_{(1)}+\log d_{(2)}$	✓
4	soft nearest	$-\log\sum_{c}e^{-d_{c}/2}$	✓
5	angular residual	$1-\max_{c}\cos(\hat{\phi},\Sigma^{1/2}\mu_{c})$	✓
6	distance variance	$-\operatorname{Var}_{c}d_{c}$	✓

Appendix

Table 4: The 17 channels in cache order. The last column marks the ones that CIFAR-100 does not define in our setup, which leaves the 13 channels that both benchmarks share. Channels 16 and 17 standardize channels 9 and 10 against the ID training split and then take the absolute value, so they answer how unusual a logit gap is rather than how large.

ImageNet-1k, 11 backbones
	Near-OOD		Far-OOD		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	77.17	73.23	85.24	53.64	82.01	61.47
MaxLogit	75.54	70.83	84.36	50.31	80.83	58.52
Energy	71.75	73.35	78.52	56.73	75.81	63.38
GEN ∗	79.89	69.40	90.09	42.30	86.01	53.14

Appendix

Table 5: Detectors that need no OOD data, averaged over backbones. AUROC and FPR@95 in percent. A star marks a detector whose constant we hand an oracle value on the test sets. CIFAR-100 rows average the 3 seeds of each architecture.

Figure 4: Detectors behave differently on different backbones. Each dot is one backbone and the highlighted dot is that detector’s worst backbone. Rows sort by that worst case. A star marks a detector whose constant we give an oracle value on the test sets.

Figure 5: The same numbers, sorted and unsorted. Both rows hold the

C+1

entries of the class distance profile in the same dimension, take the same ridge discriminant under the same splits, and are read through the same random projections. Sorting the vector first removes the width effect and lowers the inflation by a factor of

2.9

6.5

. The vertical distance between the two curves is what class identity costs.

Figure 6: The inflation against the width of the fit, on two materials. The orange line reads random orthonormal views of the whitened feature, which can respond to the distance of a named class. The blue line reads prefixes of the permutation invariant channel inventory. At any fixed point of the horizontal axis the two carry the same directions and the same free parameters. At the right edge they meet, because a full width projection is a rotation. The dotted leg is the last step of the capacity sequence, which changes the material and the width at once.

		inflation			ordering, held out
Hold out	Ridge	invariant	class aware	more	margin	ahead
ImageNet-1k, 11 backbones
normalized sum	fixed	$+2.15$	$+11.66$	11/11	$-3.87$	2/11
union	fixed	$+2.82$	$+13.78$	11/11	$-5.32$	2/11
normalized sum	nested	$+2.33$	$+7.81$	11/11	$+0.92$	8/11
union	nested	$+3.11$	$+8.33$	11/11	$+1.17$	9/11

Appendix

Table 6: The permutation pair under both hold-out constructions and both ridge rules. The inflation columns give the inflation of each member and the count of backbones on which the class aware member inflates more. The ordering columns subtract the hold-out AUROC of the permutation invariant member from that of the class aware one, and they give the count of backbones on which the class aware member is ahead. CIFAR-100 counts run over 6 architectures with 3 seeds averaged inside each.

	sorted, permutation invariant			unsorted, class aware
Backbone	within	held out	inflation	within	held out	inflation
ImageNet-1k, 11 backbones
ConvNeXtV2-B	96.13	95.07	$+1.06$	99.80	95.98	$+3.82$
SwinV2-B	94.83	92.31	$+2.52$	99.80	94.49	$+5.31$
DeiT3-B/16	93.57	90.89	$+2.69$	99.37	93.03	$+6.33$
ResNet-50 V1	91.55	88.74	$+2.81$	96.82	84.54	$+12.29$

Appendix

Table 7: The permutation pair at full width backbone by backbone, under the nested ridge rule of Appendix B . Sorted hides which class produced which distance and unsorted does not. Both views carry the same

C+1

numbers, take the same ridge discriminant and score the same rows, so only class identity separates them. AUROC in percent.

	ImageNet-1k			CIFAR-100
Input	reported	held out	infl.	reported	held out	infl.
none (VarRatio)	41.70	41.70	$+0.00$	72.72	72.72	$+0.00$
$\{d_{(1)},\operatorname{Var}\}$ , target tuned	38.73	40.75	$+2.03$	69.80	73.22	$+3.42$
2 channels	41.99	44.78	$+2.79$	70.76	74.86	$+4.10$
4 channels	41.35	45.00	$+3.65$	70.15	74.43	$+4.27$
8 channels	33.92	38.91	$+4.98$	64.59	74.82	$+10.23$

Appendix

Table 8: The inflation at FPR@95, over the 11 ImageNet backbones and the 6 CIFAR-100 architectures with their seeds averaged inside. Columns give the rate that the usual protocol reports, the rate that survives holding the OOD dataset out, and the difference. Lower is better, so a positive difference is an understated rate. Rows and splits are those of Table 1 .

ImageNet-1k, 11 backbones
Input	SSB-hard	NINCO	iNaturalist	Textures	OpenImage-O	mean
sorted profile	$+4.03$	$+0.65$	$+1.00$	$+3.59$	$+2.08$	$+2.27$
unsorted profile	$+13.21$	$+7.29$	$+10.83$	$+16.67$	$+9.26$	$+11.45$
13 channels	$+3.57$	$+0.50$	$+0.63$	$+3.49$	$+1.48$	$+1.94$
13 random views of $z$	$+9.57$	$+6.40$	$+15.24$	$+17.07$	$+6.30$	$+10.92$
$z$ , ridge	$+15.13$	$+7.61$	$+3.76$	$+7.03$	$+2.76$	$+7.26$

Appendix

Table 9: The inflation resolved by OOD dataset. Each cell gives

w(t)-h(t)

for one target dataset in AUROC points, averaged over the 11 ImageNet backbones and over the 6 CIFAR-100 architectures with their seeds averaged inside. The last column is the mean over the datasets, which is the inflation that Table 1 reports. The two profile rows read the control that Section 5.5 leads with at 13 directions, and the ImageNet block carries the channel inventory and the whitened feature as well.

	ImageNet-1k				CIFAR-100
Input	mean	sd	min	max	mean	sd	min	max	seed sd
none (VarRatio)	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$
$\{d_{(1)},\operatorname{Var}\}$	$0.52$	$0.55$	$0.02$	$1.97$	$1.53$	$1.14$	$0.75$	$3.66$	$0.40$
2 channels	$0.75$	$0.45$	$0.09$	$1.64$	$1.98$	$0.85$	$0.56$	$3.18$	$0.67$
4 channels	$1.21$	$0.72$	$0.42$	$2.85$	$2.52$	$1.47$	$0.91$	$5.24$	$0.65$
8 channels	$1.66$	$0.72$	$0.60$	$3.30$	$4.47$	$1.98$	$2.53$	$7.89$	$0.89$

Appendix

Table 10: The spread behind the capacity sequence of Table 1 . Inflation in AUROC points. The ImageNet columns run over 11 backbones and the CIFAR-100 columns over 6 architectures with the three seeds averaged inside each. The last column gives the seed standard deviation inside an architecture, averaged over the six.

ImageNet-1k, 11 backbones, 5 OOD datasets
	sorted, permutation invariant			unsorted, class aware
$k$	held out	inflation	$\cos\theta$	held out	inflation	$\cos\theta$	gap
$1$	$88.27$	$+3.78$	$0.796$	$84.80$	$+13.65$	$0.355$	$+9.87$
$2$	$88.72$	$+3.33$	$0.832$	$87.90$	$+10.55$	$0.400$	$+7.21$
$3$	$88.85$	$+3.20$	$0.847$	$89.34$	$+9.10$	$0.423$	$+5.91$
$4$	$88.94$	$+3.11$	$0.855$	$90.11$	$+8.33$	$0.439$	$+5.22$

Appendix

Table 11: The fitting suite as a variable. For each target dataset the hold-out arm reads

k

of the others, averaged over every subset of that size. Held out AUROC and inflation in percent, and

\cos\theta

between the direction the within arm gets and the one the hold-out arm gets. The within column does not depend on

k

and sits at

92.05

and

98.44

on ImageNet and at

86.85

and

92.95

on CIFAR-100. At

k=1

the two constructions of Appendix B coincide, because one dataset has nothing to be combined with. The largest

k

reproduces Section 5.4 .

Fit	parameters	within	held out	inflation	sd
ridge	$13$	$92.09$	$90.15$	$+1.94$	$0.81$
net, hidden $8$	$153$	$92.62$	$90.08$	$+2.54$	$0.94$
net, hidden $16$	$369$	$92.82$	$90.03$	$+2.79$	$0.85$
net, hidden $32$	$993$	$93.02$	$90.06$	$+2.96$	$0.82$
net, hidden $64$	$3009$	$93.18$	$90.05$	$+3.13$	$0.82$
net, hidden $128$	$1.0\cdot 10^{4}$	$93.29$	$89.93$	$+3.36$	$0.79$

Appendix

Table 12: The parameter count as an axis, on the 13 channel inventory, over the 11 ImageNet backbones. Each row fits the same rows under the same splits and differs only in the width of the net. AUROC in percent, and the last column gives the standard deviation of the inflation across backbones.

$D$	Backbones	residual width
$512$	ResNet-18	$0$
$768$	ViT-B/16, ViT-B/16 SWAG, DeiT3-B/16	$0$
$1024$	Swin-B, ConvNeXt-B, ConvNeXtV2-B, SwinV2-B	$25$
$1280$	MobileNetV3-L	$281$
$2048$	ResNet-50 V1, ResNet-50 V2	$1049$

Appendix

Table 13: What stays outside the class mean subspace on ImageNet-1k. The subspace

V

has dimension at most

\min(C-1,D)

, which is 999 here, so the residual width is

D-\min(999,D)

Backbone	( $d_{(1)}$ )	(dim. cons.)	(derived)	optimum	gain	$\beta_{\text{opt}}$	$\tau_{\text{opt}}/T_{3}$
	$\beta=0$	$\beta=\tfrac{1}{2}$	$\beta=1$	joint
ConvNeXtV2-B	$94.53$	$94.56$	$94.58$	$94.60$	$+0.03$	$0.30$	$0.079$
SwinV2-B	$92.36$	$92.52$	$92.64$	$92.82$	$+0.19$	$1.80$	$0.596$
DeiT3-B/16	$91.20$	$91.50$	$91.74$	$92.28$	$+0.54$	$1.70$	$0.398$
ResNet-50 V1	$84.94$	$86.88$	$88.18$	$89.44$	$+1.26$	$1.10$	$0.398$
Swin-B	$88.40$	$88.64$	$88.86$	$89.97$	$+1.11$	$2.00$	$0.266$

Appendix

Table 14: Sweeping the two constants of equation 7 against the test sets themselves, on the 11 ImageNet backbones. All columns are read from one grid over

\beta\in[0,2]

and

\tau/T_{3}\in[10^{-2.5},10]

. The

\beta=1

column is the derived point

(\beta=1,\tau=T_{3})

that Section 4.2 fixes,

\beta=\tfrac{1}{2}

is the only dimensionally consistent member and

\beta=0

recovers Mahalanobis++. Gain subtracts the derived point from the joint optimum. AUROC in percent, averaged over the 5 OOD sets.

Checkpoint	$\operatorname{Var}_{c}d_{c}/d_{(1)}$	AUROC at $\alpha=0$	$\alpha^{*}$	$\tilde{\alpha}$
ConvNeXtV2-B	$555.0\to 544.6$	$98.22\to 98.21$	$0.0041$	2.3 / 1.9 / 2.2
SwinV2-B	$360.8\to 353.1$	$97.95\to 97.95$	$0.0041$	1.5 / 1.8 / 1.7
DeiT3-B/16	$418.6\to 410.9$	$96.66\to 96.66$	$0.0073$	3.1 / 3.1 / 3.7
ResNet-50 V1	$24.2\to 24.1$	$90.40\to 90.54$	$0.0761$	1.8 / 1.8 / 1.8
ResNet-50 V2	$1650.7\to 1568.0$	$90.55\to 90.64$	$0.0073$	12.1 / 12.1 / 11.5

Appendix

Table 15: Shrinkage is not what moves the constant. AUROC on OpenImage-O , which is the setting that the constant was published for. The ratio column reads the ID validation rows, while equation 8 and Table 16 read the training rows. The last column gives the dimensionless product at shrinkage

0

10^{-3}

and

10^{-2}

Backbone	ratio	Maha++	VarRatio	published	shared	held out
ConvNeXtV2-B	606.0	94.53	94.58	94.56	94.57	94.57
SwinV2-B	375.4	92.36	92.64	92.65	92.75	92.70
DeiT3-B/16	485.6	91.20	91.74	91.64	91.78	91.70
ResNet-50 V1	24.0	84.94	88.18	88.28	88.88	88.60
Swin-B	1603.4	88.40	88.86	88.74	88.87	88.80
ViT-B/16	297.4	88.39	88.51	88.49	88.52	88.52

Appendix

Table 16: The mixing constant made dimensionless, on ImageNet-1k. The ratio column gives the factor that turns

\alpha

into

\tilde{\alpha}

in equation 8 . Published applies the value from the MahaVar paper after rewriting it. Shared picks one dimensionless value on the test sets and held out picks it from the other 10 backbones. AUROC in percent, averaged over the 5 OOD sets.

Figure 7: Per backbone outcome of a fit that never sees the dataset it is scored on. Each dot subtracts the zero capacity reference from the AUROC of the held-out protocol for one backbone, and the bar gives the mean. The number above each column counts the backbones for which the zero capacity score came out ahead.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	82.05	62.00	88.47	45.39	97.65	11.10	85.40	48.62	93.57	26.44	89.43	38.71
MaxLogit	84.08	52.96	88.76	34.91	98.26	6.14	85.85	41.01	94.50	18.02	90.29	30.61
Energy	84.54	49.24	88.60	33.00	98.32	5.56	85.71	39.47	94.73	15.68	90.38	28.59
GEN ∗	85.04	55.75	91.25	34.37	98.55	4.78	89.03	36.58	96.06	15.20	91.98	29.34
KL-Matching	78.96	66.86	88.28	49.42	96.57	15.20	87.40	48.49	93.56	29.98	88.96	41.99

Appendix

Table 17: ImageNet-1k with ConvNeXtV2-B (in21k).

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	79.62	65.72	86.64	49.95	94.76	21.94	84.69	50.30	91.63	33.00	87.47	44.18
MaxLogit	83.28	54.88	87.08	40.32	95.62	13.98	83.40	46.26	91.77	24.86	88.23	36.06
Energy	84.11	50.13	85.76	39.61	94.83	14.91	80.68	52.32	90.19	26.58	87.11	36.71
GEN ∗	82.11	61.18	90.11	40.49	97.24	10.67	88.93	37.66	95.13	19.31	90.70	33.86
KL-Matching	74.94	73.14	86.60	55.63	95.05	24.32	87.64	50.07	92.79	36.00	87.40	47.83

Appendix

Table 18: ImageNet-1k with SwinV2-B (in21k).

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	74.18	75.74	80.65	64.15	90.20	38.50	75.50	67.55	87.20	48.21	81.54	58.83
MaxLogit	74.09	69.06	77.29	58.61	85.10	40.14	71.64	63.58	84.63	42.12	78.55	54.70
Energy	73.55	65.15	74.52	58.32	78.01	51.79	69.26	63.05	82.14	42.49	75.49	56.16
GEN ∗	77.95	67.56	85.99	50.99	95.36	18.09	81.66	53.28	93.00	27.46	86.79	43.48
KL-Matching	71.99	78.67	83.21	64.53	92.96	35.52	82.89	63.17	89.85	49.03	84.18	58.18

Appendix

Table 19: ImageNet-1k with DeiT3-B/16 (in21k).

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	72.08	85.04	79.94	76.35	88.41	52.85	80.43	66.26	84.85	64.10	81.14	68.92
MaxLogit	72.49	84.35	80.39	76.62	91.16	50.72	86.38	54.89	89.13	57.89	83.91	64.90
Energy	72.07	84.45	79.69	77.66	90.63	53.72	86.72	52.50	89.02	57.73	83.62	65.21
GEN ∗	72.48	86.15	82.17	76.40	92.44	45.66	85.24	60.37	89.19	60.95	84.30	65.91
KL-Matching	69.63	82.68	80.70	73.31	89.90	46.09	82.68	65.04	86.31	61.22	81.84	65.67

Appendix

Table 20: ImageNet-1k with ResNet-50 V1.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	71.30	80.34	78.51	71.55	86.47	51.06	78.71	65.25	81.68	62.42	79.33	66.13
MaxLogit	65.04	81.34	71.77	72.52	78.88	55.36	73.91	62.06	70.65	66.20	72.05	67.50
Energy	58.97	86.24	64.05	79.69	67.73	73.54	69.33	66.60	60.14	76.63	64.05	76.54
GEN ∗	74.21	78.79	83.03	64.26	92.01	34.24	84.63	53.10	87.53	49.00	84.28	55.88
KL-Matching	72.31	79.89	81.89	69.92	90.56	46.08	83.55	62.45	88.07	56.28	83.28	62.92

Appendix

Table 21: ImageNet-1k with Swin-B.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	68.93	84.78	78.11	73.19	88.19	51.42	82.97	60.51	84.82	59.84	80.60	65.95
MaxLogit	64.19	85.68	72.38	73.83	85.27	52.16	81.64	56.61	81.53	58.77	77.00	65.41
Energy	58.78	88.19	66.00	78.62	79.28	63.94	79.24	58.46	76.44	64.85	71.95	70.81
GEN ∗	70.33	85.77	81.71	71.57	92.58	40.25	87.26	52.27	89.17	52.46	84.21	60.47
KL-Matching	69.17	83.71	81.59	71.98	90.35	50.18	85.31	63.00	87.67	59.74	82.82	65.72

Appendix

Table 22: ImageNet-1k with ViT-B/16.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	68.37	85.51	76.78	76.15	84.82	59.61	75.24	72.15	82.27	66.97	77.50	72.08
MaxLogit	65.60	86.52	72.91	78.68	80.39	67.26	69.56	75.55	77.71	71.05	73.23	75.81
Energy	54.16	95.16	54.74	94.57	51.19	98.09	48.57	95.62	53.92	95.79	52.52	95.85
GEN ∗	70.47	86.10	81.68	73.89	92.29	40.31	82.88	62.09	89.03	58.61	83.27	64.20
KL-Matching	69.49	84.38	81.89	73.32	91.11	45.67	86.08	56.79	88.95	57.04	83.50	63.44

Appendix

Table 23: ImageNet-1k with ResNet-50 V2.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	76.88	74.52	84.41	60.96	94.02	27.43	85.04	53.12	90.01	43.99	86.07	52.00
MaxLogit	77.57	69.70	86.44	51.21	97.21	13.16	86.23	44.84	93.33	28.09	88.16	41.40
Energy	77.03	71.24	86.10	52.16	97.36	12.62	85.84	46.65	93.40	27.98	87.95	42.13
GEN ∗	80.64	68.56	88.84	49.12	97.50	11.69	89.12	39.70	94.74	24.46	90.17	38.70
KL-Matching	74.65	73.77	84.78	58.29	94.25	27.55	87.13	49.63	90.96	40.87	86.35	50.02

Appendix

Table 24: ImageNet-1k with ViT-B/16 SWAG.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	75.46	75.94	81.08	66.64	89.53	44.36	72.49	67.85	83.13	59.20	80.34	62.80
MaxLogit	72.92	76.69	76.43	69.67	82.61	57.67	63.28	73.24	73.39	69.03	73.73	69.26
Energy	65.56	84.28	64.03	82.77	55.71	90.42	49.03	88.56	52.69	89.49	57.40	87.10
GEN ∗	78.75	74.86	85.66	61.04	94.68	25.90	79.99	55.74	89.78	44.27	85.77	52.36
KL-Matching	74.17	80.09	83.87	68.73	92.29	43.76	86.37	58.53	90.11	51.86	85.36	60.59

Appendix

Table 25: ImageNet-1k with ConvNeXt-B.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	70.61	86.52	78.81	79.74	86.96	59.09	82.10	64.52	83.30	69.56	80.36	71.89
MaxLogit	68.30	88.97	78.88	81.05	91.07	54.33	89.03	53.53	88.64	64.16	83.18	68.41
Energy	67.65	89.83	78.17	82.07	90.72	56.02	89.24	52.34	88.53	64.45	82.86	68.94
GEN ∗	70.43	86.83	80.77	79.57	91.68	50.53	88.45	51.63	88.42	64.18	83.95	66.55
KL-Matching	68.94	85.06	80.00	77.31	89.46	48.71	83.49	64.36	85.77	62.96	81.53	67.68

Appendix

Table 26: ImageNet-1k with MobileNetV3-L.

	SSB-hard		NINCO		iNaturalist		Textures		OpenImage-O		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	67.38	89.49	77.50	81.32	87.08	58.24	78.42	71.44	81.46	73.16	78.37	74.73
MaxLogit	65.45	89.37	76.48	81.29	90.28	54.24	85.56	56.91	86.31	66.28	80.82	69.62
Energy	64.81	89.46	75.57	81.78	89.82	56.13	86.25	53.03	86.35	65.02	80.56	69.08
GEN ∗	65.83	90.62	78.18	82.95	90.64	54.59	83.11	68.23	85.52	72.54	80.66	73.79
KL-Matching	65.64	85.17	78.19	77.65	88.51	52.75	80.71	72.73	83.06	69.20	79.22	71.50

Appendix

Table 27: ImageNet-1k with ResNet-18.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	79.59	78.18	80.95	75.04	76.78	88.16	82.70	73.37	78.39	79.08	77.72	80.38	79.35	79.04
MaxLogit	80.04	77.98	81.38	74.67	78.39	88.01	84.65	71.36	79.29	78.43	77.81	80.50	80.26	78.49
Energy	79.95	78.76	81.28	75.44	78.56	88.02	85.19	69.46	79.36	78.34	77.57	81.44	80.32	78.58
GEN ∗	80.02	78.62	81.40	75.23	78.46	88.11	85.04	69.73	79.40	78.28	77.78	81.14	80.35	78.52
KL-Matching	78.96	78.70	80.86	73.98	77.08	85.44	83.07	69.71	78.79	75.09	77.72	78.22	79.41	76.86

Appendix

Table 28: CIFAR-100 with ResNet-18.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	78.41	79.56	79.73	77.31	71.24	90.41	78.56	82.53	78.10	80.67	77.50	79.99	77.26	81.74
MaxLogit	78.91	79.12	80.35	76.51	72.81	90.38	79.55	82.68	79.45	79.73	77.54	79.99	78.10	81.40
Energy	78.91	79.22	80.39	76.61	73.09	90.11	79.56	83.51	79.68	78.81	77.44	80.39	78.18	81.44
GEN ∗	79.06	79.31	80.53	76.53	72.81	90.60	79.72	83.06	79.62	79.20	77.67	80.17	78.24	81.48
KL-Matching	77.77	80.08	79.59	77.09	71.89	89.05	78.63	80.77	78.12	78.94	77.23	79.81	77.21	80.96

Appendix

Table 29: CIFAR-100 with ResNet-34.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	81.72	74.96	82.33	73.36	72.25	91.72	78.67	79.22	77.70	82.21	78.82	80.39	78.58	80.31
MaxLogit	81.83	75.33	82.62	73.46	73.83	91.32	79.82	78.27	78.19	82.13	78.57	81.15	79.14	80.28
Energy	81.74	75.91	82.59	73.77	74.38	89.81	80.27	76.16	78.30	81.74	78.24	82.40	79.25	79.96
GEN ∗	81.76	75.88	82.60	73.76	74.33	89.87	80.26	76.14	78.30	81.71	78.26	82.36	79.25	79.95
KL-Matching	81.38	76.11	82.41	72.60	72.52	91.12	79.12	76.83	78.33	79.18	79.08	78.75	78.81	79.10

Appendix

Table 30: CIFAR-100 with WRN-28-10.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	77.97	79.86	78.02	77.85	76.81	84.37	78.50	77.60	74.08	82.80	75.89	81.31	76.88	80.63
MaxLogit	78.75	79.25	79.67	75.66	77.20	83.03	81.93	73.41	75.80	80.71	76.25	81.41	78.27	78.91
Energy	78.49	80.14	79.52	76.28	76.99	83.50	82.02	72.97	75.73	80.55	75.93	82.45	78.11	79.31
GEN ∗	79.10	79.62	79.89	75.90	78.15	83.67	82.45	72.49	76.20	80.70	76.63	81.78	78.74	79.03
KL-Matching	77.33	80.53	78.02	77.56	79.71	77.86	79.84	72.68	75.72	77.35	75.90	80.70	77.75	77.78

Appendix

Table 31: CIFAR-100 with DenseNet-BC.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	74.16	83.62	75.72	81.09	72.05	88.00	72.47	86.95	75.05	81.81	73.53	83.58	73.83	84.18
MaxLogit	72.77	82.98	74.72	78.74	73.34	85.37	74.83	83.02	76.33	76.01	70.85	82.26	73.81	81.40
Energy	72.72	83.18	74.76	78.45	73.76	84.69	75.28	81.62	76.65	74.31	70.72	82.17	73.98	80.74
GEN ∗	74.16	82.38	76.12	78.05	73.98	85.96	75.31	82.33	76.75	75.93	72.96	81.48	74.88	81.02
KL-Matching	74.32	83.68	76.42	81.14	74.74	86.50	72.95	87.14	75.20	82.55	74.32	83.51	74.66	84.09

Appendix

Table 32: CIFAR-100 with VGG16-BN.

	CIFAR-10		TIN		MNIST		SVHN		Textures		Places365		Avg
Method	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR	AUROC	FPR
MSP	72.91	84.68	75.23	81.64	65.23	94.39	63.61	88.47	70.33	84.95	72.88	84.30	70.03	86.41
MaxLogit	73.05	83.97	79.34	76.48	69.63	94.87	72.90	87.05	77.64	79.93	77.45	80.38	75.00	83.78
Energy	72.65	84.56	79.30	76.46	69.80	95.19	73.44	87.57	77.96	79.47	77.50	80.40	75.11	83.94
GEN ∗	73.14	84.27	79.47	76.38	69.71	95.09	73.38	86.95	77.87	79.65	77.41	80.51	75.16	83.81
KL-Matching	71.77	85.24	74.96	80.81	70.58	83.89	68.79	77.14	72.15	76.40	71.96	82.39	71.70	80.98

Appendix

Table 33: CIFAR-100 with MobileNetV2.

Explore similar work

Sep 1, 2026cs.CV

Fi-ImageNet-1k: An OOD Benchmark From the Inside of the ImageNet-1k Validation Set

Out-of-distribution (OOD) detection predicts whether a test image belongs to none of the predefined classes. To evaluate this task, benchmarks need images from outside the in-distribution (ID) data; typically, these are defined or collected in an ad hoc fashion. Since no ground truth is perfect, ID-labeled datasets themselves contain a natural source of OOD images. We exploit such annotation errors and present Fi-ImageNet-1k, an OOD dataset built from ImageNet-1k validation images that the recent ReImageNet reannotation effort assigned to no ImageNet-1k class. Each image was examined by expert human annotators supported by evidence from MLLMs, VLMs, and reverse image search, comparing it against all visually similar ID classes. We keep only images that could be assigned a specific class outside the ImageNet-1k label space. The resulting Fi-ImageNet-1k, with 655 images from 522 classes, is substantially more challenging than any commonly used OOD dataset. No evaluated combination of classifier and OOD detector achieves a false positive rate below 51% at 95% true positive rate (FPR@95). Compared to the recent NINCO, our dataset is 3.8x more challenging in the FPR@95 metric for state-of-the-art supervised OOD detection methods.

Ruslan Rozumnyi, Matěj Suchánek, Tomáš Vojíř +2

Visual Recognition Group · Department of Cybernetics · Faculty of Electrical Engineering +1

Out-Of-Distribution Detection Imagenet In-Distribution Validation

Ask PDF

Jul 29, 2026cs.CV

Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer

Selecting a zero-shot out-of-distribution (OOD) detector for a new deployment is typically based on benchmark rankings, implicitly assuming that the highest-ranked detector will transfer across domains. We show that this assumption does not hold. Through a controlled portability audit across seventeen in-distribution datasets, three vision-language models, and seven representative zero-shot OOD detectors, we find that detector rankings reverse across deployments, every detector exceeds

80\%

FPR95 on at least one domain, and the preferred detector depends on both the in-distribution data and the underlying VLM. We trace these reversals to complementary evidence channels in vision-language logits. Corpus-free detectors rely on different combinations of absolute match level and relative or spatial sharpness, while WordNet-based methods additionally depend on external semantic coverage. A simple proposition shows that level and sharpness cannot generally be recovered from one another, explaining why no single detector transfers reliably across deployments. Motivated by this diagnosis, we introduce the Complementary Evidence Guard (CEG), a detector-agnostic wrapper that preserves complementary evidence through a non-compensatory fusion of the base detector, level, and sharpness using only empirical in-distribution percentiles. Controls replacing these channels with entropy, logit variance, or random noise do not reproduce the gains. Without OOD samples, auxiliary corpora, or learned fusion, CEG reduces detector sensitivity and improves GL-MCM from

38.1

28.8

and MCM from

42.6

30.5

family-balanced FPR95.

Ignacio M. De la Jara, Cristian Rodriguez-Opazo, Stephen Gould +1

University of Adelaide · 3Naval Group Pacific · 2Australian National University

Out-Of-Distribution Detection In-Distribution Zero-Shot Robustness In-Domain Accuracy Sharpness-Aware Minimization Text Corpora

Ask PDF

Jun 29, 2026cs.LG

Exploiting Local Flatness for Efficient Out-of-Distribution Detection

Detecting out-of-distribution (OOD) data is crucial for reliable machine learning deployment. Among detection strategies, post-hoc methods are particularly attractive due to their efficiency, as they operate directly on pre-trained networks without requiring retraining. Within this paradigm, one promising direction exploits loss-landscape curvature to estimate model uncertainty; however, such methods incur substantial computational cost and rely on implicit assumptions about how landscape flatness differs between in-distribution (ID) and OOD data. In this work, we provide the first systematic investigation of this curvature discrepancy and show that OOD inputs exhibit larger Hessian curvature than ID data, with the gap widening under stronger distributional shifts. Motivated by these observations, we propose Fold, a lightweight flatness-modulated OOD detector that leverages the feature Hessian and partial feature normalization to improve ID-OOD separability while avoiding costly parameter-space curvature approximations. To optimally adapt this normalization across diverse datasets, we further introduce AutoFold, a self-supervised tuning scheme that synthesizes pseudo-OOD samples via ID logit masking for automatic calibration without requiring external data. Experiments on OOD benchmarks show that Fold outperforms prior methods, improving the average AUROC by 1.63% and reducing FPR95 by 2.30%, while maintaining computational efficiency comparable to a standard forward pass. Supported by theoretical analysis and extensive ablations, Fold provides a principled and practical solution for robust real-world deployment.

Seonghwan Park, Hyunji Jung, Dongyeop Lee +1

Korea Electronics Technology Institute (KETI) · Pohang University of Science and Technology (POSTECH)

Out-Of-Distribution Detection Out-Of-Distribution In-Distribution Distribution Shifts

Ask PDF