Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

Rahul D Ray

cs.LGApr 10, 2026

Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

Authors: Rahul D Ray

Organizations: Department of Electronics and Electrical Engineering, BITS Pilani, Hyderabad Campus

Abstract

The ability to detect out-of-distribution (OOD) inputs is fundamental to safe deployment of machine learning systems. Yet, current methods often rely on feature representations that are optimised solely for classification accuracy, neglecting the distinct requirements of epistemic uncertainty. We introduce GOEN (Geometry-Optimised Epistemic Network), a simple pipeline that combines multi-scale features, L2 normalisation, Mahalanobis distance, and a calibration head trained with real hard OOD examples. Through systematic ablation we uncover a counter-intuitive finding: CenterLoss, a popular regulariser for feature compactness, significantly degrades OOD detection performance, reducing average OOD AUROC from 0.9483 to 0.9366 despite improving classification accuracy. The best variant, GOEN-NoCenterLoss, achieves an average OOD AUROC of 0.9483, surpassing all baselines including deep ensembles (0.8827), KNN (0.8967), and ODIN (0.8870) on CIFAR-10 benchmarks, while maintaining competitive in-distribution accuracy. Our results challenge the prevailing assumption that better classification geometry automatically leads to better epistemic uncertainty. Instead, we show that overly tight feature clusters compress inter-class margins and distort the covariance structure needed for effective OOD detection. GOEN is efficient, training in under 20 minutes on a single GPU, and provides a practical blueprint for building AI systems that reliably recognise their own limitations.

Figures & tables

Figure 1: Summary of key experimental results. Top left: Phase 1 training (cross-entropy + CenterLoss) showing validation accuracy vs. epoch. Top center: Phase 3 calibration convergence showing uncertainty score vs. epoch. Top right: Feature compactness measured by inter/intra class ratio. Bottom left: Ablation study comparing different model variants. Bottom right: Reproducibility study showing AUROC vs. epoch across multiple runs.

Model	Uncertainty Score
Standard NN, Temperature Scaling	$1-\max(\text{softmax})$
MC Dropout, EpiNet	Mutual information between predictions and stochastic parameters
Deep Ensemble	Sum of per - class variances across ensemble members
Evidential Deep Learning	Vacuity $=K/\sum\alpha_{c}$
Mixture of Experts	$1-\max(\text{gate weights})$
Energy	Energy score $E(x)$ (higher is more OOD)

Table 1: Conversion of model outputs to OOD uncertainty scores.

Figure 2: Architecture of GOEN (Geometry-Optimised Epistemic Network). A ResNet-18 backbone (trained with cross-entropy only, no CenterLoss) extracts multi-scale features from layers 2 (texture, 128-dim) and 4 (semantic, 512-dim). After global average pooling, each is L2-normalised to the unit sphere, then concatenated into a 640-dim vector $\tilde{z}$ . Class means $\mu_{c}$ and tied covariance $\Sigma$ are fitted on the training set. The Mahalanobis score $m_{1}=\min_{c}(\tilde{z}-\mu_{c})^{T}\Sigma^{-1}(\tilde{z}-\mu_{c})$ is combined with two additional uncertainty features: cosine similarity $m_{2}$ from the FC head logits and entropy $m_{3}$ . These three features feed into a calibration head $g_{\psi}$ (MLP) trained with binary cross-entropy using synthetic noise and SVHN as hard OOD examples (targets: 0.05 for ID, 0.95 for OOD). The final output $u\in(0,1)$ is the OOD score (high = OOD).

Metric	StandardNN	TempScaling	MCDropout	Deep Ensemble	EDL	EpiNet	MoE - K5	Energy	ODIN	Mahalanobis	KNN	GOEN - NoCenterLoss
ID Acc $\uparrow$	0.8706	0.8706	0.9301	0.9534	0.8197	0.8902	0.8842	—	—	—	—	0.9311
ID ECE $\downarrow$	0.0223	0.0149	0.0353	0.0208	0.1944	0.0184	0.0179	—	—	—	—	—
ID NLL $\downarrow$	0.3794	0.3776	0.2627	0.1571	0.8018	0.3314	0.3565	—	—	—	—	—
ID Brier $\downarrow$	0.1893	0.1886	0.1108	0.0745	0.3130	0.1602	0.1715	—	—	—	—	—
SVHN AUROC $\uparrow$	0.8593	0.8603	0.8754	0.8769	0.8427	0.8777	0.7207	0.8377	0.8466	0.6066	0.8503	0.9372
CIFAR - 100 AUROC $\uparrow$	0.8266	0.8292	0.8387	0.8779	0.7952	0.8470	0.6440	0.8547	0.8562	0.5746	0.8487	0.9079

Table 2: Performance comparison of baseline models and GOEN on CIFAR-10 (ID) and OOD datasets. Metrics are rows; models are columns. Best per row in bold . Dashes (

-

) indicate that the metric is not applicable: post-hoc OOD detectors (Energy, ODIN, Mahalanobis, KNN) do not produce in - distribution predictions, so ID metrics (Accuracy, ECE, NLL, Brier) are omitted. GOEN-NoCenterLoss is the variant without CenterLoss, which achieved the best OOD performance in our ablation study.

Model	ID Acc	SVHN AUROC	CIFAR-100 AUROC	Synthetic AUROC	Avg AUROC
GOEN-Default	0.9340	0.9163	0.8955	0.9980	0.9366
NoCenterLoss	0.9311	0.9372	0.9079	1.0000	0.9483
SingleScale-L4	0.9351	0.8988	0.8967	0.9857	0.9270
NoSVHN-NoiseOnly	0.9352	0.9222	0.8914	0.9997	0.9377

Table 3: Ablation results for GOEN variants (seed=42).

Metric	Mean	Std
ID Accuracy	0.9344	0.0013
ID ECE	0.0355	0.0013
ID NLL	0.2345	0.0074
ID Brier	0.1044	0.0028
OOD SVHN AUROC	0.9233	0.0134
OOD CIFAR-100 AUROC	0.8930	0.0039

Table 4: Seeding results for GOEN-Default (mean

\pm

std over 5 seeds).

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 223 papers

MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion
Jun 15, 2026Rahim Hossain, Md Tawheedul Islam Bhuian, Md Farhan Shadiq +1Diffusion-Based OOD Detection OOD Detection
School of Computing State University of New York at Binghamton
Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection
Mar 24, 2026Shreen Gul, Mohamed Elmahallawy, Ardhendu Tripathy +1OOD Detection
Missouri University of Science and Technology, Rolla, MO 65401, USA · Washington State University, Richland, WA 99354, USA
CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring
Mar 18, 2026Jin Mo Yang, Hyung-Sin Kim, Saewoong BahkOOD Detection
Seoul National University, Seoul, South Korea

Jun 15, 2026J/K move · Enter open · S save

MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion

Rahim Hossain, Md Tawheedul Islam Bhuian, Md Farhan Shadiq, Kyoung-Don Kang

School of Computing State University of New York at Binghamton

Diffusion-Based OOD Detection OOD Detection

Open paper AskPDF arXiv

We introduce MM++ (Multilayer Mahalanobis++), a strictly post-hoc, and scale-invariant framework for Out-of-Distribution (OOD) detection. To address the trade-off between scale invariance and hierarchical expressivity, MM++ constructs a principled joint feature space. It first identifies discriminative intermediate layers by measuring entropy density drops, which mark the boundaries of sharp semantic compression. By fusing these selected layers with the terminal representation, the framework captures latent cross-layer correlations while mitigating early-layer noise. Crucially, a Ledoit-Wolf regularized tied covariance matrix stabilizes this unified space, enabling reliable distance estimation. Requiring no auxiliary OOD data, classifier fine-tuning, or architectural modifications, MM++ delivers robust performance across distinct architectures for both near- and far-OOD detection.

Abstract

OOD Detection

Explore similar work

MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion

Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring