cs.LGApr 10, 2026

Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

Authors: Rahul D Ray

Organizations: Department of Electronics and Electrical Engineering, BITS Pilani, Hyderabad Campus

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The ability to detect out-of-distribution (OOD) inputs is fundamental to safe deployment of machine learning systems. Yet, current methods often rely on feature representations that are optimised solely for classification accuracy, neglecting the distinct requirements of epistemic uncertainty. We introduce GOEN (Geometry-Optimised Epistemic Network), a simple pipeline that combines multi-scale features, L2 normalisation, Mahalanobis distance, and a calibration head trained with real hard OOD examples. Through systematic ablation we uncover a counter-intuitive finding: CenterLoss, a popular regulariser for feature compactness, significantly degrades OOD detection performance, reducing average OOD AUROC from 0.9483 to 0.9366 despite improving classification accuracy. The best variant, GOEN-NoCenterLoss, achieves an average OOD AUROC of 0.9483, surpassing all baselines including deep ensembles (0.8827), KNN (0.8967), and ODIN (0.8870) on CIFAR-10 benchmarks, while maintaining competitive in-distribution accuracy. Our results challenge the prevailing assumption that better classification geometry automatically leads to better epistemic uncertainty. Instead, we show that overly tight feature clusters compress inter-class margins and distort the covariance structure needed for effective OOD detection. GOEN is efficient, training in under 20 minutes on a single GPU, and provides a practical blueprint for building AI systems that reliably recognise their own limitations.

Figures & tables

Figure 1: Summary of key experimental results. Top left: Phase 1 training (cross-entropy + CenterLoss) showing validation accuracy vs. epoch. Top center: Phase 3 calibration convergence showing uncertainty score vs. epoch. Top right: Feature compactness measured by inter/intra class ratio. Bottom left: Ablation study comparing different model variants. Bottom right: Reproducibility study showing AUROC vs. epoch across multiple runs.
ModelUncertainty Score
Standard NN, Temperature Scaling1−max⁡(softmax)1-\max(\text{softmax})
MC Dropout, EpiNetMutual information between predictions and stochastic parameters
Deep EnsembleSum of per - class variances across ensemble members
Evidential Deep LearningVacuity =K/∑αc=K/\sum\alpha_{c}
Mixture of Experts1−max⁡(gate weights)1-\max(\text{gate weights})
EnergyEnergy score E(x)E(x) (higher is more OOD)
Table 1: Conversion of model outputs to OOD uncertainty scores.
Figure 2: Architecture of GOEN (Geometry-Optimised Epistemic Network). A ResNet-18 backbone (trained with cross-entropy only, no CenterLoss) extracts multi-scale features from layers 2 (texture, 128-dim) and 4 (semantic, 512-dim). After global average pooling, each is L2-normalised to the unit sphere, then concatenated into a 640-dim vector z~\tilde{z} . Class means μc\mu_{c} and tied covariance Σ\Sigma are fitted on the training set. The Mahalanobis score m1=min⁡c(z~−μc)TΣ−1(z~−μc)m_{1}=\min_{c}(\tilde{z}-\mu_{c})^{T}\Sigma^{-1}(\tilde{z}-\mu_{c}) is combined with two additional uncertainty features: cosine similarity m2m_{2} from the FC head logits and entropy m3m_{3} . These three features feed into a calibration head gψg_{\psi} (MLP) trained with binary cross-entropy using synthetic noise and SVHN as hard OOD examples (targets: 0.05 for ID, 0.95 for OOD). The final output u∈(0,1)u\in(0,1) is the OOD score (high = OOD).
MetricStandardNNTempScalingMCDropoutDeep EnsembleEDLEpiNetMoE - K5EnergyODINMahalanobisKNNGOEN - NoCenterLoss
ID Acc ↑\uparrow0.87060.87060.93010.95340.81970.89020.8842————0.9311
ID ECE ↓\downarrow0.02230.01490.03530.02080.19440.01840.0179—————
ID NLL ↓\downarrow0.37940.37760.26270.15710.80180.33140.3565—————
ID Brier ↓\downarrow0.18930.18860.11080.07450.31300.16020.1715—————
SVHN AUROC ↑\uparrow0.85930.86030.87540.87690.84270.87770.72070.83770.84660.60660.85030.9372
CIFAR - 100 AUROC ↑\uparrow0.82660.82920.83870.87790.79520.84700.64400.85470.85620.57460.84870.9079
Table 2: Performance comparison of baseline models and GOEN on CIFAR-10 (ID) and OOD datasets. Metrics are rows; models are columns. Best per row in bold . Dashes ( −- ) indicate that the metric is not applicable: post-hoc OOD detectors (Energy, ODIN, Mahalanobis, KNN) do not produce in - distribution predictions, so ID metrics (Accuracy, ECE, NLL, Brier) are omitted. GOEN-NoCenterLoss is the variant without CenterLoss, which achieved the best OOD performance in our ablation study.
ModelID AccSVHN AUROCCIFAR-100 AUROCSynthetic AUROCAvg AUROC
GOEN-Default0.93400.91630.89550.99800.9366
NoCenterLoss0.93110.93720.90791.00000.9483
SingleScale-L40.93510.89880.89670.98570.9270
NoSVHN-NoiseOnly0.93520.92220.89140.99970.9377
Table 3: Ablation results for GOEN variants (seed=42).
MetricMeanStd
ID Accuracy0.93440.0013
ID ECE0.03550.0013
ID NLL0.23450.0074
ID Brier0.10440.0028
OOD SVHN AUROC0.92330.0134
OOD CIFAR-100 AUROC0.89300.0039
Table 4: Seeding results for GOEN-Default (mean ±\pm std over 5 seeds).
More in this topic

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 223 papers

Explore similar work

CardsList
Jun 15, 2026

MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion

We introduce MM++ (Multilayer Mahalanobis++), a strictly post-hoc, and scale-invariant framework for Out-of-Distribution (OOD) detection. To address the trade-off between scale invariance and hierarchical expressivity, MM++ constructs a principled joint feature space. It first identifies discriminative intermediate layers by measuring entropy density drops, which mark the boundaries of sharp semantic compression. By fusing these selected layers with the terminal representation, the framework captures latent cross-layer correlations while mitigating early-layer noise. Crucially, a Ledoit-Wolf regularized tied covariance matrix stabilizes this unified space, enabling reliable distance estimation. Requiring no auxiliary OOD data, classifier fine-tuning, or architectural modifications, MM++ delivers robust performance across distinct architectures for both near- and far-OOD detection.

Rahim Hossain, Md Tawheedul Islam Bhuian, Md Farhan Shadiq +1

School of Computing State University of New York at Binghamton

Diffusion-Based OOD DetectionOOD Detection
AskPDF
Mar 24, 2026

Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection

Deep learning models are increasingly deployed in safety-critical applications, where reliable out-of-distribution (OOD) detection is essential to ensure robustness. Existing methods predominantly rely on the penultimate-layer activations of neural networks, assuming they encapsulate the most informative in-distribution (ID) representations. In this work, we revisit this assumption to show that intermediate layers encode equally rich and discriminative information for OOD detection. Based on this observation, we propose a simple yet effective model-agnostic approach that leverages internal representations across multiple layers. Our scheme aggregates features from successive convolutional blocks, computes class-wise mean embeddings, and applies L_2 normalization to form compact ID prototypes capturing class semantics. During inference, cosine similarity between test features and these prototypes serves as an OOD score--ID samples exhibit strong affinity to at least one prototype, whereas OOD samples remain uniformly distant. Extensive experiments on state-of-the-art OOD benchmarks across diverse architectures demonstrate that our approach delivers robust, architecture-agnostic performance and strong generalization for image classification. Notably, it improves AUROC by up to 4.41% and reduces FPR by 13.58%, highlighting multi-layer feature aggregation as a powerful yet underexplored signal for OOD detection, challenging the dominance of penultimate-layer-based methods. Our code is available at: https://github.com/sgchr273/cosine-layers.git.

Shreen Gul, Mohamed Elmahallawy, Ardhendu Tripathy +1

Missouri University of Science and Technology, Rolla, MO 65401, USA · Washington State University, Richland, WA 99354, USA

OOD Detection
AskPDF
Mar 18, 2026

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitation: logit-based methods see only the classifier's confidence signal, while feature-based methods attempt to measure membership in the training distribution but do so in the full feature space where confidence and membership are entangled, inheriting architecture-sensitive failure modes. We observe that penultimate features naturally decompose into two orthogonal subspaces: a classifier-aligned component encoding confidence, and a residual the classifier discards. We discover that this residual carries a class-specific directional signature for in-distribution data -- a membership signal invisible to logit-based methods and entangled with noise in feature-based methods. We propose CORE (COnfidence + REsidual), which disentangles the two signals by scoring each subspace independently and combines them via normalized summation. Because the two signals are orthogonal by construction, their failure modes are approximately independent, producing robust detection where either view alone is unreliable. CORE achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations, ranking first in three of five settings and achieving the highest grand average AUROC with negligible computational overhead.

Jin Mo Yang, Hyung-Sin Kim, Saewoong Bahk

Seoul National University, Seoul, South Korea

OOD Detection
AskPDF