Out-of-distribution (OOD) detection is a well-known challenge due to deep models often producing overconfident. In this paper, we reveal a key insight that trained classifiers tend to rely on sparse parameter contribution patterns, meaning that only a few dominant parameters drive predictions. This brittleness can be exploited by OOD inputs that anomalously trigger these parameters, resulting in overconfident predictions. To address this issue, we propose a simple yet effective method called Shaping Parameter Contribution Patterns (SPCP), which enhances OOD detection robustness by encouraging the classifier to learn boundary-oriented dense contribution patterns. Specifically, SPCP operates during training by rectifying excessively high parameter contributions based on a dynamically estimated threshold. This mechanism promotes the classifier to rely on a broader set of parameters for decision-making, thereby reducing the risk of overconfident predictions caused by anomalously triggered parameters, while preserving in-distribution (ID) performance. Extensive experiments under various OOD detection setups verify the effectiveness of SPCP.
Figures & tables
Figure 2: An illustrative example of parameter contribution patterns in a ResNet-18 classifier trained on CIFAR-10 using cross-entropy loss. Units are sorted in the same order. This example focuses on the ‘airplane’ class, showcasing the contributions of each unit in the classifier weights to the model outputs for both ID and OOD samples. The parameter contributions are defined with reference to Eq. ( 4 ). Since model outputs are directly determined by parameter contributions, OOD input that anomalously triggers dominant parameters can be confidently yet incorrectly classified as ID categories and hurt OOD detection.
Figure 4: Comparison of the classifier’s parameter contribution patterns before and after applying SPCP, with the average contribution matrix on the ID test set sorted for clarity.
Figure 6: Comparison of normalized OOD score distributions before and after applying SPCP.
ID datasets
CIFAR-10
CIFAR-100
OOD datasets
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
Vanilla training / SPCP training (Ours)
Near-OOD
CIFAR-10
–
–
59.21 ± 0.75 / 60.10 ± 0.95
79.05 ± 0.11 / 79.09 ± 0.22
CIFAR-100
66.60 ± 4.46 / 35.74 ± 0.51
86.36 ± 0.58 / 90.42 ± 0.17
–
–
TIN
56.08 ± 4.83 / 27.59 ± 0.48
88.80 ± 0.36 / 92.72 ± 0.07
52.03 ± 0.50 / 50.33 ± 0.85
82.76 ± 0.08 / 83.40 ± 0.28
Average
61.34 ± 4.63 / 31.67 ± 0.25
87.58 ± 0.46 / 91.57 ± 0.11
55.62 ± 0.61 / 55.21 ± 0.85
80.91 ± 0.08 / 81.25 ± 0.19
Table 1: OOD detection performance on CIFAR benchmarks. All values are percentages, and the best results are in bold. ↑ indicates that larger values are better, ↓ indicates that smaller values are better.
ID datasets
ImageNet-200
OOD datasets
FPR95 ↓
AUROC ↑
Vanilla training / SPCP training (Ours)
Near-OOD
SSB-hard
69.77 ± 0.32 / 70.23 ± 0.21
79.83 ± 0.02 / 79.80 ± 0.05
NINCO
50.70 ± 0.89 / 49.32 ± 0.40
85.17 ± 0.11 / 85.24 ± 0.06
Average
60.24 ± 0.57 / 59.77 ± 0.25
82.50 ± 0.05 / 82.52 ± 0.03
Far-OOD
iNaturalist
26.41 ± 2.29 / 23.86 ± 0.94
92.55 ± 0.50 / 92.91 ± 0.08
Table 2: OOD detection performance on large-scale ImageNet benchmark.
Method
CIFAR-10
CIFAR-100
Near-OOD
Far-OOD
ID ACC ↑
Near-OOD
Far-OOD
ID ACC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
Post-hoc methods (vanilla training with cross-entropy)
MSP ( Hendrycks and Gimpel 2017 )
48.17 ± 3.92
88.03 ± 0.25
31.72 ± 1.84
90.73 ± 0.43
95.06 ± 0.30
54.80 ± 0.33
80.27 ± 0.11
58.70 ± 1.06
77.76 ± 0.44
77.25 ± 0.10
Energy ( Liu et al. 2020 )
61.34 ± 4.63
87.58 ± 0.46
41.69 ± 5.32
91.21 ± 0.92
95.06 ± 0.30
55.62 ± 0.61
80.91 ± 0.08
56.59 ± 1.38
79.77 ± 0.61
77.25 ± 0.10
DICE ( Sun and Li 2022 )
70.04 ± 7.64
78.34 ± 0.79
51.76 ± 4.42
84.23 ± 1.89
95.06 ± 0.30
57.95 ± 0.53
79.38 ± 0.23
56.25 ± 0.60
80.01 ± 0.18
77.25 ± 0.10
Table 3: Comparison on CIFAR benchmarks. All values are percentages, and OOD detection results are averaged over multiple OOD datasets. Detailed results for each OOD dataset are provided in Appendix D. The best results are in bold , with the second and third best results underlined .
Method
Near-OOD
Far-OOD
ID ACC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
Post-hoc methods (vanilla training with cross-entropy)
MSP ( Hendrycks and Gimpel 2017 )
54.82 ± 0.35
83.34 ± 0.06
35.43 ± 0.38
90.13 ± 0.09
86.37 ± 0.08
Energy ( Liu et al. 2020 )
60.24 ± 0.57
82.50 ± 0.05
34.86 ± 1.30
90.86 ± 0.21
86.37 ± 0.08
DICE ( Sun and Li 2022 )
61.88 ± 0.67
81.78 ± 0.14
36.51 ± 1.18
90.80 ± 0.31
86.37 ± 0.08
ReAct ( Sun et al. 2021 )
62.49 ± 2.19
81.87 ± 0.98
28.50 ± 0.95
92.31 ± 0.56
86.37 ± 0.08
Table 4: Comparison on large-scale ImageNet benchmark.
Stage
Near-OOD
Far-OOD
ID ACC ↑
Training
Inferring
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
×
×
55.62 ± 0.61
80.91 ± 0.08
56.59 ± 1.38
79.77 ± 0.61
77.25 ± 0.10
✓
×
55.49 ± 0.77
81.09 ± 0.21
51.96 ± 2.25
81.71 ± 1.45
77.70 ± 0.24
×
✓
60.28 ± 0.46
79.50 ± 0.08
53.66 ± 3.13
80.59 ± 1.10
76.94 ± 0.12
✓
✓
55.21 ± 0.85
81.25 ± 0.19
50.59 ± 1.83
82.17 ± 1.32
77.70 ± 0.20
Table 5: Ablation study of contribution truncation in different stages on the CIFAR-100 benchmark.
Figure 7: Ablation studies on the hyperparameters: (a) effect of varying the percentile ρ used for threshold estimation; (b) impact of the EMA smoothing factor β ; (c) influence of initial threshold λ0 . The OOD results are averaged over both near- and far-OOD groups on the CIFAR-100 benchmark.
Model
Method
Near-OOD
Far-OOD
ID ACC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
ResNet-18
Vanilla
55.62 ± 0.61
80.91 ± 0.08
56.59 ± 1.38
79.77 ± 0.61
77.25 ± 0.10
SPCP
55.21 ± 0.85
81.25 ± 0.19
50.59 ± 1.83
82.17 ± 1.32
77.70 ± 0.20
WideResNet-28-10
Vanilla
54.48 ± 1.30
82.31 ± 0.41
55.34 ± 1.67
80.87 ± 0.80
80.53 ± 0.13
SPCP
53.48 ± 0.52
82.63 ± 0.24
52.84 ± 0.93
81.69 ± 0.31
80.56 ± 0.22
DenseNet-101
Vanilla
61.57 ± 1.36
79.27 ± 0.36
65.49 ± 2.45
76.16 ± 0.81
76.54 ± 0.42
Table 6: Generalization to different backbones on the CIFAR-100 benchmark.
Method
Near-OOD
Far-OOD
ID ACC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
MSP ( Hendrycks and Gimpel 2017 )
54.80 ± 0.33
80.27 ± 0.11
58.70 ± 1.06
77.76 ± 0.44
77.25 ± 0.10
MSP+SPCP
55.04 ± 0.68
80.41 ± 0.28
54.31 ± 0.95
80.22 ± 0.93
77.70 ± 0.20
Energy ( Liu et al. 2020 )
55.62 ± 0.61
80.91 ± 0.08
56.59 ± 1.38
79.77 ± 0.61
77.25 ± 0.10
Energy+SPCP
55.21 ± 0.85
81.25 ± 0.19
50.59 ± 1.83
82.17 ± 1.32
77.70 ± 0.20
GradNorm ( Huang et al. 2021 )
85.58 ± 0.46
70.13 ± 0.47
83.68 ± 1.92
69.14 ± 1.05
77.25 ± 0.10
Table 7: Compatibility with other OOD detection methods on the CIFAR-100 benchmark.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Visualization of the classifier’s parameter contribution patterns, with the average contribution matrix on the ID test set sorted for improved clarity.
Method
Near-OOD
Far-OOD
ID ACC ↑
FPR95 ↓
AUROC ↑
FPR95 ↓
AUROC ↑
Vanilla
55.62 ± 0.61
80.91 ± 0.08
56.59 ± 1.38
79.77 ± 0.61
77.25 ± 0.10
+ L2 (0.1)
63.48 ± 0.85
79.13 ± 0.37
65.03 ± 2.27
78.19 ± 0.59
76.90 ± 0.55
+ L2 (0.01)
56.66 ± 1.05
80.75 ± 0.15
57.20 ± 1.28
79.34 ± 1.17
78.31 ± 0.21
+ L2 (0.001)
56.54 ± 1.35
80.65 ± 0.32
53.93 ± 1.93
80.70 ± 1.18
77.60 ± 0.31
+ L2 (0.0001)
55.87 ± 1.32
80.93 ± 0.38
54.00 ± 1.84
80.85 ± 0.97
77.40 ± 0.28
Appendix
Table 8: Comparison of L2 weight regularization and SPCP on the CIFAR-100 benchmark.
Dataset
Methods
Latency (s)
Training
Inferring
CIFAR-10
Vanilla
7.82
2.35
+LogitNorm
7.99
2.37
+SPCP
8.19
2.48
ImageNet-200
Vanilla
515.71
10.94
+LogitNorm
518.43
10.97
Appendix
Table 9: Analysis of the training and inference overhead introduced by SPCP. This overhead is measured by latency ( ↓ ), defined as the time required to process the entire training set during training and the entire test set during inference in a single run. Lower latency indicates reduced computational cost. All experiments are conducted under consistent software and hardware settings as detailed in Appendix C , with results averaged over multiple independent runs.
CIFAR-10
ImageNet-200
Near-OOD
Far-OOD
Near-OOD
Far-OOD
ITP
52.72 / 87.31
35.70 / 90.30
60.09 / 82.68
32.17 / 91.28
SPCP
31.67 / 91.57
20.54 / 94.55
59.77 / 82.52
30.43 / 91.46
Appendix
Table 10: Comparison of OOD Detection Performance between SPCP and ITP.
Near-OOD
Far-OOD
Fixed
35.19 / 90.86
26.69 / 92.53
Ours
31.67 / 91.57
20.54 / 94.55
Appendix
Table 11: Ablation study on the adaptive λ conducted on the CIFAR-10 benchmark. Results (FPR95 ↓ / AUROC ↑ ) are averaged over multiple runs.
Kinetics-Sound
Vanilla
76.95 / 69.67
Ours
69.20 / 73.48
Appendix
Table 12: Extend SPCP to audio OOD detection tasks. Results (FPR95 ↓ / AUROC ↑ ) are averaged over multiple runs.
λ=β⋅λ+(1−β)⋅∣B∣1∑xi∈BTop(ρ,C(xi))
Appendix
Algorithm 1 Training Procedure of SPCP
Method
Near-OOD
Far-OOD
ID ACC
CIFAR-100
TIN
Average
MNIST
SVHN
Textures
Places365
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
87.19 ± 0.33
88.87 ± 0.19
88.03 ± 0.25
92.63 ± 1.57
91.46 ± 0.40
89.89 ± 0.71
88.92 ± 0.47
90.73 ± 0.43
95.06 ± 0.30
Energy
86.36 ± 0.58
88.80 ± 0.36
87.58 ± 0.46
94.32 ± 2.53
91.79 ± 0.98
89.47 ± 0.70
89.25 ± 0.78
91.21 ± 0.92
95.06 ± 0.30
DICE
77.01 ± 0.88
79.67 ± 0.87
78.34 ± 0.79
90.37 ± 5.97
90.02 ± 1.77
81.86 ± 2.35
74.67 ± 4.98
84.23 ± 1.89
95.06 ± 0.30
ReAct
85.93 ± 0.83
88.29 ± 0.44
87.11 ± 0.61
92.81 ± 3.03
89.12 ± 3.19
89.38 ± 1.49
90.35 ± 0.78
90.42 ± 1.41
95.06 ± 0.30
Appendix
Table 13: Fine-grained results (AUROC ↑ ) on the CIFAR-10 benchmark.
Method
Near-OOD
Far-OOD
ID ACC
CIFAR-100
TIN
Average
MNIST
SVHN
Textures
Places365
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
53.08 ± 4.86
43.27 ± 3.00
48.17 ± 3.92
23.64 ± 5.81
25.82 ± 1.64
34.96 ± 4.64
42.47 ± 3.81
31.72 ± 1.84
95.06 ± 0.30
Energy
66.60 ± 4.46
56.08 ± 4.83
61.34 ± 4.63
24.99 ± 12.93
35.12 ± 6.11
51.82 ± 6.11
54.85 ± 6.52
41.69 ± 5.32
95.06 ± 0.30
DICE
73.71 ± 7.67
66.37 ± 7.68
70.04 ± 7.64
30.83 ± 10.54
36.61 ± 4.74
62.42 ± 4.79
77.19 ± 12.60
51.76 ± 4.42
95.06 ± 0.30
ReAct
67.40 ± 7.34
59.71 ± 7.31
63.56 ± 7.33
33.77 ± 18.00
50.23 ± 15.98
51.42 ± 11.42
44.20 ± 3.35
44.90 ± 8.37
95.06 ± 0.30
Appendix
Table 14: Fine-grained results (FPR95 ↓ ) on the CIFAR-10 benchmark.
Method
Near-OOD
Far-OOD
ID ACC
CIFAR-10
TIN
Average
MNIST
SVHN
Textures
Places365
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
78.47 ± 0.07
82.07 ± 0.17
80.27 ± 0.11
76.08 ± 1.86
78.42 ± 0.89
77.32 ± 0.71
79.22 ± 0.29
77.76 ± 0.44
77.25 ± 0.10
Energy
79.05 ± 0.11
82.76 ± 0.08
80.91 ± 0.08
79.18 ± 1.37
82.03 ± 1.74
78.35 ± 0.83
79.52 ± 0.23
79.77 ± 0.61
77.25 ± 0.10
DICE
78.04 ± 0.32
80.72 ± 0.30
79.38 ± 0.23
79.86 ± 1.89
84.22 ± 2.00
77.63 ± 0.34
78.33 ± 0.66
80.01 ± 0.18
77.25 ± 0.10
ReAct
78.65 ± 0.05
82.88 ± 0.08
80.77 ± 0.05
78.37 ± 1.59
83.01 ± 0.97
80.15 ± 0.46
80.03 ± 0.11
80.39 ± 0.49
77.25 ± 0.10
Appendix
Table 15: Fine-grained results (AUROC ↑ ) on the CIFAR-100 benchmark.
Method
Near-OOD
Far-OOD
ID ACC
CIFAR-10
TIN
Average
MNIST
SVHN
Textures
Places365
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
58.91 ± 0.93
50.70 ± 0.34
54.80 ± 0.33
57.23 ± 4.68
59.07 ± 2.53
61.88 ± 1.28
56.62 ± 0.87
58.70 ± 1.06
77.25 ± 0.10
Energy
59.21 ± 0.75
52.03 ± 0.50
55.62 ± 0.61
52.62 ± 3.83
53.62 ± 3.14
62.35 ± 2.06
57.75 ± 0.86
56.59 ± 1.38
77.25 ± 0.10
DICE
60.98 ± 1.10
54.93 ± 0.53
57.95 ± 0.53
51.79 ± 3.67
49.58 ± 3.32
64.23 ± 1.65
59.39 ± 1.25
56.25 ± 0.60
77.25 ± 0.10
ReAct
61.30 ± 0.43
51.47 ± 0.47
56.39 ± 0.34
56.04 ± 5.66
50.41 ± 2.02
55.04 ± 0.82
55.30 ± 0.41
54.20 ± 1.56
77.25 ± 0.10
Appendix
Table 16: Fine-grained results (FPR95 ↓ ) on the CIFAR-100 benchmark.
Method
Near-OOD
Far-OOD
ID ACC
SSB-hard
NINCO
Average
iNaturalist
Textures
OpenImage-O
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
80.38 ± 0.03
86.29 ± 0.11
83.34 ± 0.06
92.80 ± 0.25
88.36 ± 0.13
89.24 ± 0.02
90.13 ± 0.09
86.37 ± 0.08
Energy
79.83 ± 0.02
85.17 ± 0.11
82.50 ± 0.05
92.55 ± 0.50
90.79 ± 0.16
89.23 ± 0.26
90.86 ± 0.21
86.37 ± 0.08
DICE
79.06 ± 0.05
84.49 ± 0.24
81.78 ± 0.14
91.81 ± 0.79
91.53 ± 0.21
89.06 ± 0.34
90.80 ± 0.31
86.37 ± 0.08
ReAct
78.97 ± 1.33
84.76 ± 0.64
81.87 ± 0.98
93.65 ± 0.88
92.86 ± 0.47
90.40 ± 0.35
92.31 ± 0.56
86.37 ± 0.08
Appendix
Table 17: Fine-grained results (AUROC ↑ ) on the ImageNet-200 benchmark.
Method
Near-OOD
Far-OOD
ID ACC
SSB-hard
NINCO
Average
iNaturalist
Textures
OpenImage-O
Average
Post-hoc methods (vanilla training with cross-entropy)
MSP
66.00 ± 0.10
43.65 ± 0.75
54.82 ± 0.35
26.48 ± 0.73
44.58 ± 0.68
35.23 ± 0.18
35.43 ± 0.38
86.37 ± 0.08
Energy
69.77 ± 0.32
50.70 ± 0.89
60.24 ± 0.57
26.41 ± 2.29
41.43 ± 1.85
36.74 ± 1.14
34.86 ± 1.30
86.37 ± 0.08
DICE
70.84 ± 0.30
52.91 ± 1.20
61.88 ± 0.67
29.66 ± 2.62
40.96 ± 1.87
38.91 ± 1.16
36.51 ± 1.18
86.37 ± 0.08
ReAct
71.51 ± 1.92
53.47 ± 2.46
62.49 ± 2.19
22.97 ± 2.25
29.67 ± 1.35
32.86 ± 0.74
28.50 ± 0.95
86.37 ± 0.08
Appendix
Table 18: Fine-grained results (FPR95 ↓ ) on the ImageNet-200 benchmark.
Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection methods focus on the final output layer, they neglect the rich hierarchical information present in intermediate network layers. This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from these intermediate activations. We find that in-distribution (ID) and OOD data activate distinct sets of these sparse features. We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes. Our post-hoc detection method not only achieves state-of-the-art performance on standard OOD detection benchmarks, but yields interpretable insights into how distribution shift affects learned representations.
Ayush Karmacharya, Luke Luschwitz, Lucia Romero +2
In out-of-distribution (OOD) detection, fine-tuning with auxiliary outlier data often improves detection performance at the cost of classification accuracy. This trade-off stems from the loss of the original in-distribution (ID) distribution during fine-tuning. To establish a more practical and effective paradigm, we optimize three critical factors: model reminder, data sampling, and representation learning. We propose: (1) Self-Knowledge Distillation (SKD) to mitigate accuracy reduction; (2) Semi-hard Outlier Sampling (SOS) to improve detection efficiency with minimal data; and (3) Outlier-aware Supervised Contrastive Learning (OSCL) to promote ID-OOD separability. Optimizing these factors produces cumulative gains, boosting both OOD detection performance and classification accuracy. Our framework outperforms existing methods across diverse benchmarks, particularly in long-tailed scenarios, providing a robust baseline for real-world OOD detection.
Hyunjun Choi, JaeHo Chung, Hawook Jeong
ASRI, ECE., Seoul National University, South Korea · RideFlux Inc., South Korea
Out-of-distribution (OOD) detection is essential for deploying machine learning models in open-world and safety-critical scenarios, where test inputs may deviate from the training distribution and overconfident predictions on unknown samples can lead to unreliable decisions. Outlier Exposure (OE) has emerged as a promising OOD detection paradigm by introducing auxiliary outliers during training to enlarge the margin between in-distribution (ID) and OOD samples. Existing OE-based methods typically enlarge this margin by employing uniform labels to maximize the entropy of OOD samples over ID categories. However, we theoretically show that uniform labels inevitably disregard the relations between OOD samples and ID categories, termed the over-softening effect, leading to a suboptimal margin bound. Our theoretical analysis further reveals that explicitly exploiting such relations can instead yield improved OOD detection performance. Motivated by this insight, we propose Adaptive Confidence OE (AOE), a simple yet effective method that leverages temperature scaling to recalibrate outlier labels. Specifically, AOE generates adaptive soft targets from temperature-scaled model predictions for OOD samples, where the learnable temperature smooths the prediction distribution without fully erasing class-wise relational information. By supervising OOD samples with these adaptive soft targets, AOE preserves the semantic proximity between OOD samples and ID categories while encouraging the softened targets to approach a high-entropy distribution, thereby suppressing overconfident OOD predictions and enlarging the separation margin. Extensive experiments across diverse benchmarks demonstrate the effectiveness of AOE.
Fengqiang Wan, Qing-Yuan Jiang, Yang Yang
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China.