Rank-1 Weight-Corrected Trainable Weight Averaging for Out-of-Distribution Detection

Akito Shinohara; Hiroaki Aizawa

cs.CVOct 24, 2026

Rank-1 Weight-Corrected Trainable Weight Averaging for Out-of-Distribution Detection

Authors: Akito Shinohara, Hiroaki Aizawa

Organizations: Graduate School of Advanced Science and Engineering, Hiroshima University, Japan

Abstract

Out-of-distribution (OOD) detection identifies inputs outside the training distribution. Rank-based analyses such as RankFeat and RankWeight suggest that OOD overconfidence is closely related to dominant low-rank components in feature and weight spaces. Since weight averaging operates directly in parameter space and improves model generalization, we consider it for OOD detection from a dominant-component perspective. Trainable Weight Averaging (TWA) assigns a single coefficient to each checkpoint, limiting independent control of dominant weight components. We extend TWA with leading rank-one weight components extracted by Singular Value Decomposition as correction directions, whose contributions are learned through the in-distribution classification objective. On ResNet50, our method improves over fine-tuning and standard TWA across the evaluated datasets with MSP and RankWeight, both with and without Outlier Exposure. Mahalanobis-based evaluation yields mixed results, indicating that the benefit depends on the OOD scoring protocol. Target-layer analyses with MSP and Energy favor correction near the classifier.

Figures & tables

Figure 1: Overview of the proposed dominant-component-aware extension of TWA. For each checkpoint, leading rank-one components are extracted from selected weight matrices and added as correction directions to the TWA subspace. The final model is optimized in the expanded subspace using the ID classification objective.

Protocol	Method	SVHN		Textures		Places
Protocol	Method	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$
MSP	FT	77.0	84.18	73.2	86.17	72.5	86.17
	TWA	78.7	80.11	70.2	88.51	72.1	85.34
	Ours	$\mathbf{79.0}_{+0.2}$	$\mathbf{79.02}_{-1.09}$	$\mathbf{74.3}_{+4.0}$	$\mathbf{85.59}_{-2.92}$	$\mathbf{75.2}_{+3.1}$	$\mathbf{81.61}_{-3.73}$
Energy	FT	84.8	79.72	76.9	84.57	75.0	84.49
	TWA	83.5	79.80	71.2	89.41	74.2	80.75

Table 1: OOD detection performance without Outlier Exposure (OE) using ResNet50. The final classification layer is used for the proposed method. RankWeight denotes the post-hoc RankWeight protocol. Throughout the tables, scores are reported as percentages, subscripts indicate differences from TWA in percentage points, and bold indicates the best result for each metric and experimental setting.

Method	Accuracy (%) $\uparrow$
FT	73.94
TWA	72.55
Ours	$\mathbf{75.04}_{+2.49}$

Table 2: CIFAR-100 ID accuracy without OE.

Range	Method	MSP		Energy
Range	Method	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$
(0,51)	TWA	71.1	85.60	79.4	89.30
(0,51)	Ours	$\mathbf{74.3}_{+3.2}$	$\mathbf{83.18}_{-2.42}$	$\mathbf{84.4}_{+5.1}$	$\mathbf{83.20}_{-6.10}$
(0,101)	TWA	74.1	83.50	83.5	79.80
(0,101)	Ours	$\mathbf{78.3}_{+4.2}$	$\mathbf{81.28}_{-2.22}$	$\mathbf{85.9}_{+2.4}$	$\mathbf{76.44}_{-3.36}$
(0,151)	TWA	70.8	84.83	79.9	83.93

Table 3: Effect of checkpoint averaging range on OOD detection without OE using ResNet50.

Ours: target layer for dominant-component correction
Method / Target	MSP		Energy
Method / Target	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$
FT	77.0	84.18	84.8	79.72
TWA	78.7	80.11	83.5	79.80
all	$75.6_{-3.1}$	$84.21_{+4.10}$	$84.2_{+0.7}$	$78.49_{-1.31}$
conv	$75.9_{-2.8}$	$84.53_{+4.42}$	$80.5_{-3.0}$	$86.14_{+6.34}$

Table 4: Target-layer analysis for dominant-component correction without OE on CIFAR-100/SVHN using ResNet50.

Dataset	Backbone	Method	MSP		Energy
Dataset	Backbone	Method	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$
SVHN	VGG16BN	FT	67.5	87.63	73.4	82.51
		TWA	65.7	90.25	72.8	87.46
		Ours	$66.8_{+1.1}$	$\mathbf{84.88}_{-5.37}$	$\mathbf{76.0}_{+3.2}$	$\mathbf{76.88}_{-10.58}$
	ResNet50	FT	77.0	84.18	84.8	79.72
		TWA	78.7	80.11	83.5	79.80

Table 5: Effect of backbone architectures on OOD detection without OE using CIFAR-100 as ID.

Method	Accuracy (%) $\uparrow$
FT	73.94
TWA	69.94
Ours	$72.69_{+2.75}$

Table 6: CIFAR-100 ID accuracy with OE.

Protocol	Method	SVHN		Textures		Places
Protocol	Method	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$	AUROC (%) $\uparrow$	FPR95 (%) $\downarrow$
MSP	FT	79.4	80.22	74.4	83.99	75.3	82.10
	TWA	80.5	77.27	72.1	86.91	71.1	85.08
	Ours	$\mathbf{82.9}_{+2.5}$	$\mathbf{70.34}_{-6.93}$	$\mathbf{76.6}_{+4.5}$	$\mathbf{82.39}_{-4.52}$	$\mathbf{77.3}_{+6.2}$	$\mathbf{78.62}_{-6.46}$
Energy	FT	84.2	75.80	77.4	81.33	79.9	75.48
	TWA	85.4	75.00	74.3	84.68	75.2	77.06

Table 7: OOD detection performance with Outlier Exposure (OE) using ResNet50. The final classification layer is used for the proposed method. RankWeight denotes the post-hoc RankWeight protocol.

OE	Method	SVHN		Textures		Places
OE	Method	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
w/o OE	FT	71.2	77.20	86.6	43.94	51.5	95.95
	TWA	84.1	55.20	86.6	42.23	52.9	95.50
	Ours	$66.3_{-17.8}$	$86.29_{+31.09}$	$\mathbf{88.2}_{+1.6}$	$\mathbf{41.22}_{-1.01}$	$50.8_{-2.1}$	$95.56_{+0.06}$
w/ OE	FT	71.2	79.69	88.6	38.30	55.0	94.38
	TWA	84.1	56.64	85.6	42.71	58.9	93.66

Table 8: Feature-based OOD detection using Mahalanobis scoring on ResNet50, with CIFAR-100 as ID. Features are extracted from the input to the final classification layer.

Panel of Figure 2 — Figure 2: Loss landscape visualization without Outlier Exposure (OE). The proposed method expands the TWA search subspace while retaining the flat-solution behavior of weight averaging.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 225 papers

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring
Mar 18, 2026Jin Mo Yang, Hyung-Sin Kim, Saewoong BahkOOD Detection
Seoul National University, Seoul, South Korea
AOE: Exhaustive Out-of-Distribution Detection via Recalibrating Outlier Labels
May 27, 2026Fengqiang Wan, Qing-Yuan Jiang, Yang YangOOD Detection
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China.
MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion
Jun 15, 2026Rahim Hossain, Md Tawheedul Islam Bhuian, Md Farhan Shadiq +1Diffusion-Based OOD Detection OOD Detection
School of Computing State University of New York at Binghamton

Mar 18, 2026J/K move · Enter open · S save

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

Jin Mo Yang, Hyung-Sin Kim, Saewoong Bahk

Seoul National University, Seoul, South Korea

OOD Detection

Open paper AskPDF arXiv

Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitation: logit-based methods see only the classifier's confidence signal, while feature-based methods attempt to measure membership in the training distribution but do so in the full feature space where confidence and membership are entangled, inheriting architecture-sensitive failure modes. We observe that penultimate features naturally decompose into two orthogonal subspaces: a classifier-aligned component encoding confidence, and a residual the classifier discards. We discover that this residual carries a class-specific directional signature for in-distribution data -- a membership signal invisible to logit-based methods and entangled with noise in feature-based methods. We propose CORE (COnfidence + REsidual), which disentangles the two signals by scoring each subspace independently and combines them via normalized summation. Because the two signals are orthogonal by construction, their failure modes are approximately independent, producing robust detection where either view alone is unreliable. CORE achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations, ranking first in three of five settings and achieving the highest grand average AUROC with negligible computational overhead.

Abstract

OOD Detection

Explore similar work

CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

AOE: Exhaustive Out-of-Distribution Detection via Recalibrating Outlier Labels

MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusion