Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models

Yabin Zhang; Maya Varma; Yunhe Gao; Jean-Benoit Delbrouck; Jiaming Liu; Chong Wang; Curtis Langlotz

cs.CVMar 26, 2026

Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models

Authors: Yabin Zhang, Maya Varma, Yunhe Gao, Jean-Benoit Delbrouck, Jiaming Liu, Chong Wang, Curtis Langlotz

Organizations: Harbin Institute of Technology (Shenzhen) · Stanford University

Abstract

Out-of-distribution (OOD) detection aims to identify samples that deviate from in-distribution (ID). One popular pipeline addresses this by introducing negative labels distant from ID classes and detecting OOD based on their distance to these labels. However, such labels may present poor activation on OOD samples, failing to capture the OOD characteristics. To address this, we propose Test-time Activated Negative Labels (TANL) by dynamically evaluating activation levels across the corpus dataset and mining candidate labels with high activation responses during the testing process. Specifically, TANL identifies high-confidence test images online and accumulates their assignment probabilities over the corpus to construct a label activation metric. Such a metric leverages historical test samples to adaptively align with the test distribution, enabling the selection of distribution-adaptive activated negative labels. By further exploring the activation information within the current testing batch, we introduce a more fine-grained, batch-adaptive variant. To fully utilize label activation knowledge, we propose an activation-aware score function that emphasizes negative labels with stronger activations, boosting performance and enhancing its robustness to the label number. Our TANL is training-free, test-efficient, and grounded in theoretical justification. Experiments on diverse backbones and wide task settings validate its effectiveness. Notably, on the large-scale ImageNet benchmark, TANL significantly reduces the FPR95 from 17.5% to 9.8%. Codes are available at YBZh/OpenOOD-VLM.

Figures & tables

Panel of Figure 1 — Figure 1: Activation analyses with negative labels mined in [ 29 ] . (a) Negative labels on a specific OOD dataset exhibit a long-tailed activation score distribution. Some labels activate more strongly on the ID dataset than on OOD, potentially misleading OOD detection. (b) A small subset of negative labels strongly activates on OOD, enabling effective detection. Most labels respond similarly across ID and OOD, slightly harming detection, while some activate higher on ID, significantly degrading performance. The FPR95 results are obtained with negative labels of top activations via Eq. 4 . These analyses use ground truth labels from ImageNet (ID) and Places (OOD) datasets.

Figure 2: Overall framework of TANL. We dynamically explore activated negative labels from the corpus dataset in the testing process, where the activation information is measured based on the similarity between texts and the mined positive/negative images. The activation-aware score is illustrated as a simplified example of Eq. 15 with $M=2$ and $C=2$ .

OOD datasets
Methods	INaturalist		Sun		Places		Textures		Average
Methods	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
Methods requiring training (or fine-tuning)
ZOC [ 14 ]	86.09	87.30	81.20	81.51	83.39	73.06	76.46	98.90	81.79	85.19
CLIPN [ 69 ]	95.27	23.94	93.93	26.17	92.28	33.45	90.93	40.83	93.10	31.10
LoCoOp [ 51 ]	96.86	16.05	95.07	23.44	91.98	32.87	90.19	42.28	93.52	28.66

Table 1: OOD detection results with ImageNet-1k, where a VITB/16 CLIP encoder is adopted.

Methods	FPR95 $\downarrow$		$\mathrm{AUROC}\uparrow$		ACC $\uparrow$
Methods	Near-OOD	Far-OOD	Near-OOD	Far-OOD	ID
Methods requiring training (or fine-tuning)
AugMix [ 24 ] + ReAct [ 61 ]	–	–	79.94	93.70	77.63
SCALE [ 74 ]	–	–	81.36	96.53	76.18
AugMix [ 24 ] + ASH [ 10 ]	–	–	82.16	96.05	77.63
LAPT [ 83 ]	58.94	24.86	82.63	94.26	67.86

Table 2: OOD detection results under OpenOOD setting, where ImageNet-1k is adopted as ID dataset. Full results are available in Tab. A7 .

Methods requiring training (or fine-tuning)
Methods	FPR95 $\downarrow$		$\mathrm{AUROC}\uparrow$
Methods	Near-OOD	Far-OOD	Near-OOD	Far-OOD
AugMix [ 24 ] + SHE [ 81 ]	84.45	60.26	69.66	83.06
LSA [ 46 ]	70.56	48.06	78.22	86.85
ISH + SCALE [ 74 ]	–	–	68.04	89.46
LAPT [ 83 ]	71.18	33.07	74.77	92.14

Table 3: Full-spectrum OOD detection results under the OpenOOD setting, where ImageNet-1k, ImageNet-C, ImageNet-R, ImageNet-V2 are used as ID datasets. Full results are shown in Tab. A8 .

Panel of Figure 3 — Figure 3: Analyses on (a) number $M$ of selected negative labels, (b) selection criterion of negative labels, (c) $\alpha$ values, and (d) batch size under OpenOOD setting.

Figure 4: Visualization of ranked corpus dataset, where candidate labels with higher activation scores are utilized.

Dis-adapt	Batch-adapt	AAScore	Near-OOD	Far-OOD
NegLabel Baseline			69.45	23.73
✓			61.57	17.52
	✓		60.85	17.25
✓		✓	60.59	17.27
	✓	✓	60.06	17.21

Table 4: Ablation analyses, where results (FPR95

\downarrow

) are reported with ImageNet ID dataset under the OpenOOD setup. “Dis-adapt”, “Batch-adapt”, and “AAScore” represent the distribution-adaptive activated score in Eq. 8 , batch-adaptive variant in Eq. 11 , and activation-aware score in Eq. 15 , respectively.

Methods	Backbones	Training	FPS $\uparrow$	Param.	FPR95 $\downarrow$
ZOC [ 14 ]	VITB/16	$>$ 24h	287	336M	85.19
CLIPN [ 69 ]		$>$ 24h	605	37.8M	31.10
LoCoOp [ 51 ]		9h	625	8K	28.66
MCM [ 48 ]		–	625	–	43.93
NegLabel [ 29 ]		–	592	–	25.40
AdaNeg [ 82 ]		–	476	–	18.92

Table 5: Time complexity analyses. ‘Training’ measures the training time, and ‘Param.’ presents the number of learnable parameters. ‘FPS’ reflects the inference speed with a batch size of 256.

OOD datasets
Methods	INaturalist		Sun		Places		Textures		Average
Methods	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
Methods requiring training (or fine-tuning)
MSP [ 21 ]	87.44	58.36	79.73	73.72	79.67	74.41	79.69	71.93	81.63	69.61
ODIN [ 43 ]	94.65	30.22	87.17	54.04	85.54	55.06	87.85	51.67	88.80	47.75
Energy [ 44 ]	95.33	26.12	92.66	35.97	91.41	39.87	86.76	57.61	91.54	39.89

Table A6: Complete OOD detection results with ImageNet-1k, where a VITB/16 CLIP encoder is adopted.

Settings	OOD Datasets	FPR95 $\downarrow$	AUROC $\uparrow$
Near-OOD	SSB-hard [ 67 ]	62.51	83.96
	NINCO [ 3 ]	57.61	85.10
	Mean	60.06	84.53
Far-OOD	iNaturalist [ 65 ]	0.47	99.83
	Textures [ 7 ]	11.22	97.53
	OpenImage-O [ 68 ]	39.95	91.92

Table A7 : Detailed OOD detection results on the OpenOOD benchmark, where ImageNet-1k is adopted as ID dataset.

Settings	OOD Datasets	FPR95 $\downarrow$	AUROC $\uparrow$
Near-OOD	SSB-hard [ 67 ]	70.50	79.29
	NINCO [ 3 ]	66.92	78.51
	Mean	68.71	78.90
Far-OOD	iNaturalist [ 65 ]	1.70	99.58
	Textures [ 7 ]	19.68	94.81
	OpenImage-O [ 68 ]	46.07	88.66

Table A8 : Detailed full-spectrum OOD detection results on the OpenOOD benchmark, where ImageNet-1k, ImageNet-C, ImageNet-R, ImageNet-V2 are used as ID datasets.

Table A9 : OOD detection results with CIFAR10/100 on the OpenOOD benchmark. Full results are provided in Tables A10 .

Table A10 : Detailed OOD detection results on the OpenOOD benchmark.

	OOD datasets
Methods	CT-SCAN		X-Ray-Bone		Average
Methods	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
NegLabel [ 29 ]	63.53	100.0	99.68	0.56	81.61	50.28
AdaNeg [ 82 ]	93.48	100.0	99.99	0.11	96.74	50.06
TANL (Ours)	93.94	39.06	100.0	0.0	96.97	19.53

Table A11: OOD detection results with medical images following the OpenOOD setting.

Panel of Figure 16 — Figure A5 : Analyses on (a) the queue length $L$ , (b) threshold $\gamma$ , and (c) gap value $g$ in Eq. 9 under the OpenOOD setup.

OOD datasets
Methods	INaturalist		Sun		Places		Textures		Average
Methods	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
ResNet50
NegLabel [ 29 ]	99.24	2.88	94.54	26.51	89.72	42.60	88.40	50.80	92.97	30.70
TANL (Ours)	99.73	0.99	99.04	4.00	95.97	23.76	97.01	12.71	97.88	10.37
VITB/32

Table A12: OOD detection results of TANL with different VLMs backbones, where ImageNet-1K is used as the ID dataset.

Corpus Datasets	FPR95 $\downarrow$		$\mathrm{AUROC}\uparrow$
Corpus Datasets	Near-OOD	Far-OOD	Near-OOD	Far-OOD
Vanilla WordNet	60.52	17.34	83.24	95.92
Part-of-speech-tagging	60.36	17.24	83.74	96.15
WordNet-subset Filtered by NegLabel	60.06	17.21	84.53	96.43

Table A13: OOD detection results with different corpus datasets on the OpenOOD benchmark, where ImageNet-1k is adopted as ID dataset.

Panel of Figure 19 — Figure A6 : Analyses on (a) size of corpus dataset and (b) activation metric under the OpenOOD setup.

Methods	FPR95 $\downarrow$
Methods	Near-OOD	Far-OOD
W/o mutual enhancement	63.00	19.04
With mutual enhancement (Ours)	60.06	17.21

Table A14 : Analyses on the mutual enhancement between the estimation of activation information and the activation-aware OOD score function.

OOD datasets
Methods	INaturalist		Sun		Places		Textures		Average
Methods	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
NegLabel [ 29 ]	99.49	1.91	95.49	20.53	91.64	35.59	90.22	43.56	94.21	25.40
TANL under Temporal Shifts
I-S-P-T	99.84	0.45	98.77	4.75	95.93	22.16	96.67	14.91	97.80	10.56
S-P-T-I	99.83	0.46	99.04	3.65	95.92	21.37	96.70	15.09	97.87	10.14

Table A15: OOD detection results under the temporal shifts, where ImageNet-1k ID dataset and a VITB/16 CLIP encoder are adopted.

Order of Test Samples	INaturalist	SUN	Places	Textures	Average
NegLabel (Baseline)	1.91	20.53	35.59	43.56	25.40
ID First	2.97	8.35	29.63	24.91	16.47
OOD First	2.74	9.06	36.00	19.14	16.73
Random Shuffled	0.42	3.53	21.90	13.38	9.81

Table A16 : FPR95 (

\downarrow

) with different orders of ID and OOD samples.

Panel of Figure 23 — Figure A7 : Analyses on (a) error rates in the first batch and (b) number of test samples with ImageNet (ID) and SSB-hard (OOD).

OOD Score	INaturalist	SUN	Places	Textures	Average
NegLabel (Baseline)	1.91	20.53	35.59	43.56	25.40
$S_{nl}(\mathbf{v})$	1.30	6.67	25.20	18.37	12.91
$S_{aa}^{ew1}(\mathbf{v})$	99.85	100	100	100	99.96
$S_{aa}^{ew2}(\mathbf{v})$	0.62	4.08	24.06	15.04	10.95
$S_{aa}(\mathbf{v})$	0.42	3.53	21.90	13.38	9.81

Table A17 : FPR95 (

\downarrow

) with activation-aware score variant of explicit weighting.

Panel of Figure 25 — Figure A8 : Visualization of the ranked corpus dataset with (a) far-ood setting, (b) near-ood setting, and (c) different ranking criteria.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 222 papers

TINS: Test-time ID-prototype-separated Negative Semantics Learning for OOD Detection
May 11, 2026Yifeng Yang, Jubo Feng, Jing Xu +3Test-Time Adaptation OOD Detection
Shanghai Jiao Tong University · University of Electronic Science and Technology of China · Shanghai Artificial Intelligence Laboratory +1
TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models
Apr 17, 2026Jinlun Ye, Jiang Liao, Runhe Lai +4Prompt Learning Test-Time Adaptation
Sun Yat-sen University · Peng Cheng Laboratory · Key Laboratory of Machine Intelligence and Advanced Computing, MOE +2
NegAS: Negative Label Guided Attention and Scoring for Out-of-Distribution Object Detection with Vision-Language Models
Jun 21, 2026Yingjie Zhang, Shuai Li, Peng WangVLM Robustness Object Detection
Northwestern Polytechnical University, Xi’An, China

May 11, 2026J/K move · Enter open · S save

TINS: Test-time ID-prototype-separated Negative Semantics Learning for OOD Detection

Yifeng Yang, Jubo Feng, Jing Xu, Xinbing Wang+2

Shanghai Jiao Tong University · University of Electronic Science and Technology of China · Shanghai Artificial Intelligence Laboratory+1

Test-Time Adaptation OOD Detection

Open paper AskPDF arXiv

Vision-language models enable OOD detection by comparing image alignment with ID labels and negative semantics. Existing negative-label-based methods mainly rely on static negative labels constructed before inference, limiting their ability to cover diverse and evolving OOD concepts. Although test-time expansion provides a natural solution, naively learning negative semantics from potential OOD samples may introduce hard ID contamination. To address this issue, we propose a Test-time ID-prototype-separated Negative Semantics learning method, termed TINS. TINS learns sample-specific negative text embeddings via image-to-text modality inversion and introduces ID-prototype-separated regularization to keep them separated from ID semantics. To further stabilize negative semantics expansion, TINS employs group-wise aggregation scoring and a buffer update strategy. Extensive experiments across Four-OOD, OpenOOD, Temporal-shift, and Various ID settings show consistent improvements over strong baselines. Notably, on the Four-OOD benchmark with ImageNet-1K as ID, TINS reduces the average FPR95 from 14.04% to 6.72%. Our code is available at https://github.com/zxk1212/tins.

Abstract

OOD Detection

Explore similar work

TINS: Test-time ID-prototype-separated Negative Semantics Learning for OOD Detection

TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models

NegAS: Negative Label Guided Attention and Scoring for Out-of-Distribution Object Detection with Vision-Language Models