Mitigating the Likelihood Paradox in Flow-based OOD Detection: An Entropic Perspective

Donghwan Kim; Hyunsoo Yoon

cs.LGFeb 10, 2026

Mitigating the Likelihood Paradox in Flow-based OOD Detection: An Entropic Perspective

Authors: Donghwan Kim, Hyunsoo Yoon

Organizations: Department of Industrial Engineering University of Yonsei

Abstract

Deep generative models that can tractably compute input likelihoods, including normalizing flows, often assign unexpectedly high likelihoods to out-of-distribution (OOD) inputs. We mitigate this paradox from an entropic perspective with Semantic Proportional Entropy Manipulation (SPEM), a training-free procedure that applies stronger perturbations to inputs less similar to an in-distribution (ID) memory bank. We derive theoretical lower bounds showing that this entropy gap increases the expected log-likelihood difference in favor of the ID, and experiments on standard benchmarks show that this manipulation realigns the likelihood ordering with human intuition. Strikingly, this realignment persists even when the flow is given only visually meaningless noise instead of the original input, and we analyze this phenomenon theoretically to characterize the conditions under which it occurs.

Figures & tables

Panel of Figure 1 — Figure 1: AUROC changes with entropy manipulation intensity and log-likelihood assignments with and without perturbation. We increase $\sigma$ from 0.001 to 0.02 and add $\mathbf{z}\sim N(0,\sigma^{2}I_{d})$ perturbation to SVHN to create a noisy SVHN distribution, and perform OOD detection through likelihood with Glow trained on CIFAR-10 and CelebA. The histogram shows the change in log-likelihood assignment before and after perturbation when Glow is trained with CIFAR-10.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
Likelihood	0.0256	0.0277	0.0163	0.6346	0.8797	0.9967	0.9950	0.9988	0.5819	0.9786
Complexity	0.7943	0.7331	0.7998	0.6252	0.8811	0.9912	0.9890	0.9983	0.5740	0.3411
Typicality ( $\sqrt{d}$ )	0.9341	0.9225	0.9407	0.3778	0.2065	0.9902	0.9833	0.9985	0.5973	0.0062
Typicality (Entropy)	0.8866	0.8783	0.9489	0.3633	0.7703	0.9938	0.9893	0.9987	0.6082	0.9688
Likelihood Ratio	0.8512	0.7952	0.4458	0.7018	0.9718	0.9640	0.9418	0.9962	0.1826	0.0874

Table 1: OOD detection performance on real image dataset using ResFlow. The five dataset pairs on the left are widely recognized as cases where likelihood-based OOD detection fails, and the remaining pairs do not exhibit problematic behavior in the estimated likelihood. For each dataset pair, the values that ranked within the top two across all compared models (excluding

\lambda

) are highlighted in bold. Results for SPEM and SPEM-noise are reported as the mean AUROC over three independent random seeds.

$P$ (In)	SVHN
$Q$ (Out)	CIFAR-10	CIFAR-100	CelebA
SPEM	0.8159	0.8190	0.7846
$\lambda$	0.7591	0.7598	0.7603

Table 2: OOD detection performance of SPEM and sampled

\lambda

. Results are reported as the mean AUROC over three repeated samplings. We use ResFlow as the flow model and set

\alpha=0.1

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: AUROC for IDs composed of real images as a function of $\sigma$ when the OOD is $\mathcal{N}(0,\sigma^{2}I_{d})$ .

Figure 3: OOD detection performance according to $\alpha$ , which controls the intensity of entropy manipulation. The legend indicates the experimental dataset pairs, with the former indicating in-distribution and the latter indicating OOD. All experiments were set up identically to those in Table 1 .

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
SPEM	0.9915	0.9359	1.0000	0.9830	0.9206	0.9997	0.9993	1.0000	0.9999	1.0000
$k$ -NN	0.9712	0.8668	1.0000	0.9934	0.9450	0.9999	0.9989	1.0000	0.9999	0.9999

Appendix

Table 3: OOD detection performance (AUROC) comparison between SPEM and

k

-NN ( Sun et al., 2022 ) across all ten ID/OOD dataset pairs.

k

-NN is evaluated in the same pretrained ResNet-152 feature space as SPEM, serving as the most natural label-free embedding-based baseline for a fair feature-space comparison.

	brightness	canny	dotted	fog	glass	impulse	motion	rotate	scale	shear	shot	spatter	stripe	translate	zigzag
SPEM ( $\alpha=0.4$ )	0.9917	0.9903	0.9615	0.9999	0.9997	0.9999	0.9972	0.6179	0.7679	0.6982	0.9929	0.9909	0.9993	0.7031	0.9690
$\lambda$	0.9826	0.9896	0.9589	0.9997	0.9997	0.9999	0.9968	0.6201	0.7882	0.6977	0.9930	0.9904	0.9987	0.7067	0.9662

Appendix

Table 4: OOD detection performance on MNIST-C with MNIST as the ID dataset. We evaluate SPEM and

\lambda

-only across 15 corruption types and report AUROC for each case.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
ResNet-50	0.9839	0.9435	0.9999	0.9547	0.9643	0.9991	0.9984	0.9995	0.9997	1.0000
ResNet-101	0.9894	0.9536	1.0000	0.9691	0.8623	0.9996	0.9992	0.9999	0.9999	0.9999
ResNet-152	0.9915	0.9359	1.0000	0.9830	0.9206	0.9997	0.9993	1.0000	0.9999	1.0000

Appendix

Table 5: OOD detection performance using ResFlow according to changes in feature extractor.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
ResNet-50	0.9583	0.8712	0.9634	0.9498	0.9774	0.9965	0.9959	0.9975	0.9628	1.0000
ResNet-101	0.9722	0.8897	0.9705	0.9621	0.8887	0.9979	0.9977	0.9992	0.9676	0.9999
ResNet-152	0.9768	0.8655	0.9697	0.9782	0.9424	0.9985	0.9975	0.9993	0.9701	1.0000

Appendix

Table 6: OOD detection performance using Glow according to changes in feature extractor.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
w/ ReAct	0.9915	0.9359	1.0000	0.9830	0.9206	0.9997	0.9993	1.0000	0.9999	1.0000
w/o ReAct	0.9536	0.8471	0.9999	0.9839	0.9042	0.9994	0.9988	0.9999	0.9999	1.0000

Appendix

Table 7: Performance difference in OOD detection using ResFlow with and without the ReAct module.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
w/ ReAct	0.9768	0.8655	0.9697	0.9782	0.9424	0.9985	0.9975	0.9993	0.9701	1.0000
w/o ReAct	0.9116	0.7570	0.9806	0.9795	0.9284	0.9990	0.9983	0.9998	0.9886	1.0000

Appendix

Table 8: Performance difference in OOD detection using Glow with and without the ReAct module.

$P$ (In)	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST	SVHN	SVHN	SVHN	CelebA	MNIST
$Q$ (Out)	SVHN	SVHN	SVHN	CelebA	MNIST	CIFAR-10	CIFAR-100	CelebA	CIFAR-10	FashionMNIST
Likelihood	0.0807	0.0939	0.1292	0.5144	0.7359	0.9919	0.9906	0.9991	0.7408	0.9997
Complexity	0.8718	0.8319	0.9737	0.5566	0.8604	0.5270	0.6020	0.6163	0.7564	0.9712
Typicality ( $\sqrt{d}$ )	0.6536	0.6782	0.9998	0.6296	0.2630	0.9781	0.9746	0.9976	0.9249	0.9040
Typicality (Entropy)	0.4929	0.4762	0.7300	0.4283	0.5878	0.9894	0.9876	0.9990	0.7237	0.9997
Likelihood Ratio	0.8477	0.6020	0.8550	0.7334	0.9616	0.1078	0.1398	0.1548	0.5146	0.9875

Appendix

Table 9: OOD detection performance on real image dataset using Glow. The five dataset pairs on the left are widely recognized as cases where likelihood-based OOD detection fails, and the remaining pairs do not exhibit problematic behavior in the estimated likelihood. For each dataset pair, the values that ranked within the top two across all compared models are highlighted in bold.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 222 papers

The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spaces
May 21, 2026Philipp Bomatter, Jack Geary, Henry GoukDistribution Shift Detection OOD Detection
School of Informatics, University of Edinburgh
Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection
May 30, 2026Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen +1Speech Processing Distribution Shift Detection
Department of Electronic Systems Norwegian University of Science and Technology Trondheim, Norway · School of Future Technology South China University of Technology Guangzhou, China
How Language Models Process Out-of-Distribution Inputs: A Two-Pathway Framework
Apr 30, 2026Hamidreza SaghirLLM Interpretability Attention Mechanisms
Independent Researcher

May 21, 2026J/K move · Enter open · S save

The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spaces

Philipp Bomatter, Jack Geary, Henry Gouk

School of Informatics, University of Edinburgh

Distribution Shift Detection OOD Detection

Open paper AskPDF arXiv

Deep generative models offer a natural foundation for out-of-distribution (OOD) detection, yet prior work has shown that their assigned likelihoods are notoriously unreliable indicators for in- vs out-of-distribution data. In this paper, we address this problem by leveraging the diffeomorphic and mass-preserving properties of continuous normalising flows. Our analysis shows that OOD samples are mapped to noise samples that are highly atypical under the noise prior in ways not captured by the likelihood. Based on this observation, we propose a new method -- Signal in the Noise (SITN) -- for OOD detection on the single-sample level. SITN requires no access to OOD data, incurs minimal computational overhead, and provides strict control of false positive rates. Comprehensive evaluations through standard benchmarks and synthetic perturbations highlight the method's effectiveness and the absence of the complexity bias inherent to likelihood-based methods.

Abstract

OOD Detection

Explore similar work

The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spaces

Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection

How Language Models Process Out-of-Distribution Inputs: A Two-Pathway Framework