Quasi-Binarized Autoencoders: An Architecture-Independent Information Bottleneck for Medical Image Anomaly Detection

Shouhei Hanaoka; Takahiro Nakao; Atsushi Takamatsu; Takeharu Yoshikawa; Osamu Abe

cs.CVOct 7, 2026

Quasi-Binarized Autoencoders: An Architecture-Independent Information Bottleneck for Medical Image Anomaly Detection

Authors: Shouhei Hanaoka, Takahiro Nakao, Atsushi Takamatsu, Takeharu Yoshikawa, Osamu Abe

Organizations: Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan · Department of Radiology, Kanazawa University Hospital, 13-1 Takara-machi, Kanazawa, Ishikawa, 920-8641, Japan

Abstract

Unsupervised anomaly detection, which learns only from normal images, is a central task in medical image analysis and remains an open problem. Reconstruction-based methods pass an image through an encoder-decoder network trained on normal data and detect anomalies from the residual between the image and its reconstruction. This works only if the information passed from the encoder to the decoder is limited; otherwise the network learns an identity mapping and reconstructs anomalies too. This limit is usually imposed through architectural choices, tuned per dataset, that cannot be stated in bits. We introduce the quasi-binarizing (QB) layer, which squashes each latent element into [0, 1] and adds Laplace noise of scale 1/epsilon. Each element is then epsilon-locally differentially private, and the mutual information between an image and its reconstruction is bounded by a quantity that depends only on epsilon and the number of QB elements, whatever the encoder and decoder. Placing a QB layer on every encoder-decoder path, including all skip connections, we build QBAE, a seven-level attention U-Net with 32,768 QB elements. On the seven datasets of the MedIAnomaly benchmark, QBAE with one architecture and one configuration reaches a mean image-level AUROC of 0.828, the highest among methods that do not adapt to each dataset, and the best reported results on BraTS2021 (AUROC 0.911, pixel-level AP 0.838). The noise is kept at test time, so that every reconstruction satisfies the bound. Without input corruption, the bottleneck alone prevents identity collapse (mean AUROC 0.805 vs. 0.590). Code is available at https://github.com/hanaokalog/MedIAnomalyQB.

Figures & tables

Figure 1: Fig. 1. Overview of QBAE. (a) A seven-level U-Net in which each of the six skip connections and the fully connected bottom passes through a QB layer; the decoder receives only QB outputs (32,768 elements). (b) QB layer. (c), (d) DC-AE residual down- and upsampling. Details are given in Appendix B.

Setting	Per element	Total budget	Relative to a raw 8-bit $128^{2}$ image
$\varepsilon=0.3$	0.11 bits	$3.7\times 10^{3}$ bits	0.03
$\varepsilon=1$	0.19 bits	$6.2\times 10^{3}$ bits	0.05
$\varepsilon=3$	0.65 bits	$2.1\times 10^{4}$ bits	0.16
$\varepsilon=10$	1.98 bits	$6.5\times 10^{4}$ bits	0.50
$\varepsilon=30$	3.52 bits	$1.15\times 10^{5}$ bits	0.88
$\varepsilon=100$	5.25 bits	$1.72\times 10^{5}$ bits	1.31

Table 2

Level	Resolution	Channels per pixel	QB elements	Share of $N$
1	$128^{2}$	1	16,384	50.0%
2	$64^{2}$	2	8,192	25.0%
3	$32^{2}$	4	4,096	12.5%
4	$16^{2}$	8	2,048	6.3%
5	$8^{2}$	16	1,024	3.1%
6	$4^{2}$	32	512	1.6%

Table 3

Dataset	Source	Modality	Train (normal)	Test normal	Test abnormal	Pixel labels
RSNA	Shih et al. (2019) a	Chest X-ray	3,851	1,000	1,000	–
VinDr-CXR	Nguyen et al. (2022)	Chest X-ray	4,000	1,000	1,000	–
Brain Tumor	Hamada (2025); Saleh et al. (2020); Cheng et al. (2015) b	Brain MRI	1,000	600	600	–
LAG	Li et al. (2019)	Fundus photograph	1,500	811	811	–
ISIC2018	Codella et al. (2019)	Dermoscopy	6,705	909	603	–
Camelyon16	Ehteshami Bejnordi et al. (2017); Bao et al. (2024) c	Histopathology	5,088	1,120	1,113	–

Table 4

Dataset	QBAE AUROC	QBAE AP	AE-PL	DAE	MSDE AUROC (AP)	Best in MedIAnomaly
RSNA	0.881 $\pm$ 0.002	0.858 $\pm$ 0.005	0.875	0.861	0.918 (0.906)	0.911 (FAE-SSIM † )
VinDr-CXR	0.723 $\pm$ 0.009	0.730 $\pm$ 0.010	0.753	0.686	0.819 (0.797)	0.769 (AE-U)
Brain Tumor	0.959 $\pm$ 0.003	0.920 $\pm$ 0.006	0.957	0.832	0.981 (0.981)	0.973 (MSC † )
LAG	0.844 $\pm$ 0.010	0.773 $\pm$ 0.011	0.856	0.716	0.810 (0.831)	0.856 (AE-PL † )
ISIC2018	0.751 $\pm$ 0.009	0.656 $\pm$ 0.009	0.684	0.700	0.705 (0.638)	0.807 (AnatPaste)
Camelyon16	0.726 $\pm$ 0.005	0.647 $\pm$ 0.006	0.761	0.654	0.812 (0.820)	0.807 (AutoDDPM)

Table 5

Setting	Image AUROC	Pixel AP	Best Dice
Common setting ( $\varepsilon$ = 10, KL on)	0.911 $\pm$ 0.017	0.838 $\pm$ 0.016	0.773 $\pm$ 0.017
$\varepsilon$ = 30, KL on	0.939 $\pm$ 0.003	0.849 $\pm$ 0.008	0.786 $\pm$ 0.006
$\varepsilon$ = 30, KL off (best on BraTS)	0.942 $\pm$ 0.003	0.855 $\pm$ 0.002	0.792 $\pm$ 0.003
DAE (best in MedIAnomaly)	0.859	0.755	0.711

Table 6

Figure 2: Fig. 2. BraTS2021 examples from the common setting (run with the median Dice of five seeds). Left: four test images labeled normal; right: four abnormal images. Rows: input; reconstruction (one noise draw); anomaly map $A(x)$ (shown from a noise-free pass of the same network; one square-root color scale for all eight images, without clipping); input with the ground-truth tumor outline.

Figure 3: Fig. 3. Image-level AUROC as a function of

\varepsilon

(KL on). Blue: input corruption on; orange: off. “off” = no bottleneck (

\varepsilon

10^{8}

). Shaded bands:

\pm

1 SD over 5 seeds; rings: best

\varepsilon

per curve; dashed line: common setting

\varepsilon

= 10. The top axis of the mean panel gives the information budget N

\cdot

\varepsilon

Dataset	QB off	$\varepsilon$ = 10 (common)	Best $\varepsilon$	Best of all settings
RSNA	0.832	0.881	0.890 ( $\varepsilon$ = 3)	0.893 ( $\varepsilon$ = 3, corr. off, KL on)
VinDr-CXR	0.734	0.723	0.748 ( $\varepsilon$ = 30)	0.753 ( $\varepsilon$ = 100, corr. on, KL off)
Brain Tumor	0.961	0.959	0.965 ( $\varepsilon$ = 30)	0.968 ( $\varepsilon$ = off, corr. on, KL off)
LAG	0.802	0.844	0.844 ( $\varepsilon$ = 10)	0.860 ( $\varepsilon$ = 3, corr. on, KL off)
ISIC2018	0.738	0.751	0.751 ( $\varepsilon$ = 10)	0.752 ( $\varepsilon$ = 30, corr. on, KL off)
Camelyon16	0.593	0.726	0.794 ( $\varepsilon$ = 1)	0.803 ( $\varepsilon$ = 1, corr. off, KL on)

Table 9

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$\varepsilon$	$\min\{\varepsilon,\varepsilon^{2}\}$ [nats]	$C(\varepsilon)$ [nats]	Bound per element [bits]	Total [bits]
0.3	0.09	0.078	0.112	3,685
1	1	0.131	0.189	6,205
3	3	0.449	0.648	21,238
10	10	1.374	1.982	64,941
30	30	2.438	3.518	115,267
100	100	3.638	5.249	171,994

Appendix

Table 10

Figure 4: Fig. B.1. Detailed architecture of QBAE. (a) Overall network. (b) QB layer. (c), (d) DC-AE residual down- and upsampling. (e) Convolution block. (f) Decoder (up) block. (g) Cross-attention gate. (h) Attention and feed-forward blocks. (i) Fully connected layers around the bottom QB layer. (j) Training objective and anomaly score.

Level	Resolution	Width	QB elements (channels per pixel)	Decoder attention	Parameters (enc. / dec.)
1	$128^{2}$	16	16,384 (1)	–	0.003 M / 0.03 M
2	$64^{2}$	32	8,192 (2)	channel attention + FFN	0.015 M / 0.12 M
3	$32^{2}$	64	4,096 (4)	spatial attention + FFN	0.06 M / 0.46 M
4	$16^{2}$	128	2,048 (8)	spatial attention + FFN	0.23 M / 1.84 M
5	$8^{2}$	256	1,024 (16)	spatial attention + FFN	0.92 M / 7.35 M
6	$4^{2}$	512	512 (32)	spatial attention + FFN	3.67 M / 29.39 M

Appendix

Table 12

Explore similar work

Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI
Aug 17, 2026Chiara Tappermann, Steffen Renisch, Lars Ole Schwen +3Graph Anomaly Detection Detection Performance
Fraunhofer Institute for Digital Medicine MEVIS, Max-Von-Laue-Straße 2, Bremen, 28359, Bremen, Germany · University of Bremen, Bibliothekstraße 1, Bremen, 28359, Bremen, Germany · Institute of Diagnostic and Interventional Radiology, Hannover Medical School, Carl-Neuberg-Str. 1, Hannover, 28359, Lower Saxony, Germany
Reconstruction-Shift Discrimination via Mask-Guided Latent Diffusion for Medical Anomaly Detection
Aug 1, 2026Yibo Wan, Jinyu Cai, Yunhe Zhang +2Identity-Conditioned Latent Diffusion Models
School of Computing, National University of Singapore, Singapore · Institute of Data Science, National University of Singapore, Singapore · Department of Computer and Information Science, SKL-IOTSC, University of Macau, China +1
Compression-Driven Anomaly Detection in Brain MRI Using an Interpretable Quantum Autoencoder
Jun 25, 2026Santanu Ganguly, Xing Liang, Dimitrios MakrisFunctional Magnetic Resonance Imaging Autoencoder Architectures
Kingston University, Kingston upon Thames KT1 2EE, UK Quantum AI (QAI) Research Group School of Computer Science and Mathematics (CSM) Faculty of Engineering, Computing and the Environment (ECE)

Aug 17, 2026, cs.CVJ/K move · Enter open · S save

Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI

Chiara Tappermann, Steffen Renisch, Lars Ole Schwen, Hans Meine+2

Fraunhofer Institute for Digital Medicine MEVIS, Max-Von-Laue-Straße 2, Bremen, 28359, Bremen, Germany · University of Bremen, Bibliothekstraße 1, Bremen, 28359, Bremen, Germany · Institute of Diagnostic and Interventional Radiology, Hannover Medical School, Carl-Neuberg-Str. 1, Hannover, 28359, Lower Saxony, Germany

Graph Anomaly Detection Detection Performance Image Quality Assessment Data Quality

Open paper AskPDF arXiv

Corrupted, inconsistent, or anomalous data silently threatens the safety and reliability of medical AI. Despite growing regulatory recognition of dataset quality assurance (QA) for high-risk medical AI, scalable automated detection remains underdeveloped. We employ unsupervised anomaly detection (AD) and out-of-distribution (OOD) detection as an automated dataset QA mechanism for multi-center dynamic contrast-enhanced breast MRI. We build a controlled AD benchmark of 17 realistic QA-relevant anomaly types from six public datasets (protocol violations, processing errors, incorrect anatomical regions) and propose a taxonomy of radiological image anomalies based on human visual perception, enabling fine-grained analysis of AD failure modes. The benchmark includes near-, medium-far-, far-OOD samples, as well as in-distribution and external normal data. Four methods are evaluated: a projection-based method extended with a domain-specific feature extractor and a novel positional encoding, a reconstruction-based approach extended to full 3D volumes with an augmented training objective, and two unmodified hybrid OOD detection methods. Medium-far- and far-OOD samples are detected reliably, whereas near-OOD samples and external normal data from unseen institutions expose method-specific differences. The 3D reconstruction-based approach best balances detection performance (AUROC: 0.936) and generalization to unseen institutions. The projection-based method with positional encoding achieves the highest overall detection performance (AUROC: 0.954). Both hybrid methods exhibit critical failure modes, confirming that methods validated for one modality or anatomy may not generalize without domain-specific adaptation. Implants and mastectomies remain an open challenge for all methods. Our results establish a foundation and practical guidance on scalable unsupervised QA in medical AI pipelines.

Abstract

Explore similar work

Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI

Reconstruction-Shift Discrimination via Mask-Guided Latent Diffusion for Medical Anomaly Detection

Compression-Driven Anomaly Detection in Brain MRI Using an Interpretable Quantum Autoencoder