Extending ZACH-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low-Data Regimes

Athanasios Angelakis; Marta Gomez-Barrero

cs.CVApr 7, 2026

Extending ZACH-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low-Data Regimes

Authors: Athanasios Angelakis, Marta Gomez-Barrero

Organizations: BioML Lab, RI CODE, UniBw, Munich, Germany · AUMC, Amsterdam, Netherlands

Abstract

The recently introduced ZACH-ViT (Zero-token Adaptive Compact Hierarchical Vision Transformer) formalized a compact permutation-invariant Vision Transformer for medical imaging and argued that architectural alignment with spatial structure can matter more than universal benchmark dominance. Its design was motivated by the observation that positional embeddings and a dedicated class token encode fixed spatial assumptions that may be suboptimal when spatial organization is weakly informative, locally distributed, or variable across biomedical images. The foundational study established a regime-dependent clean performance profile across MedMNIST, but did not examine robustness in detail. In this work, we present the first robustness-focused extension of ZACH-ViT by evaluating its behavior under common image corruptions and adversarial perturbations in the same low-data setting. We compare ZACH-ViT with three scratch-trained compact baselines, ABMIL, Minimal-ViT, and TransMIL, on seven MedMNIST datasets using 50 samples per class, fixed hyperparameters, and five random seeds. Across the benchmark, ZACH-ViT achieves the best overall mean rank on clean data (1.57) and under common corruptions (1.57), indicating a favorable balance between baseline predictive performance and robustness to realistic image degradation. Under adversarial stress, all models deteriorate substantially; nevertheless, ZACH-ViT remains competitive, ranking first under FGSM (2.00) and second under PGD (2.29), where ABMIL performs best overall. These results extend the original ZACH-ViT narrative: the advantages of compact permutation-invariant transformers are not limited to clean evaluation, but can persist under realistic perturbation stress in low-data medical imaging, while adversarial robustness remains an open challenge for all evaluated models.

Figures & tables

Figure 1: Clean performance and corruption-averaged robustness across the seven MedMNIST datasets. The clean performance pattern reported in the foundational ZACH-ViT study is largely preserved, and the corruption analysis further shows that ZACH-ViT provides the most favorable overall balance between baseline predictive performance and robustness to realistic image degradation.

Figure 2: Adversarial stress testing under FGSM and PGD. All models degrade substantially relative to the clean setting. ZACH-ViT remains competitive and is the best-ranked transformer-based baseline on average under PGD, but no architecture is consistently best across all datasets.

Figure 3: Representative corruption-specific severity plots from the benchmark outputs. The selected panels illustrate that the favorable corruption profile of ZACH-ViT is visible across multiple corruption families, including Gaussian noise, Gaussian blur, brightness-contrast adjustment, JPEG compression, and cutout.

Figure 4: Dataset-agnostic summary of robustness behavior. Left: mean rank across clean, corruption, FGSM, and PGD regimes (lower is better). Right: average retention relative to each model’s own clean performance. ZACH-ViT is the most balanced model overall, while ABMIL shows the strongest proportional retention under adversarial stress.

	Clean				Common corruptions
Dataset	ABMIL	Minimal-ViT	TransMIL	ZACH-ViT	ABMIL	Minimal-ViT	TransMIL	ZACH-ViT
BloodMNIST	0.232 $\pm$ 0.053	0.502 $\pm$ 0.079	0.634 $\pm$ 0.042	0.597 $\pm$ 0.025	0.214 $\pm$ 0.033	0.465 $\pm$ 0.073	0.596 $\pm$ 0.040	0.559 $\pm$ 0.023
PathMNIST	0.257 $\pm$ 0.028	0.432 $\pm$ 0.066	0.523 $\pm$ 0.067	0.461 $\pm$ 0.054	0.253 $\pm$ 0.031	0.398 $\pm$ 0.049	0.501 $\pm$ 0.097	0.443 $\pm$ 0.054
BreastMNIST	0.706 $\pm$ 0.071	0.611 $\pm$ 0.015	0.633 $\pm$ 0.016	0.642 $\pm$ 0.010	0.699 $\pm$ 0.064	0.611 $\pm$ 0.014	0.630 $\pm$ 0.011	0.639 $\pm$ 0.006
PneumoniaMNIST	0.804 $\pm$ 0.019	0.681 $\pm$ 0.031	0.808 $\pm$ 0.014	0.813 $\pm$ 0.012	0.753 $\pm$ 0.019	0.681 $\pm$ 0.029	0.774 $\pm$ 0.012	0.775 $\pm$ 0.010
DermaMNIST	0.202 $\pm$ 0.028	0.211 $\pm$ 0.029	0.298 $\pm$ 0.044	0.301 $\pm$ 0.035	0.188 $\pm$ 0.023	0.188 $\pm$ 0.025	0.259 $\pm$ 0.042	0.272 $\pm$ 0.035

Table 1: Clean and corruption-averaged performance across the seven MedMNIST datasets. The reported metric is AUC for binary tasks and Macro-F1 for multiclass tasks. Values are mean

\pm

standard deviation over five seeds after averaging across corruption settings within each seed. We note that absolute differences between top-performing models are often small and should be interpreted with caution in the absence of formal hypothesis testing.

	FGSM				PGD
Dataset	ABMIL	Minimal-ViT	TransMIL	ZACH-ViT	ABMIL	Minimal-ViT	TransMIL	ZACH-ViT
BloodMNIST	0.110 $\pm$ 0.005	0.170 $\pm$ 0.015	0.134 $\pm$ 0.021	0.152 $\pm$ 0.018	0.102 $\pm$ 0.006	0.150 $\pm$ 0.015	0.108 $\pm$ 0.014	0.125 $\pm$ 0.020
PathMNIST	0.084 $\pm$ 0.010	0.162 $\pm$ 0.010	0.142 $\pm$ 0.026	0.160 $\pm$ 0.037	0.079 $\pm$ 0.008	0.139 $\pm$ 0.011	0.111 $\pm$ 0.021	0.127 $\pm$ 0.036
BreastMNIST	0.086 $\pm$ 0.064	0.094 $\pm$ 0.021	0.111 $\pm$ 0.056	0.103 $\pm$ 0.038	0.082 $\pm$ 0.061	0.071 $\pm$ 0.019	0.079 $\pm$ 0.050	0.071 $\pm$ 0.031
PneumoniaMNIST	0.316 $\pm$ 0.070	0.153 $\pm$ 0.055	0.301 $\pm$ 0.061	0.282 $\pm$ 0.015	0.309 $\pm$ 0.068	0.141 $\pm$ 0.051	0.242 $\pm$ 0.047	0.231 $\pm$ 0.011
DermaMNIST	0.074 $\pm$ 0.026	0.050 $\pm$ 0.021	0.089 $\pm$ 0.011	0.091 $\pm$ 0.015	0.073 $\pm$ 0.026	0.043 $\pm$ 0.018	0.070 $\pm$ 0.011	0.074 $\pm$ 0.015

Table 2: FGSM- and PGD-averaged adversarial performance across the seven MedMNIST datasets. Values are mean

\pm

standard deviation over five seeds after averaging across attack strengths within each seed. FGSM uses

\epsilon\in\{1/255,2/255,4/255,8/255\}

. PGD uses the same

\epsilon

values, 10 steps, and step size

\epsilon/4

. We note that absolute differences between top-performing models are often small and should be interpreted with caution in the absence of formal hypothesis testing.

Explore similar work

ZACH-ViT: Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging
Feb 20, 2026Athanasios AngelakisVision Transformer Efficient ViTs
BioML Lab, RI CODE, UniBw, Munich, Germany · EDS, Amsterdam UMC, Amsterdam, Netherlands
hZACH-ViT: Curved Latent Geometry for Compact Vision Transformers in Low-Data Medical Imaging
May 30, 2026Athanasios AngelakisGeometric Representation Learning Efficient ViTs
BioML Lab, Research Institute CODE, UniBw, Munich, Germany · Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, Netherlands
MedFM-Robust: Benchmarking Robustness of Medical Foundation Models
May 18, 2026Xiangxiang Cui, Tianjin Huang, Yifang Wang +2Multimodal Robustness Neural Network Robustness
Beijing Normal University, China · University of Exeter, United Kingdom · University College London, United Kingdom +2

Feb 20, 2026J/K move · Enter open · S save

ZACH-ViT: Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging

Athanasios Angelakis

BioML Lab, RI CODE, UniBw, Munich, Germany · EDS, Amsterdam UMC, Amsterdam, Netherlands

Vision Transformer Efficient ViTs Medical Imaging

Open paper AskPDF arXiv

Vision Transformers rely on positional embeddings and class tokens encoding fixed spatial priors. While effective for natural images, these priors may be suboptimal when spatial layout is weakly informative, a frequent condition in medical imaging. We introduce ZACH-ViT (Zero-token Adaptive Compact Hierarchical Vision Transformer), a compact Vision Transformer that removes positional embeddings and the [CLS] token, achieving permutation-invariant patch processing via global average pooling. Zero-token denotes removal of the dedicated aggregation token and positional encodings. Patch tokens remain unchanged. Adaptive residual projections preserve training stability under strict parameter constraints. We evaluate ZACH-ViT across seven MedMNIST datasets under a strict few-shot protocol (50 samples/class, fixed hyperparameters, five seeds). Results reveal regime-dependent behavior: ZACH-ViT (0.25M parameters, trained from scratch) achieves strongest advantage on BloodMNIST and remains competitive on PathMNIST, while relative advantage decreases on datasets with stronger anatomical priors (OCTMNIST, OrganAMNIST), consistent with our hypothesis. Component and pooling ablations show positional support becomes mildly beneficial as spatial structure increases, whereas reintroducing a [CLS] token is consistently unfavorable. These findings support that architectural alignment with data structure can outweigh universal benchmark dominance. Despite minimal size and no pretraining, ZACH-ViT achieves competitive performance under data-scarce conditions, relevant for compact medical imaging and low-resource settings. Code: https://github.com/Bluesman79/ZACH-ViT

Abstract

Explore similar work

ZACH-ViT: Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging

hZACH-ViT: Curved Latent Geometry for Compact Vision Transformers in Low-Data Medical Imaging

MedFM-Robust: Benchmarking Robustness of Medical Foundation Models