A Shared-Backbone Approach for Multi-Task MedMNIST Classification

Stefan-Dorian Gavril; Andrei Arhire; Adrian Iftene

cs.CVSep 6, 2026

A Shared-Backbone Approach for Multi-Task MedMNIST Classification

Authors: Stefan-Dorian Gavril, Andrei Arhire, Adrian Iftene

Organizations: Faculty of Computer Science, Alexandru Ioan Cuza University of Iaşi, Romania

Abstract

Multi-task biomedical classification requires models to generalize across disparate modalities and class distributions. We study 11 heterogeneous MedMNIST datasets using the harmonic mean of per-task macro-F1. We evaluate three backbones with task-specific linear heads. We identify a resolution domain shift between the MedMNIST API and evaluation environment. Resolving this inconsistency and optimizing architecture-specific regularization substantially improved performance. Our best configuration, a ConvNeXt-Tiny backbone with label smoothing, achieved a leaderboard harmonic-mean macro-F1 of 0.73294 in the Tensor Reloaded: Multi-Task MedMNIST competition, ranking sixth at the close of the official competition phase. Our implementation is publicly available at: https://github.com/GavrilStefan-Dorian/A-Shared-Backbone-Approach-for-Multi-Task-MedMNIST-Classification

Figures & tables

Figure 1: Class distributions and imbalance ratios across the MedMNIST tasks.

Task	Modality	Shape	Cl.	Train	Val	Test	Imb.	Ent.	BF1
PathMNIST	Colon Pathology	28x28x3	9	89,996	10,004	7,180	1.63	0.994	0.028
DermaMNIST	Dermatoscope	28x28x3	7	7,007	1,003	2,005	58.66	0.581	0.115
OCTMNIST	Retinal OCT	28x28	4	97,477	10,832	1,000	5.94	0.836	0.160
PneumoniaMNIST	Chest X-Ray	28x28	2	4,708	524	624	2.88	0.823	0.426
RetinaMNIST	Fundus Camera	28x28x3	5	1,080	120	400	7.36	0.874	0.124
BreastMNIST	Breast Ultrasound	28x28	2	546	78	156	2.71	0.840	0.422

Table 1: Dataset characteristics. Cl. = Classes; Imb. = Imbalance Ratio; Ent. = Normalized Shannon Entropy; BF1 = Baseline F1.

Figure 2: Representative Samples from MedMNIST Datasets.

Figure 3: EfficientNet-B0 Training Setup.

Backbone	Pretrained	WS	LS	Params	Public HM-F1
ConvNeXt-Tiny	Yes	False	False	28.0M	0.72937
ConvNeXt-Tiny	Yes	False	True	28.0M	0.73294
ConvNeXt-Tiny	Yes	True	False	28.0M	0.71892
ConvNeXt-Tiny	Yes	True	True	28.0M	0.72246
ConvNeXt-Tiny	No	True	True	28.0M	0.64628
ResNet-18	Yes	False	False	11.2M	0.69219

Table 2: Impact of WS, LS, and pretraining on the public leaderboard harmonic-mean macro-F1 score.

Task	Train	Val	Test	EffNet Val	EffNet Test	ConvNeXt Val	ConvNeXt Test
PathMNIST	89,996	10,004	7,180	0.978	0.8976	0.983	0.9061
DermaMNIST	7,007	1,003	2,005	0.587	0.6235	0.636	0.6238
OCTMNIST	97,477	10,832	1,000	0.812	0.6592	0.840	0.7254
PneumoniaMNIST	4,708	524	624	0.956	0.9063	0.966	0.9219
RetinaMNIST	1,080	120	400	0.441	0.3577	0.433	0.3953
BreastMNIST	546	78	156	0.834	0.8332	0.892	0.8301

Table 3: Per-task macro F1 for the EfficientNet-B0 (pretrained, WS on, LS on) and ConvNeXt-Tiny (pretrained, WS off, LS on) configurations.

Task	Mismatched Pipeline	Matched Pipeline
PathMNIST	0.5416	0.9061
DermaMNIST	0.4584	0.6238
OCTMNIST	0.4675	0.7254
PneumoniaMNIST	0.7337	0.9219
RetinaMNIST	0.3403	0.3953
BreastMNIST	0.5744	0.8301

Table 4: Per-task test macro-F1 for ConvNeXt-Tiny using mismatched (64px train, 28px

\rightarrow

64px test) and matched (28px

\rightarrow

64px train and test) resolution pipelines.

Explore similar work

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization
Aug 10, 2026Dinh Tan Nguyen, Quang-Hien Kha, Le-Hoang Nguyen +8Detection Transformer Multi-Task Learning
University of Technology Sydney, Australia. · Saigon Precision Medicine Research Center, Vietnam. · College of Medicine, Taipei Medical University, Taiwan. +3
MedicalRec: Medical recommender system for image classification without retraining
May 23, 2026Roghayeh Taghavi, Aysa Hasanazde Bashkandi, Amir Ali Bengari +5Model Selection Healthcare
BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis
Jun 27, 2026Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed +2Medical Image Analysis Multi-Task Learning
Khulna University of Engineering & Technology, Bangladesh · University Clermont Auvergne, France

Aug 10, 2026J/K move · Enter open · S save

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization

Dinh Tan Nguyen, Quang-Hien Kha, Le-Hoang Nguyen, Minh-Toan Dinh+7

University of Technology Sydney, Australia. · Saigon Precision Medicine Research Center, Vietnam. · College of Medicine, Taipei Medical University, Taiwan.+3

Detection Transformer Multi-Task Learning Medical Image Classification Mammography

Open paper AskPDF arXiv

Joint exam-level prediction and candidate-region localization may improve the usefulness of AI support in mammography. We study this setting using a multi-task DETR framework, where shared representations support both image-level malignancy prediction and lesion localization, and evaluate its performance on OPTIMAM and a biopsy-confirmed SGM1k cohort. Across both datasets, modern backbones consistently outperformed older ResNet-style features, with ConvNeXtV2 and DINOv3 giving the strongest overall results, whereas MambaVision was less competitive. On OPTIMAM, ConvNeXtV2 achieved the best overall performance, reaching 97.96% AUC, 99.89% sensitivity, 25.08% [email protected], and 74.38% [email protected]. On SGM1k, DINOv3 gave the strongest overall results, with 90.97% AUC, 86.28% sensitivity, 82.00% specificity, 27.04% [email protected], and 77.32% [email protected]. These findings suggest that backbone quality is a critical factor in effective multi-task mammography, with ConvNeXtV2 emerging as a particularly strong and well-matched CNN backbone for mammography in this framework.

Abstract

Explore similar work

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization

MedicalRec: Medical recommender system for image classification without retraining

BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis