Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis

Youngjin Yoo; Han Liu; Bogdan Georgescu; Yanbo Zhang; Sasa Grbic; Michael Baumgartner; Thomas J. Re; Jyotipriya Das

cs.CVFeb 28, 2026

Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis

Authors: Youngjin Yoo, Han Liu, Bogdan Georgescu, Yanbo Zhang, Sasa Grbic, Michael Baumgartner, Thomas J. Re, Jyotipriya Das, +9 more

Organizations: Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA · Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany · Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA · Department of Radiology, Vancouver General Hospital, Vancouver, BC Canada · Department of Radiology, Northwell Health, New York, NY USA · Department of Surgery, UCHealth Memorial Hospital, Colorado Springs, CO USA · Department of Radiology, New York University, New York, NY USA

Abstract

Foundation models pre-trained on large-scale datasets demonstrate strong transfer learning capabilities; however, their adaptation to complex multi-label diagnostic tasks-such as comprehensive head CT finding detection-remains understudied. Standard parameter-efficient fine-tuning methods such as LoRA apply uniform adaptations across pathology types, which may limit performance for diverse medical findings. We propose a Mixture of Low-Rank Experts (MoLRE) framework that extends LoRA with multiple specialized low-rank adapters and unsupervised soft routing. This approach enables conditional feature adaptation with less than 0.5% additional parameters and without explicit pathology supervision. We present a comprehensive benchmark of MoLRE across six state-of-the-art medical imaging foundation models spanning 2D and 3D architectures, general-domain, medical-domain, and head CT-specific pretraining, and model sizes ranging from 7M to 431M parameters. Using over 70,000 non-contrast head CT scans with 75 annotated findings-including hemorrhage, infarction, trauma, mass lesions, structural abnormalities, and chronic changes-our experiments demonstrate consistent performance improvements across all models. Gains vary substantially: general-purpose and medical-domain models show the largest improvements (DINOv3-Base: +4.6%; MedGemma: +4.3%), whereas 3D CT-specialized or very large models show more modest gains (+0.2-1.3%). The combination of MoLRE and MedGemma achieves the highest average detection AUC of 0.917. These findings highlight the importance of systematic benchmarking on target clinical tasks, as pretraining domain, architecture, and model scale interact in non-obvious ways.

Figures & tables

Figure 1: MoLRE: Parameter-efficient specialization for foundation models. Illustrated for 2D models (e.g., DINOv3), MoLRE employs (1) multiple low-rank expert adapters for feature transformations, (2) an unsupervised soft router that learns to weight experts based on input features, and (3) attention-weighted pooling to aggregate slice-level features into volume-level representations. For 3D models (e.g., Pillar0-HeadCT), MoLRE is applied to spatially-pooled volumetric features, enabling conditional adaptation without explicit pathology supervision.

Category	Count	Example Findings
Hemorrhage	12	Basal ganglia hemorrhage, subdural hematoma, subarachnoid hemorrhage, epidural hematoma
Vascular	8	Acute infarct, chronic infarct, vessel occlusion, thrombosis
Trauma	6	Skull fractures, contusions, pneumocephalus, scalp hematoma
Mass/Lesion	9	Meningioma, cystic lesions, tumors, focal lesions
Structural	11	Hydrocephalus, herniation, midline shift, ventricular enlargement
Chronic/Other	29	Calcifications, atrophy, sinusitis, chronic changes

Table 1: Pathology categories for the 75 head CT findings.

General 2D Vision Models
Model	Domain	Total	Base AUC	+MoLRE	Trainable	MoLRE
		Params		AUC	Params	Params
DINOv3-Base	Natural	86.6M	0.856 $\pm$ 0.083	0.902 $\pm$ 0.054	0.59M	0.28M
DINOv3-Large	Natural	305.2M	0.910 $\pm$ 0.052	0.913 $\pm$ 0.051	1.18M	0.37M
Medical 2D Vision-Language Models
MedImageInsight	Medical	364.3M	0.863 $\pm$ 0.080	0.876 $\pm$ 0.081	1.42M	0.37M

Table 2: Multi-label head CT detection performance and model complexity. Mean AUC

\pm

std across 75 neurological findings and parameter counts for each foundation model. All 2D models use LoRA fine-tuning with attention-based slice aggregation. Pillar0-HeadCT is fully fine-tuned. DeepCNTD is excluded from MoLRE evaluation due to the absence of attention pooling.

Figure 2: Stratified multi-finding detection performance. Numbers of neurological findings achieving high-confidence performance (AUC $\geq 0.90$ ) and moderate performance ( $0.8\leq\text{AUC}<0.9$ ) for each foundation model, with and without MoLRE.

Figure 3: Per-finding detection performance with and without MoLRE. Radar plots comparing baseline models and their MoLRE-enhanced counterparts for left: DINOv3-Base vs. DINOv3-Base+MoLRE and right: MedGemma vs. MedGemma+MoLRE across 75 neurological findings.

Explore similar work

MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification
Jul 31, 2026Sebastian Doerrich, Daniel Würtinger, Francesco Di Salvo +2Medical Imaging Foundation Models Medical Image Classification
xAILab Bamberg, University of Bamberg, Bamberg, Germany
Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities
Jul 18, 2026Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim +2Multi-Source Domain Adaptation Medical Imaging
Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh. · Graduate School of Informatics, Osaka Metropolitan University, Osaka, Japan. · Dept. of Mechanical Engineering, Chittagong University of Engineering and Technology (CUET), Chattogram, Bangladesh. +2
CoM $^3$ eT: A foundation model for medical image analysis through federated, multidimensional context integration
Aug 17, 2026J. Raphael Schäfer, Kai Geissler, Till Nicke +27Medical Imaging Foundation Models Multi-Task Learning
Fraunhofer Institute for Digital Medicine MEVIS · Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany · Institute of Diagnostic and Interventional Radiology, Medizinische Hochschule Hannover, Hannover, Germany +11

Jul 31, 2026J/K move · Enter open · S save

MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification

Sebastian Doerrich, Daniel Würtinger, Francesco Di Salvo, Shyam Nandan Rai+1

xAILab Bamberg, University of Bamberg, Bamberg, Germany

Medical Imaging Foundation Models Medical Image Classification Mixture-of-Experts Models Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Adapting deep learning models to profound clinical heterogeneity typically relies on parameter-efficient fine-tuning (PEFT) to avoid the severe overfitting associated with full end-to-end network updates. Although PEFT successfully navigates limited data scenarios, it inherently forces the training of a separate, isolated adapter for every specific diagnostic task. Consolidating these isolated adapters into a single generalist network risks negative transfer, as optimization gradients from conflicting visual domains interfere. To address this, we propose MoPET, a mixture-of-experts (MoE) method that uses a learned sparse router to direct each input through a small subset of low-rank PEFT experts injected into a frozen foundation model, sharing capacity across datasets while limiting cross-domain gradient conflict. Through selected evaluations on the MedMNIST benchmark, we first establish that PEFT outperforms full network updates, improving average accuracy from 86.50% to 88.97%. We then show that a single MoPET model consolidates four heterogeneous datasets into one network, improving average accuracy over the best isolated PEFT adapters (93.46% versus 92.83%). Finally, we show that co-training with auxiliary datasets improves accuracy on data-constrained clinical targets, raising average target accuracy over the strongest isolated adapter from 81.58% to 83.58%. Our source code is publicly available at https://github.com/sdoerrich97/mopet.

Abstract

Explore similar work

MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification

Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities

CoM3^33eT: A foundation model for medical image analysis through federated, multidimensional context integration

CoM $^3$ eT: A foundation model for medical image analysis through federated, multidimensional context integration