cs.CVNov 7, 2026

Anatomy-Activated Mixture-of-Experts for 3D Medical Vision-Language Pre-training

Authors: Szymon Płotka, Gizem Mert, Pedro R. A. S. Bassi, Wenxuan Li, Zongwei Zhou, Anna Marcinkiewicz, Wiktoria Romańczyk, Jarosław B. Ćwikła, 

Organizations: Jagiellonian University · Helmholtz Munich · Technical University of Munich · Munich Center for Machine Learning · 

Abstract

Medical Vision-Language Pre-training (VLP) leverages the semantic richness of radiology reports to learn 3D representations, yet current architectures remain fundamentally anatomy-agnostic. While clinical diagnostics rely on organ-specific context, standard encoders apply a uniform, dense parameterization to all volumetric patches, and existing Mixture-of-Experts (MoE) routers gate on isolated local features without explicit access to the organ composition of the scan or crop. We propose Anatomy-Activated Mixture-of-Experts (A2^{2}MoE), a framework that incorporates anatomical inductive biases within the model's computational path. A2^{2}MoE introduces two key components: (1) a Histogram Representation Router (HRR) that conditions expert selection on global visual context and a predicted anatomical composition histogram, encouraging anatomy-dependent expert weighting; and (2) an Organ-Query Multi-Scale Projector (OQMSP) that utilizes learnable queries to extract organ-level embeddings for fine-grained alignment without requiring manual annotations at inference. By routing at the patch level rather than the token level, A2^{2}MoE reduces routing complexity by orders of magnitude while satisfying load balancing by construction. Pre-trained on over 60,000 multi-phase CT scans, A2^{2}MoE achieves state-of-the-art performance across segmentation, classification, and detection benchmarks. Furthermore, we demonstrate that our anatomically grounded inductive biases support robust generalization to MRI and ultrasound, establishing a scalable foundation for medical representation learning. The code is available at https://github.com/gmum/A2MoE.

Figures & tables

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multimodal Hierarchical Mixture-of-Experts for 3D Medical Vision-Language Modeling

    Nov 5, 2026Gizem Mert, Szymon Płotka, Pedro R. A. S. Bassi +5Medical VLMs3D VLMs

  2. ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training

    May 30, 2026Rongsheng Wang, Fenghe Tang, Zihang Jiang +10Medical VLMsMedical Image Grounding

  3. Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography

    Jun 24, 2026Bowen Shi, Weiwei Cao, Ruifeng Yuan +5Contrastive Learning3D Medical Imaging