eess.IVMar 16, 2026

Vision-Language Model Based Multi-Expert Fusion for CT Image Classification

Authors: Jianfa Bai, Kejin Lu, Runtian Yuan, Qingqiu Li, Jilan Xu, Junlin Hou, Yuejie Zhang, Rui Feng

Organizations: College of Computer Science and Artificial Intelligence, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University · University of Oxford · The Hong Kong University of Science and Technology

Abstract

Robust detection of COVID-19 from chest CT remains challenging in multi-institutional settings due to substantial source shift, source imbalance, and hidden test-source identities. In this work, we propose a three-stage source-aware multi-expert framework for multi-source COVID-19 CT classification. First, we build a lung-aware 3D expert by combining original CT volumes and lung-extracted CT volumes for volumetric classification. Second, we develop two MedSigLIP-based experts: a slice-wise representation and probability learning module, and a Transformer-based inter-slice context modeling module for capturing cross-slice dependency. Third, we train a source classifier to predict the latent source identity of each test scan. By leveraging the predicted source information, we perform model fusion and voting based on different experts. On the validation set covering all four sources, the Stage 1 model achieves the best macro-F1 of 0.9711, ACC of 0.9712, and AUC of 0.9791. Stage2a and Stage2b achieve the best AUC scores of 0.9864 and 0.9854, respectively. Stage~3 source classifier reaches 0.9107 ACC and 0.9114 F1. These results demonstrate that source-aware expert modeling and hierarchical voting provide an effective solution for robust COVID-19 CT classification under heterogeneous multi-source conditions.

Figures & tables

Explore similar work

CardsList
  1. Robust Multi-Source Covid-19 Detection in CT Images

    Apr 2, 2026Asmita Yuki Pritha, Jason Xu, Daniel Ding +4Medical Image ClassificationDomain Generalization

  2. A Heterogeneous Ensemble for Multi-Center COVID-19 Classification from Chest CT Scans

    Mar 15, 2026Aadit Nilay, Bhavesh Thapar, Anant Agrawal +1Medical Image ClassificationDomain Generalization

  3. Halfway to 3D: Ensembling 2.5D and 3D Models for Robust COVID-19 CT Diagnosis

    Mar 16, 2026Tuan-Anh Yang, Bao V. Q. Bui, Chanh-Quang Vo-Van +1Medical Image Classification3D Medical Imaging