Towards Whole-Study Screening for Congenital Heart Disease in Fetal Ultrasound Using Multiple Instance Learning
Authors: Mohamed Azzam, Ruobing Liu, Esther C. Ugwueke, Ziyang Xu, Shibiao Wan, Alex Foy, Abraham Zabih, Jason Christensen, Neil Hamill, Ling Li, Jieqiong Wang
Organizations: Department of Neurological Sciences, University of Nebraska Medical Center, Omaha, NE 68198 USA · Department of Genetics, Cell Biology and Anatomy, University of Nebraska Medical Center, Omaha, NE 68198 USA · Department of Pediatrics, University of Nebraska Medical Center, Omaha, NE 68198 USA · Department of Obstetrics and Gynecology, University of Nebraska Medical Center, Omaha, NE 68198 USA
Congenital heart disease (CHD) is the most common birth defect, yet a large fraction of cases remain undetected on prenatal ultrasound, in part because current artificial-intelligence methods assume that the key diagnostic frames have already been isolated from a study, by a clinician or by a view classifier. We remove that assumption and address CHD screening directly at the level of the whole ultrasound study. We propose a two-stage framework that first learns transferable frame representations by self-supervised masked-autoencoder pre-training on unlabeled fetal ultrasound, then identifies cardiac frames with a disease-robust module and aggregates them with a transformer-based multiple instance learning (MIL) model that produces a case-level diagnosis from study-level labels alone. The model further returns its highest-scoring frames for clinician review, and a hierarchical head separates critical from non-critical CHD. On the internal test set of our multi-source development cohort (FUSE), the proposed cardiac-gated MIL model reaches an area under the curve (AUC) of 0.985 with a specificity of 0.990, outperforming the reproduced NATMED ensemble (AUC 0.861, specificity 0.600) and the FetalCLIP foundation model (AUC 0.867, specificity 0.710). On an independent external cohort, all models initially perform near chance, but label-free CORAL adaptation raises the proposed model from an AUC of 0.513 to 0.944, whereas whole-study and view-dependent baselines do not recover. These results indicate that whole-study MIL with disease-robust cardiac-frame identification is an accurate and deployable route to prenatal CHD screening.
Figures & tables
Fig. 1: Three designs for AI-based prenatal CHD screening. (a) Expert-selected images: a clinician manually selects representative frames, each is scored by a CHD classifier, and the per-image predictions must still be combined into a diagnosis. (b) Two-stage design: a plane classifier first identifies standard cardiac views before diagnosis. Because these classifiers are designed to recognize standard cardiac structures and are predominantly trained on normal anatomy, they may discard abnormal cardiac frames. Aggregation is again required as the design produces per-image predictions. (c) Proposed whole-study design: the entire study enters at subject level. The frame encoder embeds every frame, a disease-robust cardiac-frame identifier gates which frames enter the bag, and a MIL aggregator produces one subject-level prediction while surfacing the key diagnostic frames that support it.
Fig. 2: Overview of the proposed two-stage framework. Stage 1: self-supervised pre-training of the frame encoder with a masked-autoencoder objective on unlabeled fetal ultrasound frames. Stage 2: MIL-based CHD detection. A whole-study ultrasound is preprocessed, every frame is embedded by the pre-trained encoder, which is kept frozen (snowflake) and yields one vector per frame; the cardiac-frame identification module passes only the features of the frames it identifies as cardiac, and a transformer aggregator with a classification token pools them into a study representation. A diagnosis head predicts CHD versus HC, a hierarchical severity head, trained as a separate branch, separates critical from non-critical CHD among predicted CHD cases. The attention from the classification token to each frame ranks the key supporting frames returned for clinician review. No expert frame selection is required at any point.
Source
Pre-training subjects
Frames
In-house
419
363,182
FETAL_PLANES_DB [ 59 ]
947
4,674
MFUSPAC [ 50 ]
115
402
FASSD [ 60 ]
169
1,588
Total
1,650
369,846
TABLE I: Composition of the Stage-1 self-supervised pre-training corpus.
CHD
Cohort
Source
n
HC
Non-critical
Critical
View-annotated
Purpose
FUSE
In-house
527
171
145
211
26.0%
Training ( n=692 ) and internal testing ( n=177 )
FETAL_PLANES_DB [ 59 ]
280
280
0
0
100%
MFUSPAC [ 50 ]
61
61
0
0
100%
RFCHD [ 61 ]
1
0
1
0
100%
Total
869
512
146
211
55.1%
TABLE II: Composition of the cohorts used for diagnosis training and evaluation.
Fig. 3: CHD screening performance. (A) Internal FUSE test set (177 subjects): AUC, sensitivity, and specificity of NATMED, FetalCLIP, and the proposed cardiac-gated MIL model. (B) External CARDIUM cohort: AUC of NATMED, FetalCLIP, and the proposed model without adaptation, and of the proposed model after label-free CORAL adaptation. The dashed line marks chance.
Fig. 4: Cardiac-view classification on the view-annotated internal test studies, for (A) HC and (B) CHD studies. Axes: AUC, cardiac-view AUC, precision, recall, and accuracy, for NATMED, SonoNet, and the proposed cardiac-frame identifier.
Method
AUC
Sensitivity
Specificity
NATMED [ 13 ]
0.861
0.906
0.600
FetalCLIP [ 24 ]
0.867
0.828
0.710
Proposed (cardiac-gated MIL)
0.985
0.972
0.990
TABLE III: Internal CHD detection on 177 test studies (105 HC / 72 CHD). Best per column in bold.
Model
Input
No adapt.
After adapt.
NATMED [ 13 ]
Standard views
0.506
0.557
FetalCLIP [ 24 ]
Standard views
0.571
0.616
Cardiac-gated MIL
Cardiac frames
0.513
0.944
TABLE IV: External transfer to CARDIUM by model configuration, before and after label-free CORAL adaptation. Values are AUC.
Fig. 5: UMAP projection of study-level representations of the in-house cohort, for a supervised ResNet, FetalCLIP, and the proposed encoder. Each point is one study, embedded as the average of the frame features of its apical four-chamber frames, and colored by diagnosis (CHD or HC).
Fig. 6: Hierarchical severity classification on the internal test set, computed from calibrated probabilities. Sensitivity to CCHD and to NCCHD, balanced accuracy and AUC over the two CHD sub-classes, and specificity, for the flat three-class classifier and the two-stage hierarchical classifier.
Fig. 7: Key supporting frames returned by the MIL aggregator. For two HC subjects (top, blue) and two CHD subjects (bottom, red), the five frames of the study that receive the most class-token attention in the last aggregator layer (Eq. 7 ) are shown from left (highest attention) to right.
Fig. 8: Attention mass by view group on the 177 internal test subjects. Boxes show the fraction of each study’s last-layer class-token attention (Eq. 7 ) assigned to cardiac frames and to all non-cardiac frames (labeled non-cardiac in the plot, pooling the abdominal and non-target classes); diamonds mark the mean fraction of frames in each group.
Configuration
AUC
MIL
Cardiac gating
CORAL
Internal
CARDIUM
✓
0.958
0.18
✓
✓
0.985
0.513
✓
✓
✓
0.985
0.944
TABLE V: Ablation over the three components of the framework: MIL aggregation over the whole study, cardiac-frame gating, and label-free CORAL adaptation.
Ultrasound imaging uses sound waves to visualize anatomy and function, with AI improving image reconstruction, diagnosis, measurement, and scan guidance.
The scarcity of positive cases remains a major bottleneck in developing automated diagnostic systems for fetal Congenital Heart Defects~(CHDs), particularly for rare subtypes. Although normal cases are accessible with effort, positive cases are hard to collect due to low incidence rates and privacy concerns, causing severe class imbalance or absence of positive training samples. Existing works resort to synthesizing positives with conditional generative models, but they are task-specific and require positive training samples, hindering application for fetal CHDs. To this end, we propose a positive-free mask-guided counterfactual generation framework for fetal echocardiography, namely Pos4CHD, which enables realistic, high-quality synthesis of pseudo-positive samples from semantic masks, with only normal samples at hand. Technically, we first develop FE-Gen, a mask-guided generative model tailored for fetal heart ultrasound synthesis that mitigates histogram drift and enhances image-mask spatial alignment. Building on FE-Gen, Pos4CHD employs a mask-to-image generation paradigm for CHD synthesis. It first manipulates semantic masks to simulate fetal-heart structural anomalies and then renders anatomically consistent images. Extensive experiments on in-house datasets, together with expert evaluations, demonstrate the visual fidelity~(−12.97 FID) and anatomical consistency~(+3.96% DSC) of synthesized images. Further validations across diverse real-world CHD diagnosis tasks exhibit promising results and consistent increments, including zero-shot classification~(achieving 93.25% ACC on EbA and 72.92% ACC on TAPVC), lesion detection~(+6.95% mAP50 on VSD), and anatomy segmentation~(+10.49% DSC on fetal heart) compared with baselines. Codes and models are publicly available on GitHub.
We present a semi-supervised framework for joint segmentation and classification of fetal cardiac ultrasound images. Built upon the EchoCare multi-task backbone, our method integrates SAM-Med2D for boundary refinement and leverages DINOv3 to enhance pseudo-label quality. We introduce view-specific hard masking along with a two-stage optimization strategy: an EMA phase to consolidate segmentation capabilities, followed by a Classification Fine-Tuning phase that freezes segmentation parameters and resets the classification head to recover classification performance without compromising segmentation gains. Evaluated on the FETUS 2026 leaderboard, our method achieves a Dice Similarity Coefficient at 79.99%, Normalized Surface Distance at 61.62%, and F1-score at 41.20%, validating the effectiveness of our approach for prenatal congenital heart disease screening. Source code is publicly available at: https://github.com/2826056177/zcst_fetus2026.
Tonghao Zhuang, Shanglong Hu, Yongsheng Luo +2
Zhuhai College of Science and Technology, Zhuhai, China
Congenital heart disease (CHD) refers to the abnormal anatomical structure caused by the abnormal development of the heart and great vessels during embryonic development. Traditional diagnostics often fail to achieve high accuracy and efficiency, especially given the complexity of cardiac anatomy. This study presents a specialized multi-view deep learning framework for CHD binary classification using echocardiographic images. A large-scale CHD dataset, including five views, was used to train the model, enabling it to integrate multi-angle image data. The framework utilizes advanced feature extraction and attention mechanisms to improve diagnostic precision and reliability. An uncertainty-based decision-making component is also integrated to handle low-quality images, enhancing diagnostic outcomes. Experimental results show that this method achieves top-tier performance on our dataset and provides a robust tool for early CHD detection, underscoring its potential for clinical use. The dataset and source code will be released upon paper acceptance.
Tan Zhou, Shifa Yao, Suncheng Xiang +2
School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China · Department of Ultrasonography, the International Peace Maternity and Child Health Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Embryo Original Diseases, Shanghai, China +1