Image-Based Breast Implant Detection for Mammography Dataset Curation and Near-Real-Time Deployment: Comparing Foundation Models and Task-Specific Convolutional Models

Vasisht Ishwar; Hari Trivedi; Young Seok Jeon; Beatrice Brown-Mulry; Frank Li; Rohan Satya Isaac; Mohammadreza Chavoshi; Judy Wawira Gichoya

eess.IVOct 1, 2026

Image-Based Breast Implant Detection for Mammography Dataset Curation and Near-Real-Time Deployment: Comparing Foundation Models and Task-Specific Convolutional Models

Authors: Vasisht Ishwar, Hari Trivedi, Young Seok Jeon, Beatrice Brown-Mulry, Frank Li, Rohan Satya Isaac, Mohammadreza Chavoshi, Judy Wawira Gichoya

Organizations: Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA, USA · University of Maryland, College Park, MD, USA

Abstract

Purpose: To evaluate the performance-feasibility tradeoffs of foundation models (FMs) and task-specific convolutional neural networks (CNNs) trained from scratch for breast implant classification in 2D mammography, with emphasis on suitability for near real-time clinical deployment. Methods: We evaluated four models: two FMs (RAD-DINO and MammoCLIP) and two CNNs trained from scratch for implant prediction (ResNet18 and our lightweight ResNetLite). Using the Emory Breast Imaging Dataset, 5,000 unilateral screening mammograms were used for training/validation and 1,000 manually reviewed unilateral images were held out for testing. For the FMs, global image embeddings from the pretrained encoder were classified using a support vector machine (SVM). The CNNs were trained end-to-end on 2D mammograms, with ResNetLite optimized via grid search over depth and width to balance accuracy and efficiency. Performance was evaluated using AUROC, sensitivity, specificity, accuracy, embedding visualization, and inference-latency. Results: All models demonstrated strong performance on held-out test data (n = 1,000). MammoCLIP achieved the highest AUROC (0.999) with the quickest training time of 493 seconds. RAD-DINO achieved the highest sensitivity (0.980; accuracy 0.989) but had the slowest inference and training times. ResNet18 and MammoCLIP achieved comparable accuracy (0.985). ResNetLite showed no statistically significant difference from ResNet18 (AUROC 0.993; accuracy 0.976) despite using only 1.4% of ResNet18's parameters, and had the fastest inference time. Conclusion: FMs and task-specific CNN models reliably detect breast implants on 2D mammography. Model selection is best guided by deployment context: MammoCLIP for GPU-equipped hospital settings requiring scalable integration, and lightweight CNNs such as ResNetLite for resource-constrained or edge deployments.

Figures & tables

Figure 1: Data curation workflow for the implant visibility dataset. The process starts with the EMBED dataset, merging the metadata and clinical data to establish ground truth labels. Images were preliminarily based on “BreastImplantPresent” (BIP) status and “implanfind” tags that could be found in the metadata and clinical data respectively. A subset of 6,000 images, balanced across varying view positions, underwent manual review to generate the final dataset used for training and testing.

Figure 2: Overview of the dual modeling pipeline using the balanced 6,000-image dataset. The workflow diverges into two approaches: (left) A FM approach utilizing feature extraction and support vector machine (SVM) classification, and (right) a convolutional neural network (CNN) approach using a pre-trained ResNet. Both pipelines utilize identical train/validation/test splits (4,000/1,000/1,000) and preprocessing steps (LUT transformation and resizing) before converging on final evaluation metrics.

Figure 3: ResNetLite optimization and final architecture for implant detection. (a) Grid search heatmap displaying the trade-off between model depth and width against validation accuracy. Cell contents specify the total parameters. The highlighted cell (star) indicates the optimal configuration (155.0K parameters) selected for efficiency. (b) The network takes a $224\times 224\times 1$ grayscale mammography image as input and applies a $3\times 3$ stem convolution (32 channels), followed by three residual stages. Stage 1 (32 filters, stride 1) preserves spatial resolution with an identity skip connection. Stages 2 and 3 (64 filters each, stride 2) each halve the spatial resolution ( $224\times 224\rightarrow 112\times 112\rightarrow 56\times 56$ ) and use a $1\times 1$ convolution on the skip connection to match dimensions for the residual addition. Each residual block contains two $3\times 3$ convolutional layers, for a total of 6 convolutional layers across the network. A global average pooling (GAP) layer and a fully connected (FC) layer produce the final binary classification (implant / no implant).

Category	Subcategory	Number of Images	Percentage of Total
Total Images		6000	100%
Implant Visibility	Visible (True)	3000	50%
	Not Visible (False)	3000	50%
View Position	MLO (Mediolateral Oblique)	3064	51.1%
	CC (Craniocaudal)	2824	47.0%
	Implant-Displaced	112	1.9%

Table 1: Composition of the final 6,000-image dataset, showing the balanced split between implant-visible and non-visible cases and the distribution across mammographic view positions.

Figure 4: Overlap of implant-containing exams across label sources. Venn diagram illustrating the overlap of exams of patients containing implants between three sources of labels to identify implant visibility: DICOM “BreastImplantPresent” tags, Clinical Data “implanfind” tags, and View Position tags. The minimal intersection of all three sources from the entire EMBED dataset ( $n$ = 50) compared to the large unique subset found in the DICOM data ( $n$ = 19588) demonstrates poor agreement between the tags, indicating that each source captures a partially distinct subset of implant cases.

Metric	ResNetLite	ResNet18	MammoCLIP	RAD-DINO
Sensitivity (Recall)	$0.9640\pm 0.0165$	$0.9741\pm 0.0136$	$0.9722\pm 0.0143$	$0.9802\pm 0.0124$
Specificity	$0.9880\pm 0.0094$	$0.9961\pm 0.0051$	$0.9980\pm 0.0031$	$0.9980\pm 0.0031$
Precision	$0.9877\pm 0.0094$	$0.9960\pm 0.0052$	$0.9980\pm 0.0032$	$0.9980\pm 0.0032$
F1-Score	$0.9757\pm 0.0096$	$0.9849\pm 0.0079$	$0.9849\pm 0.0076$	$0.9890\pm 0.0065$
Accuracy	$0.9760\pm 0.0095$	$0.9851\pm 0.0075$	$0.9851\pm 0.0075$	$0.9891\pm 0.0065$
AUROC	$0.9929\pm 0.0046$	$0.9943\pm 0.0051$	$0.9989\pm 0.0013$	$0.9986\pm 0.0016$

Table 2: Standard evaluation metrics for: (1) a lightweight ResNet (ResNetLite) model, (2) a ResNet18 convolutional neural network, (3) a support vector machine (SVM) classifier trained on embeddings from the MammoCLIP FM, and (4) a SVM trained on embeddings from the RAD-DINO FM. Metrics include sensitivity, specificity, precision, F1-score, and accuracy, along with AUROC and AUPRC. A row for model size (Parameters) is included to highlight the large efficiency gains of ResNetLite relative to MammoCLIP, RAD-DINO, and ResNet18, alongside rows for inference time and training time to compare computational requirements across models.

Figure 5: UMAP visualization of feature embeddings by implant status. UMAP visualizations of feature embeddings demonstrate differences in clustering quality and separation between implant (red) and no implant (blue) cases. (a) RAD-DINO shows two distinguishable clusters, though with a degree of overlap at the boundary between classes, suggesting good but imperfect feature discrimination. (b) MammoCLIP demonstrates the tightest and most distinct separation among all models, with the two classes forming compact, well-isolated clusters and minimal overlap. (c) ResNet18 embeddings form four tight clusters, reflecting strong high-level feature representations consistent with the model’s discriminative behavior before classification. (d) ResNetLite exhibits weaker clustering structure and more overlap, however, this qualitative difference in embedding structure did not correspond to a statistically significant difference in classification performance relative to ResNet18.

Figure 6: Examples of model predictions with classification accuracy. Examples of model predictions are shown with green borders indicating correct classifications (true positives and true negatives) and red borders indicating incorrect classifications (false positives and false negatives). Upon visual inspection, misclassifications were more common in cases with high breast tissue density or high image contrast, suggesting that these image characteristics present challenges for the models.

Explore similar work

Jun 30, 2026cs.CV

AEGIS: A Multi-Task Joint-Embedding Predictive Architecture for Mammography

We present Aegis, a joint-embedding predictive architecture for breast cancer detection and density assessment in mammography. We train three Vision Transformer variants (Small/Base/Large) using self-supervised joint-embedding predictive architecture (JEPA) pre-training on 71,103 studies from 14 clinical sites, followed by supervised fine-tuning with progressive resolution scaling up to 2048x1536. On a curated 785-study test set, our largest model achieves area under the receiver operating characteristic curve (AUC) 0.949 for breast cancer triage with 93% sensitivity and 75% specificity at the optimal operating point. An ensemble combining our model with a U.S. Food and Drug Administration-cleared baseline further improves discrimination to 0.952 AUC. For breast density classification, the model achieves 0.953 AUC for binary (dense vs. non-dense) classification and 62.6% exact accuracy across four Breast Imaging Reporting and Data System (BI-RADS) categories, with 98.8% adjacent accuracy comparable to reported human inter-reader agreement. External validation on the public VinDr-Mammo dataset provides evidence of cross-population transfer under a different reference standard, with the largest model achieving 0.871 AUC for triage in a zero-shot setting.

Scott Chase Waggener, Sai Karthik Navuluru, Lakshman Tamil

Department of Electrical and Computer Engineering University of Texas at Dallas Richardson, TX

Breast Cancer Joint-Embedding Predictive Architecture Medical Image Classification Mammography Oncology

Ask PDF

Aug 10, 2026cs.CV

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization

Joint exam-level prediction and candidate-region localization may improve the usefulness of AI support in mammography. We study this setting using a multi-task DETR framework, where shared representations support both image-level malignancy prediction and lesion localization, and evaluate its performance on OPTIMAM and a biopsy-confirmed SGM1k cohort. Across both datasets, modern backbones consistently outperformed older ResNet-style features, with ConvNeXtV2 and DINOv3 giving the strongest overall results, whereas MambaVision was less competitive. On OPTIMAM, ConvNeXtV2 achieved the best overall performance, reaching 97.96% AUC, 99.89% sensitivity, 25.08% [email protected], and 74.38% [email protected]. On SGM1k, DINOv3 gave the strongest overall results, with 90.97% AUC, 86.28% sensitivity, 82.00% specificity, 27.04% [email protected], and 77.32% [email protected]. These findings suggest that backbone quality is a critical factor in effective multi-task mammography, with ConvNeXtV2 emerging as a particularly strong and well-matched CNN backbone for mammography in this framework.

Dinh Tan Nguyen, Quang-Hien Kha, Le-Hoang Nguyen +8

University of Technology Sydney, Australia. · Saigon Precision Medicine Research Center, Vietnam. · College of Medicine, Taipei Medical University, Taiwan. +3

Detection Transformer Multi-Task Learning Medical Image Classification Mammography

Ask PDF

Sep 22, 2026cs.CV

Foundation model embeddings capture pre-diagnostic changes on screening mammograms

Foundation model embeddings of screening mammograms may encode pre-diagnostic tissue change without task-specific adaptation. We tested whether embeddings move faster along a data-derived "cancer direction" in women later biopsied for cancer than in matched screen-negative controls, and whether this depends on pretraining domain. We studied 1,773 biopsied women (785 malignant, 988 biopsy-negative) and 1,773 matched controls, each with at least two annual screening exams before their index exam. An identical pipeline was applied to four 2D models: Mammo-CLIP (MC, out-of-distribution mammography), HOPPR (in-distribution mammography), MedImageInsight (MII, general medical imaging), and BiomedCLIP (biomedical vision-language pretraining on literature figures). Breast-level embeddings quantified longitudinal movement along the cancer direction. We compared cases and controls using a between-patient design with complementary mixed-effects analysis, and biopsied versus healthy contralateral breasts within patients. Under matched modality in MII embedding space, malignant cases drifted significantly faster than controls in the first two screening intervals preceding the index exam; biopsy-negative cases showed significance only in the first. MC differences were significant in the first interval for both biopsy groups. Within-patient comparisons showed a broadly similar pattern, with MC significance extending to the second interval in both groups and HOPPR showing significance at interval 1. BiomedCLIP showed no significant differences in either design or biopsy group. Overall, directional embedding velocity emerges as a property of clinically grounded rather than general biomedical pretraining, showing that foundation model embeddings can encode pre-diagnostic mammographic change without task-specific adaptation.

Kalina P. Slavkova, Eric Brattain, Aditya Gowd +6

HOPPR, Chicago, IL, USA · University of Utah School of Medicine, Salt Lake City, UT, USA · 4DMedical, Woodland Hills, CA, USA +1

Breast Cancer Medical Imaging Foundation Models Longitudinal Medical Image Analysis Mammography Medical Image Representation Learning

Ask PDF