Image-Based Breast Implant Detection for Mammography Dataset Curation and Near-Real-Time Deployment: Comparing Foundation Models and Task-Specific Convolutional Models

Vasisht Ishwar; Hari Trivedi; Young Seok Jeon; Beatrice Brown-Mulry; Frank Li; Rohan Satya Isaac; Mohammadreza Chavoshi; Judy Wawira Gichoya

eess.IVOct 1, 2026

Image-Based Breast Implant Detection for Mammography Dataset Curation and Near-Real-Time Deployment: Comparing Foundation Models and Task-Specific Convolutional Models

Authors: Vasisht Ishwar, Hari Trivedi, Young Seok Jeon, Beatrice Brown-Mulry, Frank Li, Rohan Satya Isaac, Mohammadreza Chavoshi, Judy Wawira Gichoya

Organizations: Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA, USA · University of Maryland, College Park, MD, USA

Abstract

Purpose: To evaluate the performance-feasibility tradeoffs of foundation models (FMs) and task-specific convolutional neural networks (CNNs) trained from scratch for breast implant classification in 2D mammography, with emphasis on suitability for near real-time clinical deployment. Methods: We evaluated four models: two FMs (RAD-DINO and MammoCLIP) and two CNNs trained from scratch for implant prediction (ResNet18 and our lightweight ResNetLite). Using the Emory Breast Imaging Dataset, 5,000 unilateral screening mammograms were used for training/validation and 1,000 manually reviewed unilateral images were held out for testing. For the FMs, global image embeddings from the pretrained encoder were classified using a support vector machine (SVM). The CNNs were trained end-to-end on 2D mammograms, with ResNetLite optimized via grid search over depth and width to balance accuracy and efficiency. Performance was evaluated using AUROC, sensitivity, specificity, accuracy, embedding visualization, and inference-latency. Results: All models demonstrated strong performance on held-out test data (n = 1,000). MammoCLIP achieved the highest AUROC (0.999) with the quickest training time of 493 seconds. RAD-DINO achieved the highest sensitivity (0.980; accuracy 0.989) but had the slowest inference and training times. ResNet18 and MammoCLIP achieved comparable accuracy (0.985). ResNetLite showed no statistically significant difference from ResNet18 (AUROC 0.993; accuracy 0.976) despite using only 1.4% of ResNet18's parameters, and had the fastest inference time. Conclusion: FMs and task-specific CNN models reliably detect breast implants on 2D mammography. Model selection is best guided by deployment context: MammoCLIP for GPU-equipped hospital settings requiring scalable integration, and lightweight CNNs such as ResNetLite for resource-constrained or edge deployments.

Figures & tables

Figure 1: Data curation workflow for the implant visibility dataset. The process starts with the EMBED dataset, merging the metadata and clinical data to establish ground truth labels. Images were preliminarily based on “BreastImplantPresent” (BIP) status and “implanfind” tags that could be found in the metadata and clinical data respectively. A subset of 6,000 images, balanced across varying view positions, underwent manual review to generate the final dataset used for training and testing.

Figure 2: Overview of the dual modeling pipeline using the balanced 6,000-image dataset. The workflow diverges into two approaches: (left) A FM approach utilizing feature extraction and support vector machine (SVM) classification, and (right) a convolutional neural network (CNN) approach using a pre-trained ResNet. Both pipelines utilize identical train/validation/test splits (4,000/1,000/1,000) and preprocessing steps (LUT transformation and resizing) before converging on final evaluation metrics.

Figure 3: ResNetLite optimization and final architecture for implant detection. (a) Grid search heatmap displaying the trade-off between model depth and width against validation accuracy. Cell contents specify the total parameters. The highlighted cell (star) indicates the optimal configuration (155.0K parameters) selected for efficiency. (b) The network takes a $224\times 224\times 1$ grayscale mammography image as input and applies a $3\times 3$ stem convolution (32 channels), followed by three residual stages. Stage 1 (32 filters, stride 1) preserves spatial resolution with an identity skip connection. Stages 2 and 3 (64 filters each, stride 2) each halve the spatial resolution ( $224\times 224\rightarrow 112\times 112\rightarrow 56\times 56$ ) and use a $1\times 1$ convolution on the skip connection to match dimensions for the residual addition. Each residual block contains two $3\times 3$ convolutional layers, for a total of 6 convolutional layers across the network. A global average pooling (GAP) layer and a fully connected (FC) layer produce the final binary classification (implant / no implant).

Category	Subcategory	Number of Images	Percentage of Total
Total Images		6000	100%
Implant Visibility	Visible (True)	3000	50%
	Not Visible (False)	3000	50%
View Position	MLO (Mediolateral Oblique)	3064	51.1%
	CC (Craniocaudal)	2824	47.0%
	Implant-Displaced	112	1.9%

Table 1: Composition of the final 6,000-image dataset, showing the balanced split between implant-visible and non-visible cases and the distribution across mammographic view positions.

Figure 4: Overlap of implant-containing exams across label sources. Venn diagram illustrating the overlap of exams of patients containing implants between three sources of labels to identify implant visibility: DICOM “BreastImplantPresent” tags, Clinical Data “implanfind” tags, and View Position tags. The minimal intersection of all three sources from the entire EMBED dataset ( $n$ = 50) compared to the large unique subset found in the DICOM data ( $n$ = 19588) demonstrates poor agreement between the tags, indicating that each source captures a partially distinct subset of implant cases.

Metric	ResNetLite	ResNet18	MammoCLIP	RAD-DINO
Sensitivity (Recall)	$0.9640\pm 0.0165$	$0.9741\pm 0.0136$	$0.9722\pm 0.0143$	$0.9802\pm 0.0124$
Specificity	$0.9880\pm 0.0094$	$0.9961\pm 0.0051$	$0.9980\pm 0.0031$	$0.9980\pm 0.0031$
Precision	$0.9877\pm 0.0094$	$0.9960\pm 0.0052$	$0.9980\pm 0.0032$	$0.9980\pm 0.0032$
F1-Score	$0.9757\pm 0.0096$	$0.9849\pm 0.0079$	$0.9849\pm 0.0076$	$0.9890\pm 0.0065$
Accuracy	$0.9760\pm 0.0095$	$0.9851\pm 0.0075$	$0.9851\pm 0.0075$	$0.9891\pm 0.0065$
AUROC	$0.9929\pm 0.0046$	$0.9943\pm 0.0051$	$0.9989\pm 0.0013$	$0.9986\pm 0.0016$

Table 2: Standard evaluation metrics for: (1) a lightweight ResNet (ResNetLite) model, (2) a ResNet18 convolutional neural network, (3) a support vector machine (SVM) classifier trained on embeddings from the MammoCLIP FM, and (4) a SVM trained on embeddings from the RAD-DINO FM. Metrics include sensitivity, specificity, precision, F1-score, and accuracy, along with AUROC and AUPRC. A row for model size (Parameters) is included to highlight the large efficiency gains of ResNetLite relative to MammoCLIP, RAD-DINO, and ResNet18, alongside rows for inference time and training time to compare computational requirements across models.

Figure 5: UMAP visualization of feature embeddings by implant status. UMAP visualizations of feature embeddings demonstrate differences in clustering quality and separation between implant (red) and no implant (blue) cases. (a) RAD-DINO shows two distinguishable clusters, though with a degree of overlap at the boundary between classes, suggesting good but imperfect feature discrimination. (b) MammoCLIP demonstrates the tightest and most distinct separation among all models, with the two classes forming compact, well-isolated clusters and minimal overlap. (c) ResNet18 embeddings form four tight clusters, reflecting strong high-level feature representations consistent with the model’s discriminative behavior before classification. (d) ResNetLite exhibits weaker clustering structure and more overlap, however, this qualitative difference in embedding structure did not correspond to a statistically significant difference in classification performance relative to ResNet18.

Figure 6: Examples of model predictions with classification accuracy. Examples of model predictions are shown with green borders indicating correct classifications (true positives and true negatives) and red borders indicating incorrect classifications (false positives and false negatives). Upon visual inspection, misclassifications were more common in cases with high breast tissue density or high image contrast, suggesting that these image characteristics present challenges for the models.

Explore similar work

AEGIS: A Multi-Task Joint-Embedding Predictive Architecture for Mammography
Jun 30, 2026Scott Chase Waggener, Sai Karthik Navuluru, Lakshman TamilBreast Cancer Joint-Embedding Predictive Architecture
Department of Electrical and Computer Engineering University of Texas at Dallas Richardson, TX
Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization
Aug 10, 2026Dinh Tan Nguyen, Quang-Hien Kha, Le-Hoang Nguyen +8Detection Transformer Multi-Task Learning
University of Technology Sydney, Australia. · Saigon Precision Medicine Research Center, Vietnam. · College of Medicine, Taipei Medical University, Taiwan. +3
Foundation model embeddings capture pre-diagnostic changes on screening mammograms
Sep 22, 2026Kalina P. Slavkova, Eric Brattain, Aditya Gowd +6Breast Cancer Medical Imaging Foundation Models
HOPPR, Chicago, IL, USA · University of Utah School of Medicine, Salt Lake City, UT, USA · 4DMedical, Woodland Hills, CA, USA +1

Jun 30, 2026, cs.CVJ/K move · Enter open · S save

AEGIS: A Multi-Task Joint-Embedding Predictive Architecture for Mammography

Scott Chase Waggener, Sai Karthik Navuluru, Lakshman Tamil

Department of Electrical and Computer Engineering University of Texas at Dallas Richardson, TX

Breast Cancer Joint-Embedding Predictive Architecture Medical Image Classification Mammography Oncology

Open paper AskPDF arXiv

We present Aegis, a joint-embedding predictive architecture for breast cancer detection and density assessment in mammography. We train three Vision Transformer variants (Small/Base/Large) using self-supervised joint-embedding predictive architecture (JEPA) pre-training on 71,103 studies from 14 clinical sites, followed by supervised fine-tuning with progressive resolution scaling up to 2048x1536. On a curated 785-study test set, our largest model achieves area under the receiver operating characteristic curve (AUC) 0.949 for breast cancer triage with 93% sensitivity and 75% specificity at the optimal operating point. An ensemble combining our model with a U.S. Food and Drug Administration-cleared baseline further improves discrimination to 0.952 AUC. For breast density classification, the model achieves 0.953 AUC for binary (dense vs. non-dense) classification and 62.6% exact accuracy across four Breast Imaging Reporting and Data System (BI-RADS) categories, with 98.8% adjacent accuracy comparable to reported human inter-reader agreement. External validation on the public VinDr-Mammo dataset provides evidence of cross-population transfer under a different reference standard, with the largest model achieving 0.871 AUC for triage in a zero-shot setting.

Abstract

Explore similar work

AEGIS: A Multi-Task Joint-Embedding Predictive Architecture for Mammography

Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization

Foundation model embeddings capture pre-diagnostic changes on screening mammograms