A Hybrid Architecture for Benign-Malignant Classification of Mammography ROIs

Mohammed Asad; Mohit Bajpai; Sudhir Singh; Rahul Katarya

cs.CVApr 14, 2026

A Hybrid Architecture for Benign-Malignant Classification of Mammography ROIs

Authors: Mohammed Asad, Mohit Bajpai, Sudhir Singh, Rahul Katarya

Organizations: Dept. of Electronics and Communication Engineering, Delhi Technological University, Delhi, India · Dept. of Information Technology, IGDTUW, Delhi, India · Dept. of Computer Science and Engineering, Delhi Technological University, Delhi, India

Abstract

Accurate characterization of suspicious breast lesions in mammography is important for early diagnosis and treatment planning. While Convolutional Neural Networks (CNNs) are effective at extracting local visual patterns, they are less suited to modeling long-range dependencies. Vision Transformers (ViTs) address this limitation through self-attention, but their quadratic computational cost can be prohibitive. This paper presents a hybrid architecture that combines EfficientNetV2-M for local feature extraction with Vision Mamba, a State Space Model (SSM), for efficient global context modeling. The proposed model performs binary classification of abnormality-centered mammography regions of interest (ROIs) from the CBIS-DDSM dataset into benign and malignant classes. By combining a strong CNN backbone with a linear-complexity sequence model, the approach achieves strong lesion-level classification performance in an ROI-based setting.

Figures & tables

Fig. 1: Vision Mamba architecture, illustrating bidirectional state space modeling for efficient global context capture.

Fig. 2: Proposed hybrid architecture combining EfficientNetV2-M for local feature extraction and Vision Mamba for global context modeling.

Model	Params (M)	AUC	Acc. (%)	Sens.	Spec.	F1
VGG-16 [ 17 ] *	138.0	0.795	87.2	0.78	0.89	0.80
ResNet-50 [ 18 ] *	23.5	0.812	89.1	0.80	0.90	0.82
Inception-v3 [ 19 ] *	23.8	0.819	89.8	0.81	0.91	0.83
DenseNet-121 [ 20 ] *	7.0	0.825	90.3	0.82	0.92	0.84
ViT-B/16 [ 4 ] *	86.0	0.831	90.8	0.83	0.92	0.85
Two-View EfficientNet-B0 [ 3 ]	5.3	0.848	92.1	0.85	0.93	0.87

TABLE I: Performance Comparison on the CBIS-DDSM Test Set

Model	AUC	Acc. (%)	Sens.	Spec.	F1
EfficientNetV2-M	0.850	92.5	0.85	0.93	0.87
Vision Mamba	0.820	89.0	0.80	0.90	0.82
EffNetV2-M + ViM	0.875	94.2	0.89	0.95	0.90

TABLE II: Ablation Study on the CBIS-DDSM Test Set

Explore similar work

Attend what matters: Leveraging vision foundational models for breast cancer classification using mammograms
Apr 21, 2026Samyak Sanghvi, Piyush Miglani, Sarvesh Shashikumar +3Medical Imaging Vision Foundation Model Adaptation
Department of Computer Science and Engineering, IIT Delhi, New Delhi, India · Yardi School of AI, IIT Delhi, New Delhi, India · Department of Radiodiagnosis, PGIMER Chandigarh, Chandigarh, India
SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification
Apr 17, 2026Enhui Chai, Sicheng Chen, Tianyi Zhang +2Self-Supervised Learning Pathology Foundation Models
School of Computer Science, Northwest University · PuzzleLogic Pte Ltd, Singapore 229594, Singapore · Department of Electrical & Computer Engineering, National University of Singapore +1
Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification
Jul 17, 2026Zhengbo Zhou, Jiren Li, Dooman Arefan +2Radiology VLMs Vision-Language Models
Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA · Department of Radiology, University of Pittsburgh, Pittsburgh, PA, USA · Department of Biomedical Informatics and Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA

Apr 21, 2026J/K move · Enter open · S save

Attend what matters: Leveraging vision foundational models for breast cancer classification using mammograms

Samyak Sanghvi, Piyush Miglani, Sarvesh Shashikumar, Kaustubh R Borgavi+2

Department of Computer Science and Engineering, IIT Delhi, New Delhi, India · Yardi School of AI, IIT Delhi, New Delhi, India · Department of Radiodiagnosis, PGIMER Chandigarh, Chandigarh, India

Medical Imaging Vision Foundation Model Adaptation Medical Image Classification Mammography Oncology

Open paper AskPDF arXiv

Vision Transformers

(\texttt{ViT})

have become the architecture of choice for many computer vision tasks, yet their performance in computer-aided diagnostics remains limited. Focusing on breast cancer detection from mammograms, we identify two main causes for this shortfall. First, medical images are high-resolution with small abnormalities, leading to an excessive number of tokens and making it difficult for the softmax-based attention to localize and attend to relevant regions. Second, medical image classification is inherently fine-grained, with low inter-class and high intra-class variability, where standard cross-entropy training is insufficient. To overcome these challenges, we propose a framework with three key components: (1) Region of interest

(\texttt{RoI})

based token reduction using an object detection model to guide attention; (2) contrastive learning between selected

\texttt{RoI}

to enhance fine-grained discrimination through hard-negative based training; and (3) a

\texttt{DINOv2}

pretrained

\texttt{ViT}

that captures localization-aware, fine-grained features instead of global

\texttt{CLIP}

representations. Experiments on public mammography datasets demonstrate that our method achieves superior performance over existing baselines, establishing its effectiveness and potential clinical utility for large-scale breast cancer screening. Our code is available for reproducibility here: https://aih-iitd.github.io/publications/attend-what-matters

Abstract

Explore similar work

Attend what matters: Leveraging vision foundational models for breast cancer classification using mammograms

SSMamba: A Self-Supervised Hybrid State Space Model for Pathological Image Classification

Region-Grounded Vision-Language Learning for Detection-Guided Mammographic Lesion Classification