DualMiT-Net: Local-Global Transformer-Convolutional Fusion for Breast Mass Segmentation in Mammographic Regions of Interest
Organizations: Department of Mathematics and Computer Science, Lawrence Technological University · College of Business and Information Technology, Lawrence Technological University
Abstract
Breast mass segmentation is an important step in computer-aided mammography, but it remains difficult because masses can have low contrast, irregular shapes, and boundaries that blend with surrounding breast tissue. To address this problem, we present DualMiT-Net, a dual-branch network that uses both a focused view of the mass and a wider view of the surrounding tissue. The local branch uses a Mix Transformer (MiT-B5) encoder to learn mass shape, texture, and boundary information, while the global branch uses an EfficientNet-B5 encoder to learn surrounding breast context. Features from the two branches are shared at the deeper encoder levels and are then progressively fused in a single decoder. A spatial gate controls how much global information is added during decoding. We also evaluated four input representations and selected a percentile-windowed mammogram combined with a Gabor texture response. The model was trained and evaluated on the mass subset of the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM) using a patient-level split. Across three training runs, DualMiT-Net with exponential moving average weights achieved a mean Dice coefficient of 0.9375 and a mean Intersection over Union of 0.8834. It also achieved better Dice and IoU scores than six standard encoder-decoder baselines trained using the same data and training settings. These results show that combining local mass information with wider breast context can provide accurate and consistent breast mass segmentation.
Figures & tables
| Study | Method | Main idea | Dice | IoU |
|---|---|---|---|---|
| AUNet [ 18 ] | Attention-based encoder-decoder | Attention-guided dense upsampling for whole mammograms | 0.8180 | - |
| AM-MSP-cGAN [ 19 ] | Conditional GAN | Attention and multi-scale pooling with adversarial learning | 0.8449 | - |
| ARF-Net [ 22 ] | Adaptive receptive-field network | Selects receptive fields for masses with different sizes | 0.8575 | - |
| MTLNet [ 20 ] | Attentive multi-task network | Joint segmentation, localization, and classification | 0.8630 | - |
| Connected-ResUNets [ 21 ] | Connected residual U-Net | Connected U-Net stages with residual learning | 0.8952 | 0.8002 |
| YOLO-LOGO [ 15 ] | Local-global transformer | Detection followed by local and global transformer segmentation | 0.7450 | 0.6400 |
| Subset | Patients | Paired cases | Image files | Mask files |
|---|---|---|---|---|
| Training | 480 | 753 | 753 | 753 |
| Validation | 62 | 91 | 91 | 91 |
| Test | 64 | 92 | 92 | 92 |
| Architecture | Weights | Dice | IoU | Precision | Recall | Specificity | Accuracy | HD95 |
|---|---|---|---|---|---|---|---|---|
| Full DualMiT-Net | Base | |||||||
| Full DualMiT-Net | EMA | |||||||
| Without scSE | Base | |||||||
| Without scSE | EMA | |||||||
| Without G2L sharing | Base | |||||||
| Without G2L sharing | EMA |
| Input representation | Base Dice | Base IoU | Base HD95 | EMA Dice | EMA IoU | EMA HD95 |
|---|---|---|---|---|---|---|
| Window only | ||||||
| Window + CLAHE | ||||||
| Window + Gabor | ||||||
| Window + CLAHE + Gabor |
| Architecture | Weights | Dice | IoU | Precision | Recall | Specificity | Accuracy | HD95 |
|---|---|---|---|---|---|---|---|---|
| Full DualMiT-Net | Base | 0.9369 [0.9336, 0.9401] | 0.8824 [0.8767, 0.8879] | 0.9333 [0.9280, 0.9383] | 0.9418 [0.9364, 0.9467] | 0.9102 [0.9029, 0.9168] | 0.9278 [0.9242, 0.9312] | 6.85 [6.42, 7.33] |
| Full DualMiT-Net | EMA | 0.9375 [0.9342, 0.9406] | 0.8834 [0.8777, 0.8889] | 0.9282 [0.9212, 0.9344] | 0.9482 [0.9436, 0.9529] | 0.9022 [0.8924, 0.9111] | 0.9280 [0.9242, 0.9316] | 6.88 [6.45, 7.39] |
| Without G2L sharing | Base | 0.9363 [0.9331, 0.9393] | 0.8814 [0.8758, 0.8867] | 0.9267 [0.9194, 0.9345] | 0.9474 [0.9400, 0.9538] | 0.8998 [0.8892, 0.9115] | 0.9266 [0.9230, 0.9300] | 6.97 [6.57, 7.42] |
| Without G2L sharing | EMA | 0.9371 [0.9339, 0.9402] | 0.8829 [0.8772, 0.8883] | 0.9273 [0.9219, 0.9326] | 0.9484 [0.9445, 0.9522] | 0.9008 [0.8936, 0.9076] | 0.9275 [0.9239, 0.9310] | 6.93 [6.56, 7.33] |
| Without scSE | Base | 0.9368 [0.9334, 0.9400] | 0.8822 [0.8764, 0.8878] | 0.9368 [0.9310, 0.9422] | 0.9380 [0.9320, 0.9437] | 0.9158 [0.9081, 0.9230] | 0.9280 [0.9244, 0.9314] | 6.82 [6.44, 7.26] |
| Without scSE | EMA | 0.9372 [0.9340, 0.9402] | 0.8829 [0.8773, 0.8883] | 0.9238 [0.9187, 0.9287] | 0.9522 [0.9488, 0.9555] | 0.8953 [0.8889, 0.9016] | 0.9273 [0.9237, 0.9307] | 6.95 [6.54, 7.43] |
| Comparison | Metric | Mean difference | 95% CI | -value | Significance |
|---|---|---|---|---|---|
| Full vs. without scSE | Dice | n.s. | |||
| Full vs. without G2L sharing | Dice | n.s. | |||
| Full vs. without scSE | IoU | n.s. | |||
| Full vs. without G2L sharing | IoU | n.s. | |||
| Full vs. without scSE | HD95 | n.s. | |||
| Full vs. without G2L sharing | HD95 | n.s. |
| Model | Dice | IoU | Precision | Recall |
|---|---|---|---|---|
| U-Net (ResNet-50) | ||||
| U-Net++ (ResNet-50) | ||||
| U-Net++ (ResNet-34) | ||||
| U-Net (ResNet-34) | ||||
| FPN (ResNet-34) | ||||
| PSPNet (ResNet-34) |
| Method | Dice | IoU |
|---|---|---|
| YOLO-LOGO [ 15 ] | 0.7450 | 0.6400 |
| AUNet [ 18 ] | 0.8180 | - |
| AM-MSP-cGAN [ 19 ] | 0.8449 | - |
| ARF-Net [ 22 ] | 0.8575 | - |
| MTLNet [ 20 ] | 0.8630 | - |
| Connected-ResUNets [ 21 ] | 0.8952 | 0.8002 |
Medical Image Segmentation
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
See all 788 papers