Hybrid Diffusion Model for Breast Ultrasound Image Augmentation

Farhan Fuad Abir; Sanjeda Sara Jennifer; Niloofar Yousefi; Laura J. Brattain

eess.IVMar 27, 2026

Hybrid Diffusion Model for Breast Ultrasound Image Augmentation

Authors: Farhan Fuad Abir, Sanjeda Sara Jennifer, Niloofar Yousefi, Laura J. Brattain

Organizations: Electrical and Computer Engineering, University of Central Florida · Department of Medicine, University of Central Florida College of Medicine · Department of Computer Science, University of Central Florida · Industrial Engineering and Management Systems, University of Central Florida

Abstract

We propose a hybrid diffusion-based augmentation framework to overcome the critical challenge of ultrasound data augmentation in breast ultrasound (BUS) datasets. Unlike conventional diffusion-based augmentations, our approach improves visual fidelity and preserves ultrasound texture by combining text-to-image generation with image-to-image (img2img) refinement, as well as fine-tuning with low-rank adaptation (LoRA) and textual inversion (TI). Our method generated realistic, class-consistent images on an open-source Kaggle breast ultrasound image dataset (BUSI). Compared to the Stable Diffusion v1.5 baseline, incorporating TI and img2img refinement reduced the Frechet Inception Distance (FID) from 45.97 to 33.29, demonstrating a substantial gain in fidelity while maintaining comparable downstream classification performance. Overall, the proposed framework effectively mitigates the low-fidelity limitations of synthetic ultrasound images and enhances the quality of augmentation for robust diagnostic modeling.

Figures & tables

Figure 1: Overview of the proposed hybrid diffusion-based image generation framework for breast ultrasound. The method consists of three main stages. (a) The preprocessing stage converts the labels into descriptive prompts. (b) LoRA finetuning and Token Generation adapts the Stable Diffusion v1.5 using LoRA for image–prompt alignment and Textual Inversion for learning domain-specific <ultrasound> token. (c) In the final workflow, the prompts and learned token <ultrasound> are passed through the finetuned text-to-image (text2img) model to generate synthetic images, which are further refined using an image-to-image (img2img) stage with LoRA weights, yielding the final synthetic ultrasound images.

Components	Accuracy $\uparrow$	F1-Score $\uparrow$	AUC-ROC $\uparrow$	PPV $\uparrow$	FID $\downarrow$
Baseline (Original Images)	0.904	0.887	0.979	0.890	-
Original + SD1.5	0.917	0.905	0.986	0.901	45.97
Original + SD1.5 + img2img	0.898	0.879	0.978	0.878	38.34
Original + SD1.5 + TI	0.924	0.912	0.980	0.906	45.66
Original + SD1.5 + TI + img2img	0.905	0.884	0.975	0.89	37.18

Table 1: Impact of each component in the hybrid diffusion framework on classification and image quality metrics.

Figure 2: Comparison of real ultrasound images and synthetic variants generated by 4 different approaches based on SD1.5. Rows correspond to breast lesion categories: benign (top), malignant (middle), and normal (bottom). Columns show (from left to right): real images, baseline SD1.5 generations, SD1.5 with img2img refinement, SD1.5 with TI, and SD1.5 combined with TI and img2img. The img2img refinement increases the fidelity by improving ultrasound textures.

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 249 papers

AWDiff: An a trous wavelet diffusion model for lung ultrasound image synthesis
Mar 3, 2026Maryam Heidari, Nantheera Anantrasirichai, Steven Walker +2Medical Image Synthesis Synthetic Data Augmentation
University of Bristol, UK · Visual Information Lab, University of Bristol, Bristol, UK · Bristol Medical School, University of Bristol, UK +1
Feature-Space Guided Diffusion for Realistic Ultrasound Image Synthesis
Jul 13, 2026Marina Domínguez, Nélida Mirabet-Herranz, Valery NaranjoDiffusion Model Guidance Diffusion Guidance
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain · Artikode Intelligence S.L., Valencia, Spain
HADS-Net:A Hybrid Attention-Augmented Dual-Stream Network with Physics-Informed Augmentation for Breast Ultrasound Image Classification
May 19, 2026Chinedu Emmanuel Mbonu, Blessing Nwamaka Iduh, Joseph Ikechukwu Odo +1Data Augmentation Medical Image Classification
Department of Computer Science, Nazarbayev University, Astana · Department of Computer Science, Nnamdi Azikiwe University Awka, Nigeria · Department of Computer Science, Peter University Achina-Onneh, Nigeria

Mar 3, 2026J/K move · Enter open · S save

AWDiff: An a trous wavelet diffusion model for lung ultrasound image synthesis

Maryam Heidari, Nantheera Anantrasirichai, Steven Walker, Rahul Bhatnagar+1

University of Bristol, UK · Visual Information Lab, University of Bristol, Bristol, UK · Bristol Medical School, University of Bristol, UK+1

Medical Image Synthesis Synthetic Data Augmentation Diffusion-Based Image Generation Diffusion Models

Open paper AskPDF arXiv

Lung ultrasound (LUS) is a safe and portable imaging modality, but the scarcity of data limits the development of machine learning methods for image interpretation and disease monitoring. Existing generative augmentation methods, such as Generative Adversarial Networks (GANs) and diffusion models, often lose subtle diagnostic cues due to resolution reduction, particularly B-lines and pleural irregularities. We propose A trous Wavelet Diffusion (AWDiff), a diffusion based augmentation framework that integrates the a trous wavelet transform to preserve fine-scale structures while avoiding destructive downsampling. In addition, semantic conditioning with BioMedCLIP, a vision language foundation model trained on large scale biomedical corpora, enforces alignment with clinically meaningful labels. On a LUS dataset, AWDiff achieved lower distortion and higher perceptual quality compared to existing methods, demonstrating both structural fidelity and clinical diversity.

Abstract

Data Augmentation

Explore similar work

AWDiff: An a trous wavelet diffusion model for lung ultrasound image synthesis

Feature-Space Guided Diffusion for Realistic Ultrasound Image Synthesis

HADS-Net:A Hybrid Attention-Augmented Dual-Stream Network with Physics-Informed Augmentation for Breast Ultrasound Image Classification