A Diffusion-Driven Fine-Grained Nodule Synthesis Framework for Enhanced Lung Nodule Detection from Chest Radiographs

Aryan Goyal; Shreshtha Singh; Ashish Mittal; Manoj Tadepalli; Piyush Kumar; Preetham Putha

cs.CVMar 2, 2026

A Diffusion-Driven Fine-Grained Nodule Synthesis Framework for Enhanced Lung Nodule Detection from Chest Radiographs

Authors: Aryan Goyal, Shreshtha Singh, Ashish Mittal, Manoj Tadepalli, Piyush Kumar, Preetham Putha

Organizations: Qure.ai, India · Indian Institute of Technology, Bombay

Abstract

Early detection of lung cancer in chest radiographs (CXRs) is crucial for improving patient outcomes, yet nodule detection remains challenging due to their subtle appearance and variability in radiological characteristics like size, texture, and boundary. For robust analysis, this diversity must be well represented in training datasets for deep learning based Computer-Assisted Diagnosis (CAD) systems. However, assembling such datasets is costly and often impractical, motivating the need for realistic synthetic data generation. Existing methods lack fine-grained control over synthetic nodule generation, limiting their utility in addressing data scarcity. This paper proposes a novel diffusion-based framework with low-rank adaptation (LoRA) adapters for characteristic controlled nodule synthesis on CXRs. We begin by addressing size and shape control through nodule mask conditioned training of the base diffusion model. To achieve individual characteristic control, we train separate LoRA modules, each dedicated to a specific radiological feature. However, since nodules rarely exhibit isolated characteristics, effective multi-characteristic control requires a balanced integration of features. We address this by leveraging the dynamic composability of LoRAs and revisiting existing merging strategies. Building on this, we identify two key issues, overlapping attention regions and non-orthogonal parameter spaces. To overcome these limitations, we introduce a novel orthogonality loss term during LoRA composition training. Extensive experiments on both in-house and public datasets demonstrate improved downstream nodule detection. Radiologist evaluations confirm the fine-grained controllability of our generated nodules, and across multiple quantitative metrics, our method surpasses existing nodule generation approaches for CXRs.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of nodule generations across different methods: ACGAN Odena et al. (2017) , ReACGAN Lee et al. (2021) , CR-Fill Zhao et al. (2021) , and Ours.

	In-house			JSRT	CheX-ray14
	Diffusion Trainset	Downstream Trainset	Testset	Testset	Testset
Total samples	1,100,000	100,000	12,000	247	500
Nodule samples	28,000	10,000	2,000	154	66

Table 1: Summary of datasets used for training and evaluation.

Panel of Figure 2 — Figure 2: Diffusion Backbone generation on a chest X-ray patch: given an original CXR patch and a binary nodule mask, the model generates a nodule within the masked region

Figure 3: Overview of the pipeline with characteristic control. Training proceeds in 3 stages: Stage 1 (Base Model Training) -pre-train DiT-XL/2 on 2M of CXR patches; Stage 2 (Nodule Patch Fine-tuning) -finetune on nodule-centered patches with binary nodule masks for localized synthesis; Stage 3.1 (Characteristic-Specific LoRA Training) -train individual LoRA adapters (calcification, homogeneity, border regularity, subtlety) with the backbone frozen, using characteristic-curated datasets; Stage 3.2 (LoRA Merge Training) -jointly train selected adapters with an orthogonality regularizer

Panel of Figure 4 — Figure 4: Results of characteristic-specific LoRA training: (a) nodule with a regular border, (b) nodule with an irregular border, (c) nodule with homogeneous texture, (d) nodule with inhomogeneous texture, and (e) nodule with calcification.

Panel of Figure 5 — Figure 5: Subtlety slider generations on a CXR patch. From left to right: decreasing LoRA scale $\alpha$ values produce more subtle nodules.

Figure 6: Layer-wise Frobenius norm comparison of cross-trained vs. separately trained adapters across two characteristic pairs.

Nodule characteristic	Agreement (%)
Calcification	80
Regular border	90
Irregular border	100
Homogeneous texture	90
Inhomogeneous texture	100

Table 2: Radiologist evaluation of characteristic-specific LoRA modules.

Panel of Figure 7 — Figure 7: Qualitative comparison between Linear Merge and our orthogonal merging across three nodule-attribute configurations: (a) calcified + irregular, (b) calcified + homogeneous, (c) regular + homogeneous.

Train Data	In-house		JSRT		CheX-ray14
	AUC	IoU	AUC	IoU	AUC	IoU
10k real	0.9705	0.3090	0.8560	0.2475	0.9008	0.5285
10k real + 2k syn	0.9788	0.3222	0.8639	0.2743	0.9168	0.5293
10k real + 4k syn	0.9780	0.3197	0.8780	0.2589	0.9245	0.5500
10k real + 6k syn	0.9802	0.3247	0.8940	0.2894	0.9315	0.5750
10k real + 8k syn	0.9796	0.3274	0.8864	0.2923	0.9341	0.5954

Table 3: Quantitative evaluation of the effectiveness of different quantities of synthesized nodule data. Reported metrics include AUC and best IoU on three test sets.

Characteristic	5k Real	5k Real + 2k fake
Nodule	0.2696	0.3002
Calcification	0.2879	0.3199
Regular Border	0.2941	0.3301
Irregular Border	0.2733	0.3080
Homogeneous	0.2695	0.3050
Inhomogeneous	0.2706	0.2963

Table 4: Comparison of models against IOU scores trained with 5k real nodules versus 5k real nodules with 2k characteristic specific synthetic nodules across radiological features.

Augmentation	JSRT	ChestX-ray14
10k real	0.8560	0.9008
10k real + 10k ACGAN	0.8780	0.9281
10k real + 10k ReACGAN	0.8808	0.9259
10k real + 10k CRFILL	0.8786	0.9296
10k real + 10k DiT-XL/2(Ours)	0.9023	0.9318

Table 5: Comparison of effect of synthetic-data augmentation on nodule AUC scores across ChestX-ray14 and JSRT .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Model type	DiT-XL/2
Input size (latent)	32
Patch size	2
Hidden size	1152
Depth	28
Attention heads	16

Appendix

Table 6: Backbone Training Hyperparameters (DiT-XL/2)

Parameter	Value
Rank ( $r$ )	32
Scaling $\alpha$	1.0
Training method	noxattn
Epochs	150
Batch size	200
Learning rate	$5\times 10^{-5}$

Appendix

Table 7: LoRA Adapter Training Hyperparameters

Parameter	Value
Temperature $\tau$	0.07
Margin	1.0
Feature dimension	1152
Pooling	Mean pooling
Feature normalization	L2 norm
Positive pairs	Subtlety-based grouping

Appendix

Table 8: Contrastive Fine-Tuning Hyperparameters

Parameter	Value
Ortho weight	0.5
Loss type	Frobenius norm
Target	Identity matrix
Adapter pairs	(calcified, homogeneous), (irregular, homogeneous), (calcified, irregular)
Training strategy	Alternating batches + joint optimization
Gradient accumulation	2

Appendix

Table 9: Orthogonality Regularization Hyperparameters

Component	Dimensions	Per Block	Total
QKV Adapters	$32\times 1152+3456\times 32$	147,456	4,128,768
Proj Adapters	$32\times 1152+1152\times 32$	73,728	2,064,384
Total per Adapter	–	221,184	6,193,152

Appendix

Table 10: LoRA Parameter Distribution in DiT-XL/2

Train	S5	S4	S3	S2	S1
10k Real	100%	96%	70%	69%	40%
10k Real + 12k Fake	100%	100%	76%	76%	52%

Appendix

Table 11: JSRT classification accuracy across subtlety levels (S1 = most subtle , S5 = least subtle ).

Method	Full			Masked
Method	PSNR	SSIM	FID	PSNR	SSIM	FID
ACGAN	37.18	0.916	0.534	26.51	0.768	0.831
ReACGAN	37.44	0.916	0.604	27.54	0.786	1.227
CR-Fill	37.93	0.918	0.522	29.50	0.830	0.781
DiT-XL/2 (Ours)	38.74	0.920	0.390	34.26	0.892	0.475

Appendix

Table 12: Comparison of generation methods on ChestX-ray14

Characteristic	CFG Control (FID)	LoRA Control (FID)
Calcification	15.87 $\pm$ 1.51	2.29 $\pm$ 0.09
Regular Border	4.04 $\pm$ 0.84	1.96 $\pm$ 0.12
Irregular Border	4.19 $\pm$ 0.59	2.90 $\pm$ 0.03
Homogeneous Texture	8.48 $\pm$ 1.50	5.71 $\pm$ 0.36
Inhomogeneous Texture	12.58 $\pm$ 1.94	8.13 $\pm$ 0.28

Appendix

Table 13: FID comparison between CFG-based label control and LoRA-based control across nodule characteristics.

Metric	Real images		Synthetic images
	Rate	95% CI	Rate	95% CI
Pooled “Looks real” rate	0.867	$[0.703,0.947]$	0.700	$[0.521,0.833]$
Majority-vote “Looks real” rate	0.900	$[0.596,0.982]$	0.800	$[0.490,0.943]$

Appendix

Table 14: Task 1: Real vs Synthetic summary with 95% CI

Characteristic	Pooled “Yes” rate		Majority-vote “Yes” rate
Characteristic	Rate (k/n)	95% CI	Rate (k/n)	95% CI
Homogeneous	$24/30=0.800$	$[0.627,0.905]$	$9/10=0.900$	$[0.596,0.982]$
Inhomogeneous	$22/27=0.815$	$[0.633,0.918]$	$9/9=1.000$	$[0.701,1.000]$
Irregular border	$27/30=0.900$	$[0.744,0.965]$	$10/10=1.000$	$[0.722,1.000]$
Regular border	$28/30=0.933$	$[0.787,0.982]$	$9/10=0.900$	$[0.596,0.982]$
Calcified	$20/30=0.66$	$[0.488,0.808]$	$8/10=0.800$	$[0.490,0.943]$

Appendix

Table 15: Task 2: Characteristic verification Pooled + Majority-vote with 95% CI

Metric	Result (k/n)	Accuracy	95% CI
Pooled	$20/30$	$0.667$	$[0.488,0.808]$
Majority-vote	$8/10$	$0.800$	$[0.490,0.943]$

Appendix

Table 16: Task 3 (Subtlety ranking): Accuracy with 95% CI

Comparison	Separate	Two Adapters (Frobenius)	% Change
Calcified vs. Homogeneous	329.01	430.89	+30.97%
Homogeneous vs. Calcified	388.23	502.42	+29.41%
Irregular vs. Homogeneous	539.28	711.84	+32.00%
Homogeneous vs. Irregular	388.23	803.59	+106.99%

Appendix

Table 17: Comparison of adapter weight magnitudes for separately trained adapters versus two adapters trained jointly with Frobenius-norm regularization.

Lung Nodule Detection

See all 10 papers

May 28, 2026

Controllable Lung Nodule Synthesis via Histogram-Regularized Latent Diffusion Models

While automated diagnosis systems have achieved remarkable success in computed tomography (CT)-based lung cancer screening, their development remains limited by the scarcity of diverse, annotated pulmonary nodule datasets. Diffusion-based generative models offer a promising strategy for data synthesis; however, many existing conditional approaches primarily optimize spatial reconstruction losses, which encourage voxel-wise similarity but may inadequately constrain lesion-level intensity distributions. As a result, these methods may produce over-smoothed texture profiles and underrepresent the distinct attenuation characteristics of different nodule subtypes, including solid, part-solid, and ground-glass nodules. To address this challenge, we propose a controllable latent diffusion model that synthesizes pulmonary nodules within full 3D CT volumes while accurately modeling nodule-specific intensity distributions. Specifically, rather than relying solely on spatial losses, we introduce a histogram-based regularization term that constrains voxel intensity distributions during the generative process. The model combines subtype, spatial mask, and Hounsfield unit (HU) histogram conditioning with the differentiable feature-space histogram regularization term to better align lesion-level intensity distributions, improving the visual plausibility and subtype consistency of synthesized nodules. Extensive experiments on lung CT data demonstrate that our framework achieves strong visual realism, validated through both quantitative metrics and a visual Turing test. Furthermore, when used for data augmentation, the generated nodules improve performance in downstream clinical tasks, particularly for underrepresented nodule subtypes, and show a potential benefit for subtype-informed malignancy classification.

Arunkumar Kannan, Yanbo Zhang, Han Liu +5

aDigital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · bJohns Hopkins University, Baltimore, MD, USA · cDigital Technology and Innovation, Siemens Healthineers, Erlangen, Germany +1

Synthetic Data Augmentation Latent Diffusion Models Medical Image Synthesis Conditional Diffusion Models

Ask PDF

Mar 2, 2026

DiffusionXRay: A Diffusion and GAN-Based Approach for Enhancing Digitally Reconstructed Chest Radiographs

Deep learning-based automated diagnosis of lung cancer has emerged as a crucial advancement that enables healthcare professionals to detect and initiate treatment earlier. However, these models require extensive training datasets with diverse case-specific properties. High-quality annotated data is particularly challenging to obtain, especially for cases with subtle pulmonary nodules that are difficult to detect even for experienced radiologists. This scarcity of well-labeled datasets can limit model performance and generalization across different patient populations. Digitally reconstructed radiographs (DRR) using CT-Scan to generate synthetic frontal chest X-rays with artificially inserted lung nodules offers one potential solution. However, this approach suffers from significant image quality degradation, particularly in the form of blurred anatomical features and loss of fine lung field structures. To overcome this, we introduce DiffusionXRay, a novel image restoration pipeline for Chest X-ray images that synergistically leverages denoising diffusion probabilistic models (DDPMs) and generative adversarial networks (GANs). DiffusionXRay incorporates a unique two-stage training process: First, we investigate two independent approaches, DDPM-LQ and GAN-based MUNIT-LQ, to generate low-quality CXRs, addressing the challenge of training data scarcity, posing this as a style transfer problem. Subsequently, we train a DDPM-based model on paired low-quality and high-quality images, enabling it to learn the nuances of X-ray image restoration. Our method demonstrates promising results in enhancing image clarity, contrast, and overall diagnostic value of chest X-rays while preserving subtle yet clinically significant artifacts, validated by both quantitative metrics and expert radiological assessment.

Aryan Goyal, Ashish Mittal, Pranav Rao +2

Indian Institute of Technology Bombay, India · Qure.ai, India

Image Enhancement Medical Image Synthesis Diffusion Models for Image Restoration Image-to-Image Translation

Ask PDF

Mar 16, 2026

NAMD: Virtual Follow-up Computed Tomography Synthesis via Nodule-Aligned Multimodal Diffusion Models for Early Lung Cancer Diagnosis

Lung cancer remains the leading cause of cancer-related mortality worldwide, with survival outcomes critically dependent on early and accurate detection. When low-dose computed tomography (LDCT) findings are indeterminate, clinicians typically defer diagnosis pending follow-up CT imaging obtained up to 12 months later, inevitably delaying treatment for patients with malignant nodules. To address this clinical gap, we propose Nodule-Aligned Multimodal (Latent) Diffusion (NAMD), a novel generative framework that synthesizes one-year follow-up nodule CT images conditioned on the baseline CT scan, quantitative nodule biomarkers, and patient-level Electronic Health Records (EHR), enabling timely prediction of nodule malignant progression without requiring actual follow-up scans. NAMD introduces two key contributions: (i) a nodule-aligned latent space regularized so that embedding distances reflect clinically meaningful biomarker changes, and (ii) an LLM-driven multimodal conditioning mechanism encoding heterogeneous EHR data into the diffusion backbone. Evaluated on the National Lung Screening Trial (NLST), our method's synthetic follow-up images achieve an AUROC of 0.805 and an AUPRC of 0.346 for lung nodule malignancy prediction, outperforming both the baseline LDCT performance without virtual follow-up generation, and existing state-of-the-art conditional generation methods, while maintaining competitive image quality. These findings suggest that NAMD enables earlier and more accurate lung cancer diagnosis by capturing clinically meaningful features of nodule progression.

James Song, Yifan Wang, Chuan Zhou +1

University of Michigan · University of Michigan Medical School

Medical Image Analysis Longitudinal Medical Image Analysis Medical Image Synthesis Computed Tomography Multimodal Diffusion Models

Ask PDF

Abstract

Lung Nodule Detection

Explore similar work

Controllable Lung Nodule Synthesis via Histogram-Regularized Latent Diffusion Models

DiffusionXRay: A Diffusion and GAN-Based Approach for Enhancing Digitally Reconstructed Chest Radiographs

NAMD: Virtual Follow-up Computed Tomography Synthesis via Nodule-Aligned Multimodal Diffusion Models for Early Lung Cancer Diagnosis