HUydra: Full-Range Lung CT Synthesis via Multiple HU Interval Generative Modelling

António Cardoso; Pedro Sousa; Tania Pereira; Hélder P. Oliveira

cs.CVMar 24, 2026

HUydra: Full-Range Lung CT Synthesis via Multiple HU Interval Generative Modelling

Authors: António Cardoso, Pedro Sousa, Tania Pereira, Hélder P. Oliveira

Organizations: INESC TEC, Portugal · Faculty of Engineering, University of Porto, Portugal · Faculty of Sciences, University of Porto, Portugal

Abstract

Currently, a central challenge and bottleneck in the deployment and validation of computer-aided diagnosis (CAD) models within the field of medical imaging is data scarcity. For lung cancer, one of the most prevalent types worldwide, limited datasets can delay diagnosis and have an impact on patient outcome. Generative AI offers a promising solution for this issue, but dealing with the complex distribution of full Hounsfield Unit (HU) range lung CT scans is challenging and remains as a highly computationally demanding task. This paper introduces a novel decomposition strategy that synthesizes CT images one HU interval at a time, rather than modelling the entire HU domain at once. This framework focuses on training generative architectures on individual tissue-focused HU windows, then merges their output into a full-range scan via a learned reconstruction network that effectively reverses the HU-windowing process. We further propose multi-head and multi-decoder models to better capture textures while preserving anatomical consistency, with a multi-head VQVAE achieving the best performance for the generative task. Quantitative evaluation shows this approach significantly outperforms conventional 2D full-range baselines, achieving a 6.2% improvement in FID and superior MMD, Precision, and Recall across all HU intervals. The best performance is achieved by a multi-head VQVAE variant, demonstrating that it is possible to enhance visual fidelity and variability while also reducing model complexity and computational cost. This work establishes a new paradigm for structure-aware medical image synthesis, aligning generative modelling with clinical interpretation.

Figures & tables

Tissue	Hounsfield Unit (HU)
Air	$-1000$
Lung	$[-700,-600]$
Fat	$[-120,-90]$
Water	$0$
Abscess/pus	$0$ or $20$ , $40$ or $45$
Blood	$[30,45]$

Table 1: Hounsfield units for different tissues and materials ( Lev and Gonzalez, 2002 )

Figure 1: General GAN framework, where the Discriminator

D

predicts the probability of authenticity for both real images from the dataset or synthetic ones, generated from the output of the Generator

G

given a latent vector

z

Figure 2: Effect of forward and reverse diffusion processes on a chest CT scan slice.

Figure 3: General VQVAE framework. A given image

x

passes through the Encoder network

E

to produce a continuous latent representation

z

. Next, given the codebook

\mathcal{Z}

with

C

vectors, the quantization operator

\mathbf{q}

produces a discrete latent representation

z_{\mathbf{q}}

. Finally the decoder network

D

predicts a reconstruction of the original image

\hat{x}

by decoding

z_{\mathbf{q}}

. The transformer module auto-regressively generates sequences of discrete values that try to match those produced by

z_{\mathbf{q}}

, allowing the synthesis of new images.

Panel of Figure 4 — Figure 4: Illustration of the proposed CT reconstruction networks. (a) An MLP-based model that receives, for each voxel, a vector of clipped-scaled HU values sampled from the corresponding voxel location across multiple input slices, and predicts the scaled full-range HU value using a sequence of fully connected–ReLU blocks followed by a sigmoid output layer. (b) A CNN-based model that takes a K-channel image in which each channel encodes a clipped-scaled HU slice, and reconstructs the corresponding scaled full-range CT image through a series of convolution–ReLU blocks and a final convolutional layer with sigmoid activation.

Figure 5: Multi-channel WGAN-GP framework.

Figure 6: Multi-channel score-based diffusion model framework.

Figure 7: Multi-channel VQVAE framework.

Figure 8: Multi-decoder VQVAE framework.

Figure 10: Example of a full-range CT sample and its respective HU-clipped views for the HU intervals $[-950,-700]$ , $[-500,-200]$ , $[30,70]$ and $[100,1000]$ .

Reconstruction Model	Model Type	Parameters	Description
MLP 0	MLP	5	Zero hidden layers
MLP 4	MLP	25	One hidden layer with feature size 4
MLP 4×4	MLP	45	Two hidden layers, all with feature size 4
MLP 4×4×4	MLP	65	Three hidden layers, all with feature size 4
CNN 3	CNN	37	One convolution with kernel size 3
CNN 7	CNN	197	One convolution with kernel size 7

Table 2: Configuration of the experimented reconstruction models

Model Type	Approach	Parameters (Millions)	Reconstruction Model
WGAN-GP	Baseline	6.4	-
	Multi-channel	6.8	MLP 0
	Multi-channel	6.8	CNN 3×3×3
Score-based DM	Baseline	72.3	-
	Multi-channel	72.3	MLP 0
	Multi-channel	72.3	CNN 3×3×3

Table 3: Configuration of the experimented CT and HU generative models

Reconstruction Model	MSE $(\times 10^{-4})\downarrow$	FID $\downarrow$	MMD $(\times 10^{-2})\downarrow$	Precision $\uparrow$	Recall $\uparrow$	MS-SSIM $\uparrow$
MLP 0	4.68	44.9	3.29	0.969	0.989	0.962
MLP 4	4.74	44.6	3.21	0.974	0.988	0.962
MLP 4×4	4.74	44.6	3.21	0.974	0.988	0.962
MLP 4×4×4	4.46	41.9	2.93	0.977	0.995	0.964
CNN 3	4.62	40.4	2.66	0.989	0.993	0.962
CNN 7	4.54	40.1	2.68	0.988	0.993	0.962

Table 4: MSE, FID, MMD, Precision, Recall and MS-SSIM metrics for each full-rage CT reconstruction model.

Figure 11: Three full-range CT images reconstructed from four distinct models. The first column presents the original content of the full-range CT slices. The following columns frame the outputs of the reconstruction models MLP 0 , MLP 4×4×4 , CNN 3 , and CNN 3×3×3 , respectively, given the HU-clipped images of the original slice.

Figure 12: Full-range CT and HU-windowed samples obtained from the proposed VQVAE methods and CNN 3×3×3 reconstruction model.

Model Type	Approach	Reconstruction Model	FID per HU Interval
Model Type	Approach	Reconstruction Model	Full-range	$\mathbf{[-950,-700]}$	$\mathbf{[-500,-200]}$	$\mathbf{[30,70]}$	$\mathbf{[100,1000]}$
WGAN-GP	Baseline	-	141.8	117.5	131.5	157.2	177.5
	Multi-channel	MLP 0	211.3	196.1	137.7	134.7	219.4
	Multi-channel	CNN 3×3×3	184.9	145.0	121.9	138.2	205.9
Score-based DM	Baseline	-	66.8	81.3	56.3	85.6	81.2
	Multi-channel	MLP 0	84.9	106.5	134.9	134.5	192.3

Table 5: FID computed per HU interval for each generative model configuration.

Model Type	Approach	Reconstruction Model	MMD ( $\times 10^{-2}$ ) per HU Interval
Model Type	Approach	Reconstruction Model	Full-range	$\mathbf{[-950,-700]}$	$\mathbf{[-500,-200]}$	$\mathbf{[30,70]}$	$\mathbf{[100,1000]}$
WGAN-GP	Baseline	-	17.00	8.93	14.51	15.78	16.24
	Multi-channel	MLP 0	28.06	19.96	15.08	9.00	20.56
	Multi-channel	CNN 3×3×3	23.14	11.84	13.19	9.94	18.69
Score-based DM	Baseline	-	4.81	5.31	2.69	3.42	4.07
	Multi-channel	MLP 0	9.84	9.69	17.72	12.98	20.53

Table 6: MMD computed per HU interval for each generative model configuration.

Model Type	Approach	Reconstruction Model	Precision per HU Interval
Model Type	Approach	Reconstruction Model	Full-range	$\mathbf{[-950,-700]}$	$\mathbf{[-500,-200]}$	$\mathbf{[30,70]}$	$\mathbf{[100,1000]}$
WGAN-GP	Baseline	-	0.022	0.121	0.132	0.262	0.056
	Multi-channel	MLP 0	0.001	0.004	0.122	0.269	0.034
	Multi-channel	CNN 3×3×3	0.001	0.062	0.213	0.210	0.029
Score-based DM	Baseline	-	0.747	0.642	0.780	0.680	0.739
	Multi-channel	MLP 0	0.311	0.218	0.066	0.313	0.182

Table 7: Precision computed per HU interval for each generative model configuration.

Model Type	Approach	Reconstruction Model	Recall per HU Interval
Model Type	Approach	Reconstruction Model	Full-range	$\mathbf{[-950,-700]}$	$\mathbf{[-500,-200]}$	$\mathbf{[30,70]}$	$\mathbf{[100,1000]}$
WGAN-GP	Baseline	-	0.016	0.060	0.048	0.023	0.006
	Multi-channel	MLP 0	0.003	0.005	0.045	0.079	0.002
	Multi-channel	CNN 3×3×3	0.000	0.038	0.119	0.078	0.008
Score-based DM	Baseline	-	0.263	0.282	0.483	0.427	0.464
	Multi-channel	MLP 0	0.100	0.073	0.027	0.044	0.007

Table 8: Recall computed per HU interval for each generative model configuration.

Model Type	Approach	Reconstruction Model	MS-SSIM per HU Interval
Model Type	Approach	Reconstruction Model	Full-range	$\mathbf{[-950,-700]}$	$\mathbf{[-500,-200]}$	$\mathbf{[30,70]}$	$\mathbf{[100,1000]}$
WGAN-GP	Baseline	-	0.392	0.371	0.384	0.371	0.617
	Multi-channel	MLP 0	0.382	0.335	0.357	0.402	0.642
	Multi-channel	CNN 3×3×3	0.373	0.357	0.373	0.404	0.653
Score-based DM	Baseline	-	0.343	0.291	0.356	0.339	0.659
	Multi-channel	MLP 0	0.352	0.293	0.355	0.340	0.642

Table 9: MS-SSIM computed per HU interval, over all possible pairs of images of the synthetic sample set, for each generative model configuration.

Figure 13: Sample examples with respective full-range and HU-windowed representations from the multi-channel approaches and CNN 3×3×3 reconstruction model.

Figure 14: Synthetic full-range CT images sampled from the multi-head VQVAE and CNN 3×3×3 reconstruction model and respective segmentation masks outputted from the segmentation model from ( Sousa et al., 2022 ) .

Figure 15: Visual representation of the computed intensity heatmaps for both the full LIDC-IDRI dataset (left) and a population of 100 synthetic samples (center). The difference in sample number (1018 vs. 100) explains the texture difference between the real and fake heatmaps (blurry vs blocky). The rightmost image displays the difference between the two heatmaps and a color scale, revealing the maximum discrepancy between intensity distributions at any given pixel is valued at 30 out of 255.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 309 papers

May 28, 2026

Controllable Lung Nodule Synthesis via Histogram-Regularized Latent Diffusion Models

While automated diagnosis systems have achieved remarkable success in computed tomography (CT)-based lung cancer screening, their development remains limited by the scarcity of diverse, annotated pulmonary nodule datasets. Diffusion-based generative models offer a promising strategy for data synthesis; however, many existing conditional approaches primarily optimize spatial reconstruction losses, which encourage voxel-wise similarity but may inadequately constrain lesion-level intensity distributions. As a result, these methods may produce over-smoothed texture profiles and underrepresent the distinct attenuation characteristics of different nodule subtypes, including solid, part-solid, and ground-glass nodules. To address this challenge, we propose a controllable latent diffusion model that synthesizes pulmonary nodules within full 3D CT volumes while accurately modeling nodule-specific intensity distributions. Specifically, rather than relying solely on spatial losses, we introduce a histogram-based regularization term that constrains voxel intensity distributions during the generative process. The model combines subtype, spatial mask, and Hounsfield unit (HU) histogram conditioning with the differentiable feature-space histogram regularization term to better align lesion-level intensity distributions, improving the visual plausibility and subtype consistency of synthesized nodules. Extensive experiments on lung CT data demonstrate that our framework achieves strong visual realism, validated through both quantitative metrics and a visual Turing test. Furthermore, when used for data augmentation, the generated nodules improve performance in downstream clinical tasks, particularly for underrepresented nodule subtypes, and show a potential benefit for subtype-informed malignancy classification.

Arunkumar Kannan, Yanbo Zhang, Han Liu +5

aDigital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · bJohns Hopkins University, Baltimore, MD, USA · cDigital Technology and Innovation, Siemens Healthineers, Erlangen, Germany +1

Synthetic Data Augmentation Latent Diffusion Models Medical Image Synthesis Conditional Diffusion Models

Ask PDF

Jun 17, 2026

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

We introduce the first generative foundation model for chest radiograph synthesis trained from scratch at the billion-parameter scale. Existing radiographic AI models often suffer from poor generalisation across patient subpopulations, institutions, and acquisition settings, resulting in limited real-world clinical utility. Controlled, high-fidelity synthesis of chest radiographs is a promising path toward diversifying clinical datasets and evaluating the robustness of diagnostic models. Therefore, we present the largest specialist generative foundation model for chest radiographs to date, with over 1.3B parameters, trained for 1.6T tokens on a curated, heterogeneous dataset comprising 1.2M radiographs and clinical expert-guided metadata. Our model supports controllable radiograph generation and editing across multiple demographic subgroups, acquisition views, and a dozen pathologies. Moreover, we significantly advance the state of the art in radiograph synthesis fidelity, producing images that are indistinguishable from real radiographs to clinical experts.

Fabio De Sousa Ribeiro, Emma A. M. Stanley, Charles Jones +7

Imperial College London · Causality in Healthcare AI Hub · Cleveland Clinic London +4

Medical Imaging Foundation Models Medical Image Synthesis Conditional Generative Modeling

Ask PDF

Jun 18, 2026

Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation

Generating high-resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel-Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level. PRDiT introduces a two-stage training architecture comprising 1) a local denoiser in the form of an MLP-based blind estimator operating on overlapping 3D patches to separate low-frequency structures efficiently, and 2) a global residual diffusion transformer employing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC-IDRI and RAD-ChestCT datasets demonstrate that PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.

Zhenkai Zhang, Markus Hiller, Krista A. Ehinger +1

School of Computing and Information Systems, The University of Melbourne · School of Computing

Diffusion Transformer 3D Medical Imaging Medical Image Synthesis Computed Tomography

Ask PDF

Abstract

Medical Image Synthesis

Explore similar work

Controllable Lung Nodule Synthesis via Histogram-Regularized Latent Diffusion Models

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation