A cross-modal generative model for incomplete and degraded prostate MRI with multicentre clinical validation

Siyuan Ma; Liang He; Mengying Zhu; Yi Chai; Mengyao Lyu; Haowei Wang; Qizhen Lan; HaoBo Sun

eess.IVAug 17, 2026

A cross-modal generative model for incomplete and degraded prostate MRI with multicentre clinical validation

Authors: Siyuan Ma, Liang He, Mengying Zhu, Yi Chai, Mengyao Lyu, Haowei Wang, Qizhen Lan, HaoBo Sun, +9 more

Organizations: College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science and Technology, Tongji University, Shanghai, China · Department of Radiology, Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China · National University of Singapore, Singapore · Department of Medical Oncology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, China · McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston (UTHealth Houston), Houston, Texas, USA · Jiangsu Key Laboratory of Urban ITS, Jiangsu Province Collaborative Innovation Center of Modern Urban Traffic Technologies, School of Transportation, Southeast University, Nanjing 210096, China · Department of Radiology, Ningbo Hangzhou Bay Hospital, Ningbo, Zhejiang, China · Department of Radiology, Changhai Hospital, Naval Medical University, Shanghai, China

Abstract

Missing or degraded sequences can limit prostate multiparametric MRI. We developed MSCNet, a sequence-conditioned cross-modal generative framework for reconstructing unavailable contrasts and restoring degraded acquisitions. Across ten completion tasks, task-specific MSCNet achieved mean structural similarity of 0.818 versus 0.798 for the strongest task-matched comparators; matched-capacity analyses showed larger differences in lesion fidelity and boundary preservation. In a blinded 1,000-case reader study, overall image quality met the prespecified non-inferiority criterion for DWI, ADC and T2W completion, but not T1W. In a separate 200-case diagnostic assessment, AUCs for clinically significant cancer were 0.860 with acquired images, 0.841 with MSCNet and 0.797 with baseline-generated images. A locked 186-case three-hospital cohort supported multicentre transportability. These retrospective results support quality-controlled cross-modal reconstruction as an adjunct to acquired prostate MRI.

Figures & tables

Figure 1: Study overview, cohorts, completion tasks and MSCNet framework. A, Clinical motivation and workflow for incomplete or degraded prostate mpMRI, cross-modal completion or enhancement, and radiologist review. B, Public, institutional, artefact and reader-study cohorts. C, Matrix of 2-to-1 and 3-to-1 completion tasks. D, MSCNet architecture, synthesis outputs and optimization objectives, including modality-specific encoders, multi-scale cross-modal gated fusion, global-context modelling, attention-gated decoding, deep supervision and edge-aware reconstruction.

Figure 2: Visual and spatial behaviour of multi-sequence prostate MRI completion. A, Input sequences, acquired reference targets, baseline outputs, MSCNet outputs, residual maps and enlarged regions. B, Residual decomposition in the gland, boundary ring and peri-gland regions. C, Local structural similarity, residual error and boundary error at the apex, mid-gland and base. D, Adjacent-slice consistency, slice-to-slice variation, volumetric smoothness and z-trend correlation. E, Edge preservation, boundary SSIM, gradient similarity and capsule sharpness.

Figure 3: Optimized quantitative performance and clinical-anatomical effect size. A, Task-level SSIM for MSCNet and the strongest task-specific non-MSCNet comparator. B, Absolute SSIM gain for each completion task. C, Grouped macro SSIM for physical-coupling, cross-contrast and T1W tasks. D, Task-level LPIPS for MSCNet and the strongest task-specific comparator. E, Matched-capacity analysis on the locked subset, showing whole-image SSIM, lesion fidelity and boundary SSIM. F, Internal and external-hospital SSIM, including the three external sites and pooled estimate.

Figure 4: Clinical artefact transfer and virtual enhancement. A, Workflow from clinical artefact collection and pattern extraction to cross-domain transfer, strict pairing, restoration and safety review. B, Representative susceptibility, motion, banding, Gibbs-ringing and chemical-shift cases before and after restoration. C, Before-and-after image-quality distributions by artefact type. D, Correlation among distortion, structural and perceptual metrics. E, Comparison with baseline restoration models. F, Artefact-type and severity distribution in the clinical cohort.

Figure 5: Blinded radiologist reader study. A, Sampling, task strata, anonymization, block randomization, reader experience, Likert endpoints and statistical plan. B, Mean scores for five clinical dimensions across four completion tasks. C, Overall image-quality distributions. D, Diagnostic-confidence distributions. E, Task-level lesion-conspicuity scores. F, Reference and MSCNet score distributions relative to the prespecified reference-minus-0.5 margin. G, Improvement of MSCNet over the baseline across tasks and clinical dimensions.

Figure 6: Diagnostic validation and clinical usability. A, Separate diagnostic cohort, image conditions, blinded reading and pathology or clinical follow-up reference standard. Summary values in the endpoint panel correspond to the MSCNet condition and match the quantitative results in B. B, AUC, sensitivity, specificity, positive predictive value, negative predictive value and AUC difference for acquired, MSCNet and baseline images. C, Lesion Dice, centre-distance error, volume overlap and boundary-distance error. D, PI-RADS agreement matrix in the 1,000-case reader-study set. E, Paired reading time and diagnostic confidence before and after AI assistance. F, Retrospective usability categories after AI assistance.

Cohort	Source	Sequences	Size or split	Primary role
PI-CAI	Public, multicentre	T2W, ADC, high-b-value DWI	1,476; 1,000/200/276	Public-domain development and held-out evaluation of 2-to-1 tasks
PROSTATEx	Public, Siemens	T2W, DWI, ADC	346	Vendor-specific public-cohort analysis
Institutional three-sequence	Ren Ji Hospital, Shanghai, China	T2W, DWI, ADC	3,189; 2,232/318/639	Training, validation and testing of 2-to-1 tasks
Institutional four-sequence	Ren Ji Hospital, Shanghai, China	T2W, DWI, ADC, T1W	1,595; 1,111/157/327	Training, validation and testing of 3-to-1 tasks
Clinical artefact	Ren Ji Hospital archive	Degraded prostate MRI sequences	182 patients; 262,306 DICOM slices; 204 repeat pairs	Controlled artefact transfer, same-patient repeat-scan validation and safety evaluation
Reader study	Ren Ji Hospital held-out cohort	Four representative tasks	1,000; 250 per task	Three-reader blinded assessment

Extended Data Table 1: Study cohorts and analytical roles. Splits were defined at the patient level. PROSTATEx was treated as a vendor-specific public cohort rather than independent external validation.

Dataset	Task	Input $\rightarrow$ target	PSNR	SSIM	LPIPS	MAE
PI-CAI	P1	T2W + ADC $\rightarrow$ high-b DWI	25.91 (+0.53)	0.8600 (+0.0180)	0.0890 ( $-$ 0.0100)	0.0384 ( $-$ 0.0036)
PI-CAI	P2	T2W + high-b DWI $\rightarrow$ ADC	28.43 (+0.48)	0.8890 (+0.0170)	0.0700 ( $-$ 0.0110)	0.0261 ( $-$ 0.0029)
PI-CAI	P3	ADC + high-b DWI $\rightarrow$ T2W	23.07 (+0.66)	0.7300 (+0.0240)	0.1480 ( $-$ 0.0160)	0.0573 ( $-$ 0.0047)
Institutional	A	T2W + ADC $\rightarrow$ DWI	24.17 (+0.46)	0.8460 (+0.0180)	0.0950 ( $-$ 0.0110)	0.0621 ( $-$ 0.0039)
Institutional	B	T2W + DWI $\rightarrow$ ADC	27.46 (+0.44)	0.8660 (+0.0180)	0.0830 ( $-$ 0.0100)	0.0382 ( $-$ 0.0038)
Institutional	C	ADC + DWI $\rightarrow$ T2W	21.59 (+0.61)	0.7140 (+0.0240)	0.1620 ( $-$ 0.0160)	0.0662 ( $-$ 0.0048)

Extended Data Table 2: Main quantitative performance of MSCNet. Values in parentheses are absolute changes relative to the strongest non-MSCNet baseline. PSNR and SSIM are higher-is-better; LPIPS and MAE are lower-is-better.

Domain	Endpoint	Comparison or statistic	Result
Reader study	Overall image quality	Reference / MSCNet / strongest baseline	4.71 / 4.32 / 3.23
Reader study	Anatomical fidelity	Reference / MSCNet; secondary descriptive endpoint	4.67 / 4.29
Reader study	Lesion conspicuity	Reference / MSCNet; lesion-containing cases	4.46 / 4.07
Reader study	Diagnostic confidence	Reference / MSCNet / strongest baseline	4.76 / 4.38 / 3.31
Reader study	Artefact absence	Reference / MSCNet / strongest baseline	4.82 / 4.56 / 3.41; $P<0.001$
Reader agreement	Inter-reader agreement	ICC; weighted $\kappa$	0.87; 0.83

Extended Data Table 3: Clinical reader-study and diagnostic-validation summary. Reader scores were obtained in 1,000 held-out cases; diagnostic discrimination was evaluated in a separate 200-case set.

Explore similar work

Posterior Samplings are Missing Modalities Generators for Medical Image Translation
Jul 21, 2026Jonghun KimMissing-Modality Learning Image-to-Image Translation
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea
Backbone-Conditional Behavior of Modality Gating in Multi-Modal Prostate MRI Segmentation: A 5-Fold Cross-Validation and Gate Mechanism Analysis
Apr 12, 2026Yongbo Shu, Wenzhao Xie, Shanhu Yao +4Missing-Modality Learning Cross-Modality Medical Image Segmentation
The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China · School of Life Sciences, Central South University, Changsha, 410013, Hunan, China · Hunan Provincial Key Laboratory of Medical Information Research (Central South University), Changsha, 410011, Hunan, China +2
MPFlow: Multi-modal Posterior-Guided Flow Matching for Zero-Shot MRI Reconstruction
Mar 4, 2026Seunghoi Kim, Chen Jin, Henry F. J. Tregidgo +2Cross-Modal Alignment Multimodal Pretraining
University College London, UK · Centre for AI, DS&AI, AstraZeneca, UK

Jul 21, 2026J/K move · Enter open · S save

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

Jonghun Kim

Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Missing-Modality Learning Image-to-Image Translation Posterior Sampling Medical Image Synthesis

Open paper AskPDF arXiv

Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete multimodal acquisitions are often unavailable due to time and protocol constraints. This leads to real-world datasets with missing modalities, where conventional medical image translation methods are typically limited to fixed source-target settings or require retraining for each observed source-target pair. We propose a unified framework that formulates missing-modality generation as a linear inverse problem under a joint distribution and solves it via posterior sampling with a flow matching model. By learning a joint prior over the complete modality set, our method can reconstruct arbitrary missing modalities at inference time by guiding the sampling trajectory to enforce measurement consistency with observed modalities. We further mitigate inter-modality error propagation in multi-target generation by adopting a many-to-one sampling strategy. Experiments on BraTS and IXI datasets show that our method achieves the best performance over baselines across most missing-modality scenarios. In downstream tumor segmentation, synthesized images from our method result in higher segmentation performance, indicating better preservation of clinically relevant structures.

Abstract

Explore similar work

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

Backbone-Conditional Behavior of Modality Gating in Multi-Modal Prostate MRI Segmentation: A 5-Fold Cross-Validation and Gate Mechanism Analysis

MPFlow: Multi-modal Posterior-Guided Flow Matching for Zero-Shot MRI Reconstruction