The ability to leverage images from co-available modalities to inform target-domain reconstruction is highly desirable in imaging algorithms. In this work, we introduce Generative Translation Priors (GTP)--a Bayesian framework that transforms diffusion-based image-to-image translation models into cross-modality image priors for ill-posed imaging inverse problems. GTP incorporates target-domain measurements through likelihood guidance, steering the translation process toward the desired posterior distribution. The framework is grounded in a theoretical analysis of the resulting posterior dynamics, which reveals an intrinsic bias introduced by likelihood guidance. We further characterize this bias and derive a ground-truth-free formulation for its estimation, enabling it to serve as a practical metric for assessing posterior sampling quality. Building on this analysis, we derive two discretized GTP algorithms based on gradient and proximal likelihood guidance, respectively. We validate GTP on computed tomography reconstruction with magnetic resonance side information, and on positron emission tomography reconstruction with computed tomography side information. Experiments demonstrate that GTP effectively incorporates complementary cross-modality information and achieves high-fidelity reconstruction even under severely undersampled measurements.
Figures & tables
Figure 1: Schematic visualization of the proposed GTP framework. The left panel illustrates a generative image translation model that induces a target-domain prior by stochastically transporting images from a source modality to a target modality (exemplified by MRI-to-CT translation). The right panel illustrates how GTP transforms this translation prior into a posterior sampler by incorporating the measurement likelihood, steering the generative dynamics toward the target posterior distribution.
Figure 2
Figure 2: Numerical validation of the intrinsic bias as a reference-free criterion for assessing posterior accuracy. Prox-GTP (Algorithm 2 ) is used to recover the posterior across different likelihood strengths. The recovered posterior mean and pixel-wise standard deviation are shown together with the intrinsic bias (upper bound in Theorem 1 ) and two reference-based metrics: (squared) maximum mean discrepancy (MMD) and negative log-likelihood (NLL) . Note how the intrinsic bias closely tracks the reference-based metrics, detecting both under- and overweighting of the measurement likelihood and reaching its minimum when the recovered posterior best matches the ground truth.
Figure 3: Visual comparison of the proposed GTP methods against baselines for 8-view SV-CT reconstruction. The shaded panels indicate methods that leverage the reference MR image during reconstruction. Zoomed-in regions highlight differences in anatomical detail. Note how the GTP methods more accurately reconstruct important anatomical structures, particularly the kidneys, compared with the baseline methods.
Figure 5: Visual comparison of 90∘ LA-CT reconstructions obtained using the proposed GTP methods and baselines. The shaded panels indicate methods that leverage the reference MR image during reconstruction. Zoomed-in regions highlight differences in anatomical detail. Note how the GTP methods more accurately reconstruct major anatomical structures, particularly bones, that are poorly observed due to the missing angular measurements.
Figure 7: Visual comparison of low-dose PET reconstructions with 5×105 total counts. The shaded panels indicate methods that leverage the reference CT image during reconstruction. Zoomed-in regions highlight differences in fine-detail recovery. Note how Prox-GTP more clearly reconstructs the small tumor lesion, which is substantially less visible in the baseline reconstructions.
Method
105 counts
5×105 counts
106 counts
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
OSEM
32.15
0.890
35.11
0.926
35.66
0.936
PET-DDS
34.74
0.935
36.24
0.946
36.42
0.948
Prox-GTP
34.34
0.934
37.34
0.951
38.66
0.959
Table 1: Average PSNR and SSIM results attained by the proposed GTP methods and baselines for three PET reconstruction settings. In each column, the best result is marked in bold , while the second best is underlined . For reference, the translation model used by GTP attains a PSNR of 29.80 dB and an SSIM of 0.891 without measurement guidance.
Image reconstruction recovers images, volumes, or scenes from incomplete, noisy, indirect, or compressed measurements using physical models and learned priors.
Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete multimodal acquisitions are often unavailable due to time and protocol constraints. This leads to real-world datasets with missing modalities, where conventional medical image translation methods are typically limited to fixed source-target settings or require retraining for each observed source-target pair. We propose a unified framework that formulates missing-modality generation as a linear inverse problem under a joint distribution and solves it via posterior sampling with a flow matching model. By learning a joint prior over the complete modality set, our method can reconstruct arbitrary missing modalities at inference time by guiding the sampling trajectory to enforce measurement consistency with observed modalities. We further mitigate inter-modality error propagation in multi-target generation by adopting a many-to-one sampling strategy. Experiments on BraTS and IXI datasets show that our method achieves the best performance over baselines across most missing-modality scenarios. In downstream tumor segmentation, synthesized images from our method result in higher segmentation performance, indicating better preservation of clinically relevant structures.
Jonghun Kim
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea
Cross-modal image translation in remote sensing must preserve source-observed content while matching the target-domain distribution. Existing methods jointly learn the target prior and cross-modal dependence from scarce paired data, overlooking a key asymmetry: only the latter intrinsically requires cross-modal correspondence. We formalize this distinction through conditional-score and denoising-risk analyses and propose Learning the Target Priors Before Image Translation (LTP-BIT), a prior-first paradigm that decouples the two learning tasks. LTP-BIT first learns a target-domain generative prior from large-scale unpaired imagery, then retains the pretrained backbone weights and learns source-conditioned control through P-DART, a parameter-efficient dual-stream architecture. Controlled experiments show that prior matching and scaling primarily improve target-domain realism, whereas instance fidelity relies more strongly on conditional adaptation. LTP-BIT achieves state-of-the-art performance across SAR-to-RGB and NIR-to-RGB benchmarks using only 9.81% task-specific parameters. On QXS-SAROPT, it retains near-full-data instance fidelity with only 25% of the paired samples.
Keyan Hu, Mingtao Wang, Ziyu Zhou +4
Central South University, Changsha, China · Wuhan University, Wuhan, China · Guangzhou University, Guangzhou, China
Positron Emission Tomography (PET) provides important metabolic information for tumor detection and staging but remains significantly less accessible than Computed Tomography (CT) due to its high cost, infrastructure requirements, and reliance on injected radioactive tracers. CT-to-PET image translation using generative models aims to address this disparity, yet state-of-the-art diffusion-based methods are limited by the scarcity of paired training data. To overcome this, we propose a domain-adapted latent diffusion framework that transfers natural-image priors using a relatively small dataset of 21,930 paired CT/PET slices, incorporating a signed power transform to resolve extreme intensity sparsity in PET data. Compared to CPDM, a state-ofthe-art baseline for CT-to-PET image translation, our approach yields superior performance in pixel-wise reconstruction and localized tumor fidelity, demonstrating effective transfer of natural-image priors to the medical domain. We further provide a systematic analysis of evaluation metric reliability for sparse PET data, showing that standard full-image metrics can be misleading indicators of translated image quality, and propose background-filtered and mask-restricted metrics as more clinically meaningful alternatives.
Yann Bourdé, Lea Bogensperger, Arnela Hadzic +1
Department of Quantitative Biomedicine, University of Zurich · Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz