Diffusion models have emerged as powerful priors for solving inverse problems in computed tomography (CT). In certain applications, such as neutron CT, it can be expensive to collect large amounts of measurements even for a single scan, leading to sparse data sets from which it is challenging to obtain high quality reconstructions even with diffusion models. One strategy to mitigate this challenge is to leverage a complementary, easily available imaging modality; however, such approaches typically require retraining the diffusion model with large datasets. In this work, we propose incorporating an additional modality without retraining the diffusion prior, enabling accelerated imaging of costly modalities. We further examine the impact of imperfect side modalities on cross-modal guidance. Our method is evaluated on sparse-view neutron computed tomography, where reconstruction quality is substantially improved by incorporating X-ray computed tomography of the same samples.
Figures & tables
Figure 1: The cross-modal approach decouples fitting main modality measurements from enforcing cross-modal consistency. In the first step, we fine-tune the diffusion model weights to better fit the data consistency loss, then obtain the reconstruction estimate via any diffusion-based inverse problem solver. After that stage, cross-modal consistency is enforced via a lightweight image translation model. The cross-modal consistency block can be seamlessly incorporated as a separate module without modifying the prior.
Figure 2: Paired degraded NCT/XCT samples (left) and their high-quality reconstructions (right). These examples illustrate the training data used for cross-modal translation.
Figure 3: Comparison between XCT images (top row) and corresponding ideal NCT images (bottom row). XCT quality varies widely relative to NCT.
Steps
#Views
Unimodal
Cross-modal
Δ
PSNR ↑
SSIM ↑
PSNR ↑
SSIM ↑
PSNR
SSIM
5
8
20.10
0.374
21.17
0.375
+1.07
+0.001
16
23.34
0.455
23.97
0.471
+0.63
+0.016
32
25.82
0.490
27.45
0.620
+1.63
+0.130
64
28.62
0.648
29.12
0.762
+0.50
+0.114
128
28.74
0.662
28.54
0.777
-0.20
+0.115
Table 1: Comparison of reconstruction quality (PSNR [dB] / SSIM) for Unimodal vs. Cross-modal for different numbers of views, no measurement noise, and at T=10 alternating steps between enforcing the prior and data consistency. Δ=Cross−Uni .
Figure 4: Qualitative comparison between D3IP and our cross-modal guided algorithm across different numbers of projection views ( 8 , 16 , 32 , and 64 views, in rows 1–4, respectively). Within each row, the first column shows Unimodal (D3IP) reconstructions, the second column shows our cross-modal guided algorithm reconstructions, and the third column shows the ground truth samples.
Computed tomography (CT) throughput is limited by scan time, which grows with both the number of projections acquired and the detector integration time for each projection. Reconstructing high-quality volumes from sparse-view or low-dose measurements therefore depends on using an informative prior, typically a neural network trained for one specific scan setting and retrained whenever the modality, geometry, or material changes. We investigate whether a single diffusion model trained across several imaging domains can instead serve as a reusable prior for heterogeneous CT reconstruction problems. We evaluate the proposed method using the same diffusion visual transformer model and normalized denoising strength on three datasets that differ in modality, beam geometry, material, and degradation type, spanning additively manufactured metal parts and concrete microstructure imaged with cone-bean X-ray CT and parallel-beam neutron CT respectively. The proposed method improves upon analytic reconstructions in all three cases, demonstrating transferability across the evaluated problems and providing a step toward a reusable foundation prior for heterogeneous CT reconstruction.
Diffusion models have recently emerged as powerful priors for solving inverse problems. While computed tomography (CT) is theoretically a linear inverse problem, it poses many practical challenges. These include correlated noise, artifact structures, reliance on system geometry, and misaligned value ranges, which make the direct application of diffusion models more difficult than in domains like natural image generation. To systematically evaluate how diffusion models perform in this context and compare them with established reconstruction methods, we introduce DM4CT, a comprehensive benchmark for CT reconstruction. DM4CT includes datasets from both medical and industrial domains with sparse-view and noisy configurations. To explore the challenges of deploying diffusion models in practice, we additionally acquire a high-resolution CT dataset at a high-energy synchrotron facility and evaluate all methods under real experimental conditions. We benchmark ten recent diffusion-based methods alongside seven strong baselines, including model-based, unsupervised, and supervised approaches. Our analysis provides detailed insights into the behavior, strengths, and limitations of diffusion models for CT reconstruction. The real-world dataset is publicly available at zenodo.org/records/15420527, and the codebase is open-sourced at github.com/DM4CT/DM4CT.
Jiayang Shi, Daniel M. Pelt, K. Joost Batenburg
LIACS, Leiden University · Centrum Wiskunde en Informatica
Generative models, particularly Diffusion Models (DM), have shown strong potential for Computed Tomography (CT) reconstruction serving as expressive priors for solving ill-posed inverse problems. However, diffusion-based reconstruction relies on Stochastic Differential Equations (SDEs) for forward diffusion and reverse denoising, where such stochasticity can interfere with repeated data consistency corrections in CT reconstruction. Since CT reconstruction is often time-critical in clinical and interventional scenarios, improving reconstruction efficiency is essential. In contrast, Flow Matching (FM) models sampling as a deterministic Ordinary Differential Equation (ODE), yielding smooth trajectories without stochastic noise injection. This deterministic formulation is naturally compatible with repeated data consistency operations. Furthermore, we observe that FM-predicted velocity fields exhibit strong correlations across adjacent steps. Motivated by this, we propose an FM-based CT reconstruction framework (FMCT) and an efficient variant (EFMCT) that reuses previously predicted velocity fields over consecutive steps to substantially reduce the number of Neural network Function Evaluations (NFEs), thereby improving inference efficiency. We provide theoretical analysis showing that the error introduced by velocity reuse is bounded when combined with data consistency operations. Extensive experiments demonstrate that FMCT/EFMCT achieve competitive reconstruction quality while significantly improving computational efficiency compared with diffusion-based methods. The codebase is open-sourced at https://github.com/EFMCT/EFMCT.
Jiayang Shi, Lincen Yang, Zhong Li +3
Centrum Wiskunde en Informatica · LIACS, Leiden University · Great Bay University +1