Prior-Guided Physics-Informed Deep Learning for Fast 4D MRI Reconstruction
Authors: Mojtaba Safari, Parag J. Parikh, Shansong Wang, Mingzhe Hu, Philip Connell, Stanley L. Liauw, Ralph R. Weichselbaum, Zhen Tian, Xiaofeng Yang
Organizations: Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA · Department of Radiation Oncology, Henry Ford Health, Detroit, Michigan, USA
Background and purpose: We propose PGPI4MR (prior-guided, physics-informed 4D MRI reconstruction), a deep learning method that leverages information from other respiratory phases or a setup scan, the first 30 s of the same scan, to improve 4D MRI reconstruction without extra scan time. Materials and methods: PGPI4MR is an unrolled network that reconstructs 16 respiratory phases from 60, 30, or 15 s of data, and single 3D volumes, each from 0.29-0.49 s of data, using a setup-scan image. The networks were trained once, on other volunteers, and were not retrained for new volunteers. We evaluated PGPI4MR on measured k-space from three datasets: free-breathing MR-linac (1.5T) and abdominal (3T) scans, and thigh scans with cuff compression (1.5T). All methods, including comparative models, received identical samples and were scored similarly. Results: In vivo, PGPI4MR had the lowest held-out error for 4D MRI at every scan time, and its 16 phases were ready a median of 7.5-13 s after the scan on the 2 GPUs tested. Coupling the 16 respiratory phases reduced the error at 60 s by 9.1% compared with reconstructing each phase alone. For sub-second MRI, the error at 0.49 s was 15.5% lower than that of the best comparison method, classical reconstruction toward the same prior, and on the faster GPU, every 0.29 s window was acquired and reconstructed within 0.5 s. Without retraining, PGPI4MR matched the best methods on the 3T abdominal scans but not on the thigh scans; after retraining on thigh data, it reduced the held-out error by 6.6% relative to the best comparison method. Conclusion: Prior information allowed PGPI4MR to reconstruct 4D and sub-second MRI more accurately than compressed sensing and a published 4D network, within seconds and without extra scan time. This could support 4D MRI for treatment planning and 3D motion monitoring during MR-guided radiotherapy.
Figures & tables
In vivo: held-out error
Simulation: NRMSE
Question
Applied to
Tested
Compared with
Analysis
% lower
95% CI
% lower
95% CI
Coupling the phases (4D MRI)
PGPI4MR, 60 s †
Phases coupled (4D)
Each phase alone (3D)
Primary
9.1 ∗
8.1 to 10.2
28.8 ∗
24.9 to 32.5
PGPI4MR, 30 s
Phases coupled (4D)
Each phase alone (3D)
Registered
9.9 ∗
8.9 to 10.8
35.9 ∗
32.4 to 39.3
PGPI4MR, 15 s
Phases coupled (4D)
Each phase alone (3D)
Registered
7.8 ∗
6.2 to 9.4
33.2 ∗
29.6 to 36.7
No prior information
4D MRI, 60 s
3D network
BART
Exploratory
− 7.3 ∗
− 9.2 to − 5.4
− 14.2 ∗
− 19.6 to − 9.1
Sub-second MRI
PGPI4MR without prior
BART
Exploratory
− 8.2 ∗
− 13.1 to − 3.5
− 8.6 ∗
− 16.5 to − 1.3
Table 2: Ablation: the effect of each design choice in vivo (MR-linac) and in physics-based simulation. Each value is the percentage by which the tested setting lowered the error of the setting it is compared with (negative, a higher error), with its 95% confidence interval.
Figure 2Figure 3Figure 4Figure 5
Figure S1: Organs below the heart in sub-second 3D MRI of one volunteer (axial slice 93 mm below the liver dome). (A) In-vivo data, the first 0.29, 0.39, and 0.49 s windows after the setup scan (top to bottom); the box on the MOTUS reference marks the region enlarged in B. (B) The boxed region. Red arrows, the pancreas, identified visually on the MOTUS reference; amber arrows, the aorta, from the contour provided with the data. The arrows mark the same positions in every column of a row. The MOTUS reference is shown for anatomy only.
Figure S2: Held-out k-space error against image NRMSE on the UMC thigh data (real k-space with fully sampled references; 0.69 s windows). Each point is the mean of one method over the 9 volunteers. Open circles, methods without a prior; filled circles, methods given the setup prior. Networks not labeled as trained on UMC volunteers were applied zero-shot. E2E-VarNet lies outside the plotted range (Table S4 ).
60 s
30 s
15 s
Data
Volunteer
ZF
CG-S
BART
3D
4D
ZF
CG-S
BART
3D
4D
ZF
CG-S
BART
3D
4D
4D MRI, real data: held-out k-space error ↓
1
1.84
1.59
1.20
1.35
1.24
1.90
1.70
1.34
1.43
1.31
1.93
1.76
1.50
1.52
1.45
2
1.83
1.61
1.34
1.40
1.26
1.88
1.68
1.46
1.48
1.35
1.94
1.79
1.62
1.60
1.53
3
1.85
1.60
1.28
1.37
1.27
1.92
1.75
1.48
1.52
1.37
1.97
1.88
1.65
1.66
1.50
4
1.84
1.64
1.33
1.40
1.26
1.92
1.77
1.48
1.50
1.33
1.95
1.82
1.60
1.62
1.50
5
1.84
1.60
1.27
1.38
1.27
1.91
1.71
1.48
1.53
1.41
1.95
1.80
1.61
1.61
1.53
Table S1: Per-volunteer results of every method. Arrows mark the better direction; bold, best; underlined, second best.
MODEST, 4D MRI
Data
Volunteer
60 s (simulation: 61 s)
30 s
15 s
Real data: held-out k-space error ↓
6
1.86
1.85
1.88
9
1.94
1.93
1.80
10
1.90
1.89
1.81
Simulation: NRMSE ↓
6
0.234
0.287
0.471
9
0.300
0.379
0.608
Table S1. Per-volunteer results of every method (continued)
Simulation
Motion
Metric
Method
0.49 s
0.39 s
0.29 s
Normal breathing
NRMSE ↓
CG-SENSE
0.731
0.766
0.799
Unmoved setup prior alone
0.320
0.319
0.313
PGPI4MR, unmoved prior
0.261
0.267
0.284
Prior moved along the registration-based field, alone
0.310
0.309
0.308
PGPI4MR, prior with registration-based field
0.251
0.258
0.277
Table S1. Results of sub-second MRI in physics-based simulation under 3 kinds of motion (continued). Values are means over 10 volunteers; bold, best; underlined, second best.
60 s
30 s
15 s
Task
Comparison
Unit
% lower
95% CI
% lower
95% CI
% lower
95% CI
4D MRI
PGPI4MR (4D, phases coupled) vs PGPI4MR without coupling (3D), primary end point
Volunteers ( n = 10)
9.1 ∗
8.1 to 10.2
9.9 ∗
8.9 to 10.8
7.8 ∗
6.2 to 9.4
Folds ( n = 5)
9.1 ∗
7.8 to 10.4
9.9 ∗
8.9 to 10.8
7.8 ∗
6.4 to 9.3
Without the 2 development volunteers ( n = 8)
9.0 ∗
7.7 to 10.3
9.7 ∗
8.6 to 10.9
7.6 ∗
5.5 to 9.7
PGPI4MR (4D, phases coupled) vs BART
Volunteers ( n = 10)
2.5 ∗
0.1 to 4.9
6.5 ∗
5.0 to 8.0
5.9 ∗
4.5 to 7.3
Folds ( n = 5)
2.5
− 0.7 to 5.6
6.5 ∗
4.3 to 8.6
5.9 ∗
3.9 to 7.9
Table S2: Sensitivity analyses in vivo: the main comparisons with the fold as the unit and without the development volunteers. Each value is the percentage by which the first method lowered the held-out k-space error of the second (negative, a higher error), with its 95% confidence interval.
Compute, ms ↓ : median / 95th percentile
Latency of the slowest window, s ↓ : from the middle / from the start
Window
Method
Ada
Blackwell
Ada
Blackwell
Windows (volunteers)
MR-linac, 3 frames (0.29 s)
Zero-filled
< 1 / < 1
< 1 / < 1
0.150 / 0.297
0.149 / 0.297
1,015 (10)
CG-SENSE
31 / 39
20 / 22
0.190 / 0.338
0.170 / 0.318
1,015 (10)
BART
673 / 705
586 / 660
0.870 / 1.018
0.825 / 0.973
405 (4)
E2E-VarNet
204 / 288
132 / 173
0.462 / 0.610
0.321 / 0.469
1,015 (10)
PGPI4MR without prior
145 / 169
109 / 130
0.318 / 0.465
0.277 / 0.425
1,015 (10)
Table S3: Compute time and image latency. Values are the compute time per window or volume and the latency of the slowest window, with the statistic given in each header.
Stanford abdomen, 6 scans with normal breathing:
held-out k-space error ↓
Scan
ZF
CG-S
BART
E2E
NP
CGP
PGPI4MR
101
1.92
1.35
1.15
2.56
1.67
1.17
1.18
102
1.93
1.68
1.45
2.44
1.92
1.42
1.41
104
1.95
1.75
1.45
2.59
1.85
1.46
1.46
106
1.92
1.58
1.29
2.47
1.67
1.30
1.31
Table S4: External data sets. Values are the held-out k-space error of each Stanford scan, correlations ( r ) with the scanner navigator, and mean ± standard deviation over the UMC volunteers. Paired effects are the percentage by which the first method lowered the error of the second. Arrows mark the better direction; bold, best; underlined, second best.
UMC thigh, 9 volunteers, 0.69 s windows (exploratory)
CG toward the
PGPI4MR, trained on
Paired result
Scan
Score
moved prior
other UMC volunteers
% lower
95% CI
Dynamic scan
Held-out k-space error ↓
1.15 ± 0.04
1.08 ± 0.06
6.6 ∗
3.8 to 9.3
Image NRMSE, all windows ↓
0.267 ± 0.02
0.256 ± 0.03
4.4
− 0.4 to 9.0
Image NRMSE, 80 mm Hg ↓
0.281 ± 0.03
0.269 ± 0.03
4.4 ∗
0.2 to 8.3
Shading-insensitive NRMSE ↓
0.356 ± 0.04
0.328 ± 0.04
8.1 ∗
4.0 to 12.0
Table S4. External data sets (continued)
Comparison (first vs second)
Data
n
Effect (95% CI)
P
Table 1: the best value of a column against the second best
PGPI4MR (4D) vs BART, 60 s
Real
10
2.52 (0.06 to 4.91)
0.045
PGPI4MR (4D) vs BART, 30 s
Real
10
6.49 (4.95 to 8.01)
6.5×10−6
PGPI4MR (4D) vs BART, 15 s
Real
10
5.92 (4.51 to 7.30)
6.4×10−6
PGPI4MR (4D) vs BART, 61 s
Simulated
10
18.67 (16.86 to 20.44)
5.3×10−9
PGPI4MR (4D) vs 3D network, 30 s
Simulated
10
35.95 (32.36 to 39.35)
1.8×10−8
Table S5: Exact P values of the comparisons in Tables 1, 2, S2, and S4 and of the registered analyses.
Serial MRI is routinely performed in current clinical practice for disease surveillance and applications such as MR-guided radiotherapy. However, long MRI scan times increase costs and limit patient access, making accelerated image reconstruction of broad clinical importance. Previous scans of the same patient contain image information highly correlated with the current exam, yet existing reconstruction methods rarely exploit this information. Major challenges in the effective use of prior scans include inter-visit misregistration, pathological changes, and image phase variation. In this work, we propose Longitudinal Guided Group Sparsity (LGGS), an interpretable deep-learning framework for longitudinally informed MRI reconstruction derived from classical group sparsity. The prior scan is incorporated into the reconstruction regularizer through learned phase-invariant circulant attention that identifies corresponding features across exams. This formulation provides robustness to local in-plane and through-plane misregistration as well as inter-visit phase variation. Across acceleration rates, LGGS achieves state-of-the-art performance on longitudinal T1-weighted multi-coil brain MRI data, outperforming the evaluated unguided and longitudinally informed reconstruction methods, with substantially fewer learned parameters. We further evaluated the contributions of multiple guide slices and phase-invariant similarity and demonstrated that accelerated prior images can also serve as effective guides with minimal degradation in reconstruction quality.
Nikola Janjušević, Hersh Chandarana, Yao Wang +1
Radiology Department, New York University Grossman School of Medicine, New York, NY 10016 USA · Electrical and Computer Engineering Department, New York University Tandon School of Engineering, Brooklyn, NY 11201 USA
MRI reconstruction from undersampled k-space measurements is an ill-posed inverse problem. Physics-driven deep learning (PD-DL) methods have shown strong performance for this task by combining the MRI forward model with learned image regularization within algorithm-unrolling frameworks. However, most existing PD-DL methods reconstruct complex-valued images directly, thereby implicitly coupling magnitude and phase within a single learned representation. This coupled regularization may be suboptimal in reconstruction settings where accurate phase modeling plays an important role, such as partial Fourier (PF) imaging, where recovery of the omitted asymmetric k-space measurements depends on the underlying image phase. In such scenarios, explicit modeling of magnitude and phase as separate components may reduce the reliance on externally estimated or predefined phase information. To this end, we propose UMPIRE-Net (Unrolled Magnitude-Phase In REgularization Network), a PD-DL method that introduces separate learned regularizers for magnitude and phase components, together with a novel data-fidelity formulation that enforces measurements consistency. We evaluate UMPIRE-Net for accelerated MRI with PF across different datasets and acceleration factors. Experimental results demonstrate that our proposed method improves reconstruction quality compared with a conventional complex-valued PD-DL baseline, yielding sharper images and reduced artifacts. Code available at: https://github.com/MahdiSaberii/UMPIRE-Net
Mahdi Saberi, Toygan Kiliç, Mehmet Akçakaya
Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA · Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
MRI reconstruction methods for undersampled k-space data naturally utilize complex-valued measurements. Parallel developments in sparse phase retrieval have shown that magnitude-only measurements may provide complementary information for signal recovery. However, their use in MRI reconstruction remains largely unexplored, due to lack of practical settings where informative magnitude measurements can be obtained without additional scan time. In this work, we investigate the use of auxiliary k-space magnitude information for accelerated steady-state dynamic MRI reconstruction, and demonstrate strong consistency of k-space magnitudes across time-frames. Building on this observation, we propose C+Mag, a magnitude-informed physics-driven deep learning reconstruction method. The proposed method employs an ADMM-based unrolling framework with a novel magnitude-aware data-fidelity formulation, where quadratically smoothed optimization and momentum-based updates are introduced to address the non-differentiability and non-convexity of the magnitude constraints. Experiments on retrospectively undersampled cine MRI and phase-contrast flow MRI datasets, as well as prospectively undersampled real-time cine MRI acquisitions, demonstrate improved artifact suppression, sharper anatomical recovery, and better preservation of phase information compared to conventional PD-DL methods, which is further supported through blinded expert reader evaluations.
Mahdi Saberi, Yaşar Utku Alçalar, Merve Gülle +2
Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA · Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA · Department of Medicine (Cardiology), University of Minnesota, Minneapolis, MN, USA