Physics-based simulation unlocks scaling laws for generalizable MRI reconstruction
Authors: Hong Shang, Zijian Zhan, Sen Jia, Xiang Fan, Jintao Meng, Yang Zhou, Xiang Zhou, Tianyu He, Xinran Chen, Qiaoling Zeng, Keyan Yu, Jianhua Yao, Dong Liang
Organizations: Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China. · School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China. · Department of Medical Imaging, Peking University Shenzhen Hospital, Shenzhen, China. · State Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences, Shenzhen, China. · AI for Life Sciences Lab, Tencent, Shenzhen, China.
Deep-learning-based magnetic resonance imaging (MRI) reconstruction holds great promise for accelerating acquisitions, yet its clinical translation remains bottlenecked by poor generalizability across unseen scanners, populations, anatomies, and protocols. Gathering sufficiently diverse clinical datasets to overcome this bottleneck is hindered by high acquisition costs, privacy restrictions, and institutional biases. Here, we introduce Scalable Inverse Model (SIM), a framework that systematically co-designs physics-simulated training data, model architecture, and learning paradigms around scalability--leveraging compute scaling to overcome historical data bottlenecks and achieve robust zero-shot MRI reconstruction. Across 14 public and in-house datasets--spanning seven international centers, five vendors, field strengths of 0.3-7 T, over ten anatomical regions, more than ten acquisition protocols, downstream quantitative and molecular tasks, and a 150-scan blinded radiologist study--SIM demonstrates robust zero-shot generalizability, recovering fine anatomical details while suppressing aliasing artifacts where conventional deep-learning methods fail. Attention-map analyses confirm that SIM learns physically explainable, dynamically adaptive reconstruction primitives rather than anatomy-specific priors. SIM establishes a scalable, privacy-preserving, and democratized infrastructure for universal MRI reconstruction, providing a blueprint for generalizable AI across inverse problems in medical imaging.
Figures & tables
Figure 1: Overview of SIM, a compute-scalable framework for zero-shot MRI reconstruction. a, The physics-simulated data engine transforms arbitrary images into paired training data comprising undersampled multi-coil k -space measurements ( X ) and ground-truth targets ( Y ). This pipeline strictly obeys multi-coil MRI forward physics to align simulated and real measurement characteristics, effectively closing the simulation-to-real gap. Simultaneously, it expands image and acquisition diversity to overcome historical data bottlenecks, enabling models to learn fundamental inverse mappings. b, SIM adopts the multi-channel Vision Transformer (mcViT)—a high-capacity architecture with minimal MRI-specific inductive bias—to absorb diverse simulated data and learn robust reconstruction priors at scale, replacing hand-crafted architectures designed for data-scarce regimes. c, Pairing large-scale simulated data with high-capacity architecture reveals power-law scaling relationships with dataset size, model size, and patch count, mirroring scaling behaviors in natural language and vision. Crucially, these scaling gains transfer directly to real clinical data, confirming that the simulation engine effectively bridges the simulation-to-real gap. d, Comprehensive zero-shot evaluation across public and in-house datasets—spanning diverse scanner hardware, populations, anatomies, protocols, downstream quantitative and molecular imaging tasks, and a clinical reader study—demonstrates that SIM achieves vastly superior generalizability over traditional real-data learning paradigms.
Figure 2: Data simulation pipeline, feature representation, and ablation study. a, Data simulation pipeline. The workflow transforms arbitrary natural images into paired training data comprising undersampled multi-coil k -space measurements and ground-truth targets. The physical mapping between measurement and target strictly obeys multi-coil MRI forward physics. FT, Fourier transform; iFT, inverse Fourier transform; rss, root-sum-of-squares. b, Data visualization in a shared representation space. While real datasets show a clustered distribution with limited diversity, simulated data display diverse local contrast patterns relevant to clinical imaging at the patch level. Each dot represents a 14×14 image patch in the DINOv2 feature space Oquab et al. (2023) after dimension reduction using principal component analysis and linear discriminant analysis to two primary discriminant axes, color-coded by dataset source. LDA, linear discriminant analysis. c, Ablation study demonstrating the effectiveness of key simulated data design features in closing the simulation-to-real gap, as well as the impact of model architectural choices and learning objectives engineered to leverage simulated training data. Error bars represent mean ± s.d. Retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 3: Scaling laws in reconstruction. a, In-distribution scaling for simulated data. Reconstruction performance improves smoothly as dataset and model size increase in tandem. Scaling model size yields sustained performance gains when not bottlenecked by data diversity, analogous to scaling laws observed in LLMs Kaplan et al. (2020) . b, In-distribution scaling for real fastMRI. Scaling model size fails due to pronounced overfitting (Extended Data Fig. 9 b) from inadequate data diversity; despite comparable dataset size to simulated data, real fastMRI exhibits high inter-dataset similarity. c, Transfer scaling comparing training data. Scaling behavior learned on simulated data transfers effectively to diverse real-world datasets, successfully bridging the sim-to-real gap. Real fastMRI alone shows no scaling benefit and transfers poorly to OOD test sets; combining fastMRI with simulated data restores OOD scaling, demonstrating that high-diversity simulated data are essential to unlock scaling laws in MRI reconstruction. d, Transfer scaling comparing model architectures. When trained on 1.1 million simulated samples, conventional architectures designed for small-data regimes with hard-coded domain priors (such as E2EVarNet) are outperformed on OOD test sets by mcViT—a generic Vision Transformer with minimal inductive bias—underscoring the need to co-design data and models for scaling regimes. Experiments in a–c utilize mcViT. Across a–d, retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor. ID, in-distribution.
Figure 4: Zero-shot reconstruction. a, Quantitative comparison. Both SIM and SIM-Zero (trained exclusively on simulated data) outperform all competing baselines across 8 out-of-distribution (OOD) benchmarks, with SIM exceeding the top baseline by 1.06–3.95 dB in PSNR and 0.008–0.045 in SSIM, and SIM-Zero performing only marginally below SIM. Error bars indicate mean ± s.d. Full comparisons against 9 state-of-the-art methods are in Extended Data Fig. 11 . ID, in-distribution. b, Qualitative comparison. Representative reconstructions, error maps, and reference images. SIM recovers fine anatomical detail matching ground truth without perceptible aliasing artifacts (see Extended Data Fig. 12 for additional visual comparisons). XGY, Xingaoyi. c, Cross-protocol transfer. Models were trained on fastMRI Brain data partitioned by protocol (T1, contrast-enhanced T1 [CE], T2, FLAIR). E2EVarNet trained on real data exhibits a diagonal performance, degrading on unobserved protocols due to overfitting. Conversely, SIM displays uniform performance across protocols due to domain-agnostic simulated data. Across a–c, retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 5: Zero-shot quantitative MRI. a,b, Representative single-echo reconstructions, derived T2∗ maps, and corresponding error maps evaluated on the public AHEAD dataset (MP2RAGEME sequence, Philips 7 T) (a) and an in-house cohort (MULTIPLEX sequence, UIH 3 T) (b). SIM’s zero-shot capability enables reliable downstream parametric mapping. Skull stripping is applied, and display windowing and leveling are adjusted per echo image and matched between reconstructions and reference images. Retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor. c, Quantitative comparison of T2∗ mapping across all slices evaluated by mean absolute error (MAE) at effective acceleration factors of 3.2× , 4.2× , and 5.1× . SIM consistently outperforms baselines across sequences and acceleration factors. Error bars represent mean ± s.d. AF, acceleration factor.
Figure 6: Zero-shot molecular MRI. a, Representative SIM reconstructions of dynamic CEST images (shown at offset frequencies of 1.1 ppm and 3.6 ppm) and derived dynamic maps of PCr and tCr∗ (a weighted sum of PCr and Cr Chen et al. (2026) ), evaluated on an in-house dataset monitoring calf-muscle recovery using an in-scanner exercise protocol Chen et al. (2026) (UIH 5 T). b, Dynamic recovery curves within the calf-muscle region of interest (ROI). SIM achieves image quality virtually identical to the reference across all dynamic timepoints, enabling faithful preservation of dynamic physiological responses. For b, data points represent mean values, and error bars indicate s.d. across all pixels within the calf-muscle ROI. For a and b, retrospective equispaced undersampling is applied at a 5.1× effective acceleration factor. PCr, phosphocreatine; Cr, creatine; τ , metabolic recovery time constant.
Figure 7: Zero-shot clinical applications. a, Representative reconstructions of SIM and E2EVarNet at effective acceleration factors of 3.2× , 4.2× , and 5.1× for an in-house Alzheimer’s disease cohort with three contrasts: T1 -weighted, T2 -weighted, and T2 -FLAIR. Red arrows indicate residual aliasing artifacts. b,c, Quantitative evaluation in terms of PSNR (b) and SSIM (c). Error bars represent mean ± s.d. d–f, Blinded reader study in which 1,050 MRI volumes were evaluated independently by two board-certified radiologists. d, Radiologist ratings of overall image quality evaluated on a 5-point Likert scale ( 5=excellent , 1=unacceptable ); each dot represents an individual rating. e, Radiologist ratings of clinical usability evaluated on a 3-point categorical scale (diagnostic, screening-only, non-diagnostic). f, Inter-reader agreement for image quality and clinical usability ratings. Statistical significance was assessed using two-sided paired Student’s t -tests for quantitative metrics (b,c) and two-sided clustered Wilcoxon signed-rank tests for radiologist ratings (d,e); significance levels are denoted as **** P<0.0001 . Inter-reader agreement was quantified using quadratically weighted Cohen’s κ for image quality ratings and linearly weighted Cohen’s κ for clinical usability. For d and e, ratings from two radiologists are pooled for visualization. AF, acceleration factor; VN, E2EVarNet; x3, x4, and x5, effective acceleration factors of 3.2× , 4.2× , and 5.1× , respectively.
Figure 8: Visualization of SIM attention maps. a, Attention maps of individual heads across layers, demonstrating that SIM intrinsically captures generic image structures and spatial relationships tied to undersampling physics via large-scale domain-agnostic training, thereby achieving physically explainable and broadly generalizable reconstruction. b, Attention maps aggregated across all layers and heads, showing that SIM dynamically adapts its attention to aliasing source regions as changes in acceleration alter the spatial aliasing distance. Two example query patches are shown under effective acceleration factors of 3.2× and 4.2× . For a and b, arrows and boxes on input images indicate query patches. AF, acceleration factor.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Training Curves. a, b, Training and validation loss for a series of mcViT models trained on simulated data (a) and real fastMRI (b) across various dataset and model sizes. For both data, increasing model capacity leads to severe overfitting when data diversity is inadequate, whereas increasing dataset size reduces the training–validation gap. Scaling simulated training data to 50,000 samples prevents overfitting for models up to 567 million parameters, whereas severe overfitting persists for 50,000 real fastMRI samples, demonstrating the substantially higher diversity of simulated data at comparable dataset sizes required for continuous benefits from larger models. Valid, validation. For a and b, retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 10: Scaling patch size. a, Illustration of patch size ( p ) and its inverse relationship to patch count. b, Validation loss for a series of mcViT models trained on simulated data across various patch sizes, ranging from 32 to 5. Reducing patch size—thereby increasing the Transformer sequence length and computational cost—yields consistent performance gains. c, Transfer scaling. Patch size scaling behavior learned on simulated data transfers effectively to diverse real-world datasets, analogous to model size scaling. ID, in-distribution. For b and c, retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 11: Full quantitative comparison of zero-shot reconstruction. a, Quantitative comparison across in-distribution (ID) and out-of-distribution (OOD) benchmarks in terms of PSNR and SSIM. Both SIM and SIM-Zero (trained exclusively on simulated data) outperform all 9 competing baselines across 8 OOD benchmarks. b, Runtime comparison measured on 15-coil fastMRI knee data using a single NVIDIA RTX 5090 GPU. SIM operates via a single feed-forward inference step, achieving low latency compared to diffusion-based and untrained approaches that require multi-step iteration per sample. In a and b, error bars represent mean ± s.d., and retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 12: Visual comparison of zero-shot reconstruction. Representative reconstructions from SIM, SIM-Zero (trained exclusively on simulated data), and 9 baseline methods evaluated on examples from fastMRI Brain, CC-359 Axial, CC-359 Sagittal, M4Raw GRE, AHEAD Echo1, and Stanford 2D datasets. PSNR (dB) and SSIM (%) values for individual slices are indicated in the top-left corner of each reconstruction. Retrospective equispaced undersampling is applied at a 3.2× effective acceleration factor.
Figure 13: Zero-shot reconstruction across broad anatomies and contrasts. a, Broad anatomies. Representative SIM reconstructions on an in-house cohort (Multi-Anatomy, UIH 3 T) and a public cohort (Stanford 2D, GE 3 T). The cardiac panel is cropped and enlarged around the heart region. b, Broad contrasts. Representative SIM reconstructions on in-house cohorts (Multi-Contrast, UIH 3 T; DCE, Siemens 3 T). Red arrows indicate lesions. In a and b, display windowing and leveling are matched between SIM reconstructions and reference images, and retrospective equispaced undersampling is applied at a 4.2× effective acceleration factor. Details for each dataset are provided in Extended Data Table 1 .
Figure 14: Training strategies for multi-acceleration support. a, Quantitative comparison of three training strategies evaluated on fastMRI Brain and Knee test sets: bespoke training (single fixed acceleration factor per model), all-mixed training (randomly mixed continuous acceleration factors from 2 to 8), and even-mixed training (randomly mixed discrete acceleration factors of 2, 4, 6, and 8). Mixed-acceleration training achieves performance comparable to bespoke training provided the target acceleration factor is included during training. All training runs used simulated data to train an mcViT model under matched settings, varying only in acceleration factor configurations. Error bars represent mean ± s.d. b, Visualization of equispaced undersampling masks with an 8% fully sampled k -space center and nominal acceleration factors (strides) of 2 to 8, yielding effective acceleration factors of 1.9 to 5.1. AF, acceleration factor.Training strategies for multi-acceleration support. a, Quantitative comparison of three training strategies evaluated on fastMRI Brain and Knee test sets: bespoke training (single fixed acceleration factor per model), all-mixed training (randomly mixed continuous acceleration factors from 2 to 8), and even-mixed training (randomly mixed discrete acceleration factors of 2, 4, 6, and 8). Mixed-acceleration training achieves performance comparable to bespoke training provided the target acceleration factor is included during training. All training runs used simulated data to train an mcViT model under matched settings, varying only in acceleration factor configurations. Error bars represent mean ± s.d. b, Visualization of equispaced undersampling masks with an 8% fully sampled k -space center and nominal acceleration factors (strides) of 2 to 8, yielding effective acceleration factors of 1.9 to 5.1. AF, acceleration factor.
Image reconstruction recovers images, volumes, or scenes from incomplete, noisy, indirect, or compressed measurements using physical models and learned priors.
Zero-Shot Self-Supervised Learning (ZS-SSL) has emerged as a promising paradigm for accelerated Magnetic Resonance Imaging (MRI) reconstruction, eliminating the reliance on fully-sampled external datasets. However, learning solely from a single under-sampled scan suffers from supervision scarcity and optimization instability, often leading to overfitting or artifacts. To address these challenges, we propose a robust physics-driven ZS-SSL framework that synergizes physical consistency with image-domain non-local priors. Our method introduces three core innovations: (1) a Coil Sensitivity Map (CSM)-Guided Dynamic Repository, which stabilizes the training trajectory by filtering physically inconsistent artifacts based on coil sensitivity constraints; (2) a SPIRiT-based regularization, which enforces k-space self-consistency via a learned correlation kernel and stochastic masking; (3) a Non-Local Self-Similarity (NSS) Pixel Bank, which leverages the high-fidelity reference established by the former modules to explicitly mine non-local anatomical similarities, thereby augmenting supervision in the image domain. Extensive experiments on the FastMRI dataset demonstrate that our approach achieves state-of-the-art performance, particularly under high acceleration factors, effectively bridging the gap between zero-shot learning and supervised methods. The code is available at https://github.com/Zolento/NS-SSL.
Lingtong Zhang, Wenlei Li, Mu He +2
School of Information Science and Technology, University of Science and Technology of China, Hefei, China
MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions. This ambiguity becomes more severe under high acceleration, where pixel-domain continuous predictors tend to average over feasible reconstructions and suppress high-frequency anatomy. We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction. Leveraging discrete priors proven effective in visual autoregressive modeling, our method restricts the solution to compact sequences of codebook tokens, enabling sharp reconstructions even from extremely sparse measurements. This discrete autoregressive formulation also aligns naturally with modern large language model post-training techniques. Building on this observation, we introduce on-policy privileged information distillation for visual autoregressive modeling, where a teacher is provided training only privileged context that is unavailable at inference, in our case fully sampled acquisitions, and supervises a student trained on its own rollouts, leading to consistent reconstruction gains. Through extensive experiments on the fastMRI benchmark, we show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling. Project website is here.
High-resolution Magnetic Resonance Imaging (MRI) is vital for clinical diagnosis but limited by long acquisition times and motion artifacts. Super-resolution (SR) reconstructs low-resolution scans into high-resolution images, yet existing methods are mutually constrained: paired-data methods achieve efficiency only by relying on costly aligned datasets, while implicit neural representation approaches avoid such data needs at the expense of heavy computation. We propose a zero-shot MRI SR framework using explicit Gaussian representation to balance data requirements and efficiency. MRI-tailored Gaussian parameters embed tissue physical properties, reducing learnable parameters while preserving MR signal fidelity. A physics-grounded volume rendering strategy models MRI signal formation via normalized Gaussian aggregation. Additionally, a brick-based order-independent rasterization scheme enables highly parallel 3D computation, lowering training and inference costs. Experiments on two public MRI datasets show superior reconstruction quality and efficiency, demonstrating the method's potential for clinical MRI SR.
Shuting Liu, Lei Zhang, Wei Huang +2
School of Artificial Intelligence, Sichuan University, Chengdu 610065, China · Institute of High Performance Computing, A*STAR, Singapore 138632