Organizations: School of Mathematics and Statistics, Xi’an Jiaotong University, No.28, Xianning West Road, Xi’an, 710049, Shaanxi, China. · Research Center for Intelligent Medical Equipment and Devices (IMED), Xi’an Jiaotong University, No.28, Xianning West Road, Xi’an, 710049, Shaanxi, China. · Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi’an Jiaotong University, No.28, Xianning West Road, Xi’an, 710049, Shaanxi, China.
Magnetic Resonance Imaging (MRI) is a cornerstone in medicine and healthcare but suffers from long acquisition times. Traditional accelerated MRI methods optimize for generic image quality, lacking adaptability for specific clinical tasks. To address this, we introduce PASS (Personalized, Anomaly-aware Sampling and reconStruction), an intelligent MRI framework that leverages a Vision-Language Model (VLM) to guide a deep unrolling network for task-oriented, fast imaging. PASS dynamically personalizes the imaging pipeline through three core contributions: (1) a deep unrolled reconstruction network derived from a physics-based MRI model; (2) a sampling module that generates patient-specific k-space trajectories; and (3) an anomaly-aware prior, extracted from a pretrained VLM, which steers both sampling and reconstruction toward clinically relevant regions. By integrating the high-level clinical reasoning of a VLM with an interpretable, physics-aware network, PASS achieves superior image quality across diverse anatomies, contrasts, anomalies, and acceleration factors. This enhancement directly translates to improvements in downstream diagnostic tasks, including fine-grained anomaly detection, localization, and diagnosis.
Figures & tables
Figure 1: PASS is a closed-loop system for personalized MRI. It introduces a paradigm where a fine-tuned VLM ( NetAD ) guides the entire imaging pipeline from acquisition to reconstruction. As detailed in (a), the system overview, anomaly attention maps from the VLM direct a sampling network to acquire patient-specific k -space data, while simultaneously conditioning a deep unrolling reconstructor to enhance clinical features. The VLM’s adaptation process (b) and the specialized architectures for sampling (c) and reconstruction (d) enable this co-design, ensuring the output is optimized for diagnostic relevance. Central k -space regions (autocalibration signals, ACS) provide a low-resolution prior, while image- and pixel-level adapters (ILA/PLA) enable precise VLM fine-tuning.
Figure 2: Quantitative comparison of reconstruction methods across datasets and acceleration factors. Left and right columns correspond to 4× and 8× acceleration, respectively. a–b, T1w brain, c–d, FLAIR brain, e–f, PD knee. PSNR and SSIM are reported for each method, with bars indicating mean values across the test set and error bars representing standard deviation.
Figure 3: Visual comparison of reconstructed images at 4× (a) and 8× (b) acceleration across different methods. Representative results for T1w brain, FLAIR brain, and PD knee datasets. Each panel shows the reconstructed image and its corresponding error map relative to the fully sampled reference. PASS leads to sharper anatomical details and reduced residuals, particularly within pathological regions.
Figure 4: Lesion-focused reconstruction evaluation. Comparison of reconstruction quality across methods for T1w brain images. Left and right columns correspond to 4× and 8× acceleration, respectively. a-b, PSNR for the five most frequent lesions in the cohort: Type 1, Posttreatment change; Type 2, Mass; Type 3, Craniotomy; Type 4, Enlarged ventricles; Type 5, Edema. c-d, PSNR for five less frequent lesions in the cohort: Type 1, Resection cavity; Type 2, Normal variant; Type 3, Extra-axial mass; Type 4, Motion artifact; Type 5, Paranasal sinus opacification. e-f, Overall lesion-focused metrics (LF-PSNR and LF-SSIM) across all annotated regions. PASS consistently achieves superior fidelity in pathological areas.
Figure 5: Comparative analysis of sampling strategies for T1w brain images. Four strategies are evaluated: “No-learn”, a fixed non-learnable mask, “LS-MoDL”, a model-based baseline that jointly optimizes a reconstruction network and a learnable sampling mask, “Global Learn”, a simplified variant of our framework employing LOUPE-based global sampling shared across all subjects without anomaly-aware sampling, and “Ours”, the full PASS model integrating adaptive anomaly-aware sampling. a–b, Quantitative reconstruction results at 4× and 8× , respectively, including PSNR, SSIM, LF-PSNR, and LF-SSIM. c–d, Visual comparisons of reconstructed images and corresponding sampling masks at 4× and 8× . Red arrows indicate regions where PASS adaptively increases sampling density in lesion-relevant k -space areas compared with other strategies.
Figure 6: Evaluation of downstream pathological tasks for T1w brain images. a, Anomaly detection AUC of a VLM-based detection model on reconstructed images generated by different methods under 4× and 8× , evaluated across all lesions collectively. b, Localization of pathological regions by the VLM-based detection model on PASS-reconstructed images. c–d, Multi-label classification accuracy for individual lesion types using a fine-tuned CLIP image encoder with an MLP classifier, at 4× (c) and 8× (d), respectively.
MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions. This ambiguity becomes more severe under high acceleration, where pixel-domain continuous predictors tend to average over feasible reconstructions and suppress high-frequency anatomy. We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction. Leveraging discrete priors proven effective in visual autoregressive modeling, our method restricts the solution to compact sequences of codebook tokens, enabling sharp reconstructions even from extremely sparse measurements. This discrete autoregressive formulation also aligns naturally with modern large language model post-training techniques. Building on this observation, we introduce on-policy privileged information distillation for visual autoregressive modeling, where a teacher is provided training only privileged context that is unavailable at inference, in our case fully sampled acquisitions, and supervises a student trained on its own rollouts, leading to consistent reconstruction gains. Through extensive experiments on the fastMRI benchmark, we show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling. Project website is here.
Recent advances in large-scale foundation models have enabled strong generalization across a wide range of clinical and research tasks in magnetic resonance imaging (MRI). However, existing vision-language MRI models remain limited in anatomy-specific localization and pathology-aware representation, particularly in real-world settings. We introduce A4-MR, a 3D vision-language foundation model trained on approximately 50,000 MRI studies comprising 491,000 series, roughly half of which are paired with radiology reports. A4-MR is designed to: (1) learn anatomy localization through large-scale pretraining with noisy labels; (2) align image and text representations at the anatomical level; and (3) learn distinct representations of healthy and abnormal anatomy. Across a diverse set of downstream tasks, A4-MR outperforms existing foundation models, including a +1.7 AUROC improvement over MedSigLIP for abnormality detection. Using a frozen encoder, A4-MR also surpasses an end-to-end task-specific MedNeXt model for knee segmentation by +1.4 Dice and performs comparably to the best task-specific model for abdominal organ segmentation on AMOS, within 0.4 Dice. Finally, without anatomical prompts, A4-MR localizes 3D anatomy comparably to MedSAM2 provided with ground-truth bounding boxes. Together, these results demonstrate that anatomy-level multimodal alignment and abnormality-aware representation learning can enable a single MRI foundation model to support localization, segmentation, and pathology-related tasks.
Noel D'souza, Zhijian Yang, Nishank Singla +14
GE Healthcare, Seattle, USA · GE Healthcare, Budapest, Hungary · Amazon, USA
MRI provides excellent soft-tissue contrast without ionizing radiation, but long acquisition times increase patient discomfort while also raising exam costs and limiting scanner throughput. A common approach to reduce scan time is to acquire fewer measurements, which yields an ill-posed linear inverse problem; recovering diagnostic-quality images therefore requires incorporating prior knowledge beyond the measured data. In follow-up exams, the most recent prior scan of a patient can provide a highly informative subject-specific context, but practical use is complicated by temporal changes (including pathology progression), misalignment between scans, and protocol drift across acquisitions. In this work, we introduce L-TGVN, a Longitudinal Trust-Guided Variational Network that leverages prior scans as side information to reconstruct the current scan from heavily undersampled measurements. Crucially, L-TGVN constrains the influence of prior scans to be consistent with the acquired measurements. Unlike many existing longitudinal reconstruction methods, it does not require explicit pre-registration between prior and current scans. It further accommodates differences in acquisition protocols across visits (e.g., changes in sequence parameters). We evaluate L-TGVN against matched-capacity baselines, including prior-guided methods and methods that do not use longitudinal priors, and observe consistent improvements in standard quantitative metrics together with better preservation of fine structures at challenging accelerations. Source code is available at github.com/sodicksonlab/L-TGVN.
Arda Atalık, Sumit Chopra, Daniel K. Sodickson
NYU Center for Data Science, NY, USA · Center for Advanced Imaging Innovation and Research (CAI²R), Department of Radiology, NYU Grossman School of Medicine, NY, USA · Courant Institute of Mathematical Sciences, NY, USA +1