cs.AIOct 30, 2026
SaveA generative foundation model for computational brain imaging
Organizations: Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA. · School of Computer Science and Engineering, Southeast University, Nanjing, China. · Department of Computer and Information Science, University of Macau, Macau, China. · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China. ·
Abstract
Computational brain imaging is constrained by a mismatch between increasingly diverse tasks and scarce, heterogeneous clinical data. Although existing brain foundation models transfer pretrained representations, their knowledge remains entangled within model-specific features and fixed architectures. This paper proposes BrainGFM, a generative foundation model that extends knowledge transfer beyond representation reuse through a reusable and reconfigurable brain-imaging prior. By pretraining on a corpus of about 1.3 million MRI and CT slices, BrainGFM supported task-aligned data synthesis for low-data predictive learning and task-adapted post-training for image reconstruction. Across 48 downstream tasks, BrainGFM improved segmentation and classification under limited supervision and achieved high-fidelity image enhancement across heterogeneous imaging conditions. Multicenter clinical validation involving 4,796 stroke patients across two internal and four external centers demonstrated its applicability to three clinically relevant tasks around stroke cohorts. Blinded reader assessments showed that BrainGFM-generated FLAIR images approached acquired FLAIR in overall diagnostic quality. In a human-AI collaboration study, BrainGFM assistance yielded a 14% relative improvement in mean radiologist stroke-subtyping accuracy. These findings support generative priors as a reusable foundation for heterogeneous computational brain-imaging applications.
Figures & tables
Figure 1: Overview of BrainGFM for Computational brain imaging. a. Heterogeneity in brain diseases, patient populations, clinical status, acquisition protocols, and clinical centers produces heterogeneous brain imaging distributions and limits the generalization of DL models. b. Our BrainGFM-1.3M comprises about 1.3 million axial slices spanning major MRI sequences and CT, providing a large-scale dataset for brain generative pretraining. c. Our BrainGFM learns a reusable brain-imaging prior from generative pretraining on our BrainGFM-1.3M dataset. d. Task-specific conditioning controls the pretrained generator where class labels are used for class-conditioned synthesis, segmentation masks for spatially conditioned synthesis, and source images for brain image reconstruction. These conditions are combined with task-specific adaptation as detailed in Fig. 6 b,c. e. Evaluation spans 48 downstream tasks, comprising 20 segmentation, 8 classification, and 20 image-enhancement tasks. f. Clinical validation in a multicenter stroke cohort assesses BrainGFM for TOAST stroke subtyping, lesion segmentation, and MRI sequence transformation, as well as its clinical utility in reader studies and radiologist-AI collaboration.
Figure 2: Task-aligned data synthesis enables low-data predictive learning. a. Overview of the task-aligned data synthesis framework. BrainGFM is adapted with few brain images and synthesizes large-scale task-aligned data to train downstream segmentation and classification models. b. Task-family-level performance. BrainGFM is compared with direct low-data training, the strongest competing low-data method, and full-data training references. BrainGFM achieves strong Dice and AUC across heterogeneous prediction targets. c. Performance across 20 segmentation and 8 classification tasks. BrainGFM maintains strong performance across heterogeneous datasets, imaging modalities, and prediction targets. d. Task robustness measured by mean and worst-task performance. BrainGFM achieves a favourable balance between overall and lower-bound performance, indicating its robustness and broad applicability. e. Amplifiability of low-data supervision. The lollipop plot shows BrainGFM’s performance gain compared with direct low-data training (with without BrainGFM), measured as Dice or AUC. BrainGFM achieves positive gains across all tasks, demonstrating its powerful amplifiability for the value of few supervision. Superscript denotes models trained using the full dataset, and superscript denotes brain-imaging foundation models.
Figure 3: Task-adapted post-training enables high-fidelity brain-image enhancement. a. Overview of the task-adapted post-training framework. BrainGFM is adapted with paired task-specific brain images and directly enhances non-ideal measurements to enhanced brain images. b. The radar plot shows PSNR across tasks, and the heat map reports BrainGFM’s rank for PSNR, SSIM, RMSE, and LPIPS. BrainGFM maintains strong performance across heterogeneous datasets, modalities, and enhancement targets. c. Representative qualitative comparisons. BrainGFM reduces image degradation while preserving anatomical structures and local details. d. Task-family-level performance. BrainGFM outperforms CAR and the strongest competitors across the three task families and four evaluation metrics. e. Robustness between mean and worst-task PSNR. BrainGFM achieves a favourable balance between overall and lower-bound performance. f. Amplifiability of task-adapted post-training. The lollipop plot reports BrainGFM’s PSNR gain over CAR as PSNR. Positive gains across all tasks demonstrate that task-adapted post-training amplifies the enhancement capability of the model without brain-image pretraining.
| Internal cohort | External cohort | ||
| Task | Method | ||
| UNet (few) | 58.0 | 50.8 | |
| 60 cases | (57.3-58.7) | (48.3-53.3) | |
| CAR | 69.6 | 67.2 | |
| (68.8-70.3) | (66.4-67.9) | ||
| MedSAM | 72.0 | 70.3 |
Table 1: Multicenter validation across stroke cohorts. Performance was evaluated on pooled eligible cases from the two internal centers and four external centers for MRI sequence transformation, lesion segmentation, and TOAST stroke subtyping.
Figure 4: Blinded reader study and human-AI collaboration study in stroke cohorts. a-b Workflow of the blinded reader study for acquired and DWI-synthesized FLAIR images and the human-AI collaboration study for ischemic stroke subtyping. c. Radiologist ratings across five image-quality criteria: overall diagnostic quality (ODQ), anatomical fidelity (AF), FLAIR contrast fidelity (CF), artifact- and hallucination-free quality (AHQ), and diagnostic confidence (DC). d. Proportion of reader assessments meeting all five image-quality criteria at increasing Likert-score thresholds. e. F1 score, one-versus-rest accuracy, and sensitivity for experts, AI-assisted experts, and BrainGFM across LAA, CE, and SAO. f. Relationship between recorded AI recommendation acceptance and changes in diagnostic accuracy among initially high-confidence assessments (confidence score 3). Solid and dashed contours represent the 50% and 95% regions of the bootstrap distributions. Error bars in panels c-e indicate 95% CIs.
Figure 5: Ablation study and model analysis of BrainGFM. a. Synthetic-data scaling across representative segmentation and classification tasks shows consistent performance gains followed by task-dependent saturation, demonstrating that task-aligned synthesis effectively amplifies limited supervision. b. Data scaling across three enhancement tasks shows progressive improvement with increasing post-training data, with task-dependent saturation indicating efficient adaptation from limited paired data. c. t-SNE analysis of real, BrainGFM-generated, and CAR-generated features shows greater overlap between BrainGFM and real-data distributions, supporting improved distribution alignment. d. Representative BrainGFM-generated images across modalities and sequences preserve modality-specific contrast, anatomical structure, and local image details, demonstrating multimodal generation fidelity. e. Intermediate image and mask predictions across successive feature scales reveal progressive formation of global structure and fine spatial details, illustrating the coarse-to-fine next-scale generation process.
Figure 6: BrainGFM architecture, generative pretraining and downstream adaptation. a. BrainGFM pretraining consists of brain-domain adaptation of a multi-scale visual tokenizer and next-scale autoregressive generative pretraining. A shared encoder ( ) and vector quantizer ( ) map brain images and segmentation masks into multi-scale discrete tokens using a shared visual codebook. The autoregressive Transformer learns the distribution of tokens through coarse-to-fine next-scale prediction under categorical modality and sequence conditions. The pretrained decoder ( ) reconstructs brain images or segmentation masks from generated tokens. b. Task-aligned data synthesis adapts BrainGFM with limited supervision using LoRA. For classification, task-specific class conditioning enables the synthesis of class-labelled images for training ResNet50. For segmentation, BrainGFM generates task-specific masks and synthesizes corresponding images, producing image-mask pairs for training UNet. c. Task-adapted post-training adapts BrainGFM for image enhancement using paired source-target images. Source-image tokens condition autoregressive prediction of target-image tokens, while LoRA is optimized using token-level prediction and image-level reconstruction losses.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Data flow and study design of BrainGFM. a. Eleven public brain-imaging resources comprising 64,754 3D volumes and 1,295,080 axial slices were collected and partitioned into pretraining and held-out subsets. The pretraining subset was used for two-stage generative pretraining, including visual tokenizer training and autoregressive modeling.Held-out samples from nine shared datasets, together with 15 additional public datasets, supported downstream evaluation across 48 tasks, including segmentation (20 tasks), classification (8 tasks), and image enhancement (20 tasks). Task-specific training data were constructed through labeled image–mask pair generation, class-conditioned image synthesis, and source–target image pairing. b. Six clinical stroke cohorts comprising 4,796 patients and 4,829 cases from two internal centers (JLH and NFH) and four external centers (NJH, WXPH, NJPH, and CZSPH) were collected. Task-eligible cases were selected according to task-specific imaging and annotation criteria for lesion segmentation, TOAST subtyping, and DWI-to-FLAIR transformation. Internal-center data were used for task-specific training, including low-data settings, while separate internal and external test sets were used for clinical validation.
| Dataset | License or Use Terms | Link |
| *ADNI Mueller and others (2005) | Data Use Agreement | https://adni.loni.usc.edu/data-samples/adni-data/#AccessData |
| FOMO-60K Cerri et al. (2026) | CC BY-NC-SA 4.0 | https://huggingface.co/datasets/FOMO-MRI/FOMO50K/tree/main |
| *ABIDE II Di Martino and others (2017) | CC BY-NC-SA | https://fcon_1000.projects.nitrc.org/indi/abide/abide_II.html |
| TCGA Weinstein and others (2013) | Varies by data type | https://portal.gdc.cancer.gov/analysis_page?app=Downloads |
| *TCIA Clark and others (2013) | Varies by data type | https://www.cancerimagingarchive.net/browse-collections/ |
| IBSR18 (2026) | Non-commercial use only | https://www.nitrc.org/frs/?group_id=48 |
Table 2: Public or access-controlled brain imaging datasets used in this study. The datasets used for BrainGFM pretraining and downstream evaluation are detailed below. The download link or access website is provided in the second column. * indicates the datasets in the BrainGFM-1.3M.
| Dataset | Modality | Total volumes | Total slices | Train volumes | Train slices | Held-out volumes | Held-out slices |
| ADNI | MRI | 3,840 | 76,800 | 3,413 | 68,260 | 427 | 8,540 |
| FOMO-60K | MRI | 47,591 | 951,820 | 42,303 | 846,060 | 5,288 | 105,760 |
| ABIDE II | MRI | 2,164 | 43,280 | 1,924 | 38,480 | 240 | 4,800 |
| TCGA | MRI | 668 | 13,360 | 592 | 11,840 | 76 | 1,520 |
| TCIA | MRI | 2,684 | 53,680 | 2,384 | 47,680 | 300 | 6,000 |
| IBSR18 | MRI | 18 | 360 | 16 | 320 | 2 | 40 |
Table 3: Composition of BrainGFM-1.3M. The 11 source datasets comprise 64,754 3D volumes and 1,295,080 axial slices (20 slices per volume). Training and held-out partitions were constructed at the patient level using a ratio of 8:1, with all available volumes from the same patient assigned to the same partition. Owing to differences in the number of imaging sequences per patient, the resulting volume counts do not necessarily follow an exact 8:1 ratio. The table reports the corresponding volume and slice counts for each source.
| Low-data training | Full-data training | Testing | |||||||
| Task-Dataset-Modality | Native | Modality | Volumes | Slices | Volumes | Slices | Volumes | Slices | Target |
| TuS: Brain tumour segmentation (13 tasks) | |||||||||
| TuS-BraTS21-T1 | 3D | T1 | - | 20 | - | 2,240 | - | 440 | Tumour regions (ET, TC, WT) |
| TuS-TCIA-T1 | 3D | T1 | - | 20 | - | 1,240 | - | 240 | Brain tumour masks |
| TuS-TCGA-T1 | 3D | T1 | - | 20 | - | 300 | - | 60 | Brain tumour masks |
| TuS-BraTS21-T1ce | 3D | T1ce | - | 20 | - | 2,240 | - | 440 | Tumour regions (ET, TC, WT) |
Table 4: Training and testing data partitions for the 48 downstream BrainGFM tasks. The table summarizes the data partitions for 20 segmentation, eight classification, and 20 image-enhancement tasks, with sample counts reported separately as cases/volumes and slices/images. For segmentation and classification, the low-data training partition denotes the few annotated samples used for task-specific adaptation, and the full-data training partition denotes the full setting reference for UNet (full) and ResNet50 (full). For image enhancement, the experiments are implemented in the full-data setting. For volumetric tumour-segmentation tasks, one case comprising 20 axial slices was reserved for low-data adaptation, and the remaining cases were divided approximately 5:1 into full-data training and testing partitions. Dashes indicate quantities that are not separately reported or are not applicable to the corresponding task. All sample counts represent real image data and exclude synthetically generated samples.
| Internal cohorts | External cohorts | ||||||
| JLH | NFH | NJH | WXPH | NJPH | CZSPH | Total | |
| Cohort characteristics | |||||||
| Patients, | 2,617 | 1,669 | 116 | 143 | 125 | 126 | 4,796 |
| Imaging examinations, | 2,617 | 1,702 | 116 | 143 | 125 | 126 | 4,829 |
| Age, years | 67.7 , =2,251 | 61.4 , =1,582 | 67.1 , =116 | 67.2 , =143 | 69.2 , =125 | 67.2 , =126 | 65.4 , =4,343 |
| Female sex, (%) | 710/2,251 (31.5%) | 465/1,582 (29.4%) | 31/116 (26.7%) | 72/143 (50.3%) | 55/125 (44.0%) | 52/126 (41.3%) | 1,385/4,343 (31.9%) |
Table 5: Characteristics of the multicenter clinical stroke cohorts. Data are reported as mean s.d., median (IQR), or (%), as appropriate. Percentages are based on available cases. JLH, Jinling Hospital; NFH, Nanjing First Hospital; NJH, Nanjing Jiangning Hospital; WXPH, Wuxi People’s Hospital; NJPH, Northern Jiangsu People’s Hospital; CZSPH, Changzhou Second People’s Hospital; NIHSS, National Institutes of Health Stroke Scale; TOAST, Trial of Org 10172 in Acute Stroke Treatment; mRS, modified Rankin Scale.
| Experts | Metric | Real | BrainGFM | MAE | CAR |
| ODQ | 3.65 | 3.35 | 2.70 | 2.48 | |
| (3.44-3.86) | (3.14-3.56) | (2.49-2.91) | (2.27-2.69) | ||
| AF | 3.50 | 3.22 | 2.70 | 2.43 | |
| (3.29-3.71) | (3.01-3.43) | (2.49-2.91) | (2.22-2.64) | ||
| CF | 3.50 | 3.23 | 2.70 | 2.47 | |
| (3.29-3.71) | (3.02-3.44) | (2.49-2.91) | (2.26-2.68) |
Table 6: Radiologist assessment of real and generated FLAIR images. Expert-specific and pooled ratings across five image-quality domains. Values are presented as mean ratings with the reported CIs in parentheses. All images were assessed on a five-point Likert scale, with higher scores indicating better perceived quality. ODQ, overall diagnostic quality; AF, anatomical fidelity; CF, contrast fidelity; AHQ, artifact- and hallucination-free quality; DC, diagnostic confidence. The “Mean” row represents the arithmetic mean across the five quality domains. “All readers” denotes the pooled ratings from the three experts. Bold and underlined values indicate the highest and second-highest mean scores, within each reader and evaluation metric.
| Expert 1 ( 15 yr) | Expert 2 ( 10 yr) | Expert 3 ( 10 yr) | |||||||
| Cohort | Metric | Subtype | Human | Human + AI | Human | Human + AI | Human | Human + AI | BrainGFM |
| LAA | 67.3 | 63.3 (-4.0) | 54.7 | 80.7 (+26.0) | 65.1 | 62.3 (-2.7) | 80.7 | ||
| CE | 84.7 | 86.7 (+2.0) | 87.3 | 90.7 (+3.3) | 89.7 | 91.1 (+1.4) | 91.3 | ||
| SAO | 73.3 | 67.3 (-6.0) | 60.7 | 82.0 (+21.3) | 67.1 | 67.1 (0.0) | 82.7 | ||
| [0.8pt/2pt] | Accuracy | Macro mean | 75.1 | 72.4 (-2.7) | 67.6 | 84.4 (+16.9) | 74.0 | 73.5 (-0.5) | 84.9 |
| LAA | 69.7 | 69.7 (0.0) | 90.9 | 90.9 (0.0) | 71.4 | 77.8 (+6.3) | 84.9 | ||
Table 7: Human-AI collaboration study: Reader-specific diagnostic performance and response to BrainGFM assistance in internal and external stroke cohorts. Diagnostic performance was compared before and after AI assistance, with standalone BrainGFM as a reference. Values in parentheses indicate absolute changes (%) in percentage points following AI assistance. Diagnostic confidence, AI acceptance, and diagnostic corrections were summarized across both cohorts. Expert 2 exhibited the largest performance improvements, alongside the lowest baseline high-confidence rate (8.3%), highest AI acceptance rate (90.3%), and greatest proportion of incorrect diagnoses corrected (27.3%). In contrast, Experts 1 and 3 showed higher baseline confidence, lower AI acceptance, and more limited or variable improvements. These findings highlight substantial inter-reader heterogeneity in the clinical benefits of BrainGFM assistance.
| Internal 2 cohorts | External 4 cohorts | ||||||||||||
| Task | Method | PSNR | SSIM | RMSE | LPIPS | PSNR | SSIM | RMSE | LPIPS | ||||
| RegGAN | 23.95 | 87.1 | 16.31 | 14.02 | 22.92 | 87.3 | 18.28 | 14.57 | |||||
| (21.24-26.66) | (80.9-93.2) | (15.99-16.63) | (5.38-22.66) | (20.40-25.44) | (80.9-93.7) | (17.26-19.29) | (6.53-22.61) | ||||||
| CycleGAN | 24.14 | 87.4 | 15.93 | 14.15 | 23.29 | 87.1 | 17.57 | 13.80 | |||||
| (20.48-27.80) | (80.7-94.1) | (15.62-16.23) | (5.71-22.59) | (19.34-27.24) | (80.3-93.9) | (16.59-18.55) | (5.02-22.58) | ||||||
| MAE | 25.34 | 88.7 | 13.87 | 14.17 | 24.52 | 89.1 | 15.07 | 16.10 | |||||
Table 8: Performance of BrainGFM in multicenter clinical stroke-imaging validation. Performance was evaluated on pooled task-eligible cases from two internal and four external centers. Higher PSNR, SSIM, Dice, ACC, F1, and AUC indicate better performance, whereas lower LPIPS and HD95 indicate better performance.
| TuS-BraTS21-T1 | TuS-TCIA-T1 | TuS-TCGA-T1 | TuS-BraTS21-T1ce | TuS-TCIA-T1ce | |
| Method | Dice | Dice | Dice | Dice | Dice |
| UNet (few) | 39.6 (36.1-43.1) | 43.1 (39.8-46.4) | 43.2 (39.6-46.8) | 58.2 (55.7-60.7) | 60.4 (58.1-62.7) |
| CAR | 46.3 (42.9-49.7) | 49.0 (45.8-52.2) | 45.1 (41.6-48.6) | 70.0 (67.9-72.1) | 68.9 (66.9-70.9) |
| MedSAM | 48.7 (45.6-51.8) | 51.2 (48.2-54.2) | 46.8 (43.5-50.1) | 73.5 (71.6-75.4) | 70.9 (69.0-72.8) |
| MAE | 47.9 (44.6-51.2) | 49.8 (46.7-52.9) | 45.9 (42.4-49.4) | 71.4 (69.3-73.5) | 69.8 (67.7-71.9) |
| BrainIAC | 49.6 (46.6-52.6) | 50.6 (47.7-53.5) | 47.6 (44.4-50.8) | 72.9 (71.1-74.7) | 71.4 (69.6-73.2) |
Table 9: Performance of BrainGFM in low-data segmentation across multiple datasets. Values are Dice scores (%) with 95% CIs. For datasets with multiple target regions, scores are averaged across regions. Higher Dice indicates better performance. The best low-data result is shown in bold, and the strongest competing low-data baseline is underlined. UNet (full) is reported as a full-data reference and is not included in the low-data ranking.
| SeC-BraTS23-# | TuC-Br35H-Unknow | |||||
| Method | ACC | F1 | AUC | ACC | F1 | AUC |
| ResNet50 (few) | 83.9 (81.7-86.0) | 84.0 (81.9-86.1) | 91.6 (89.9-93.3) | 87.3 (85.4-89.3) | 87.2 (85.2-89.3) | 93.9 (92.2-95.7) |
| CAR | 85.1 (83.1-87.0) | 85.1 (83.1-87.1) | 92.7 (91.1-94.3) | 89.4 (87.3-91.6) | 88.7 (86.5-90.8) | 94.8 (93.4-96.3) |
| MedSAM | 85.3 (83.1-87.5) | 85.4 (83.4-87.4) | 92.8 (91.3-94.4) | 89.2 (87.2-91.2) | 89.0 (87.0-91.0) | 94.77 (93.4-96.1) |
| MAE | 87.1 (85.1-89.2) | 86.5 (84.6-88.4) | 93.76 (92.3-95.2) | 89.8 (87.8-91.7) | 89.7 (87.6-91.9) | 95.6 (94.4-96.9) |
| BrainIAC | 91.0 (89.1-93.0) | 90.3 (88.2-92.3) | 97.8 (97.0-98.7) | 89.6 (87.5-91.8) | 89.2 (87.1-91.2) | 95.2 (94.1-96.3) |
Table 10: Performance of BrainGFM in low-data classification across multiple datasets. Values are ACC, F1, and AUC (%) with 95% CIs. Multiclass AUC is macro-averaged using one-vs-rest. Bold and underlined values indicate the best low-data result and strongest competing baseline. ResNet50 (full) is a full-data reference excluded from ranking. Dashes indicate unavailable results.
| LFMR-BraTS21-T1 | LFMR-BraTS21-T2 | |||||||
| Method | PSNR | SSIM | RMSE | LPIPS | PSNR | SSIM | RMSE | LPIPS |
| RegGAN | 33.66 (29.11-38.21) | 94.4 (90.6-98.2) | 29.11 (0.27-57.95) | 5.17 (1.12-9.22) | 34.03 (29.45-38.61) | 94.0 (90.2-97.8) | 37.21 (7.62-66.80) | 6.46 (1.99-10.93) |
| CycleGAN | 37.78 (34.10-41.46) | 97.2 (95.4-99.0) | 17.28 (2.45-32.11) | 1.71 (0.30-3.12) | 38.16 (34.54-41.78) | 97.4 (95.9-98.9) | 22.27 (5.17-39.37) | 1.31 (0.36-2.26) |
| MAE | 38.09 (34.69-41.49) | 97.4 (95.8-99.0) | 16.49 (1.49-31.49) | 1.88 (0.67-3.09) | 38.19 (34.84-41.54) | 97.6 (96.2-98.9) | 22.08 (5.26-38.90) | 1.65 (0.69-2.61) |
| DDIM | 37.10 (33.48-40.72) | 96.7 (94.7-98.7) | 18.59 (3.01-34.17) | 1.28 (0.35-2.21) | 37.88 (34.40-41.36) | 97.4 (95.9-98.9) | 22.95 (5.13-40.77) | 1.08 (0.45-1.71) |
| SDM | 37.75 (34.94-40.56) | 96.9 (95.7-98.1) | 16.08 (0.36-31.80) | 1.50 (0.58-2.42) | 37.90 (35.08-40.72) | 97.0 (95.8-98.1) | 22.01 (7.13-36.89) | 1.65 (0.85-2.45) |
Table 11: Performance of BrainGFM in medical-image enhancement across multiple datasets. Values are reported with 95% confidence intervals. Higher PSNR and SSIM indicate better performance, whereas lower RMSE and LPIPS indicate better performance. The best result is shown in bold, and the strongest competing baseline is underlined.
| NCMR-TCIA-T1,T1ce | NCMR-MSDBrain-T1,T1ce | |||||||
| Method | PSNR | SSIM | RMSE | LPIPS | PSNR | SSIM | RMSE | LPIPS |
| RegGAN | 26.91 (24.56-29.26) | 91.1 (86.6-95.7) | 46.07 (7.72-84.42) | 7.42 (2.51-12.33) | 25.98 (23.16-28.80) | 90.9 (86.2-95.6) | 42.58 (8.14-77.02) | 7.61 (2.53-12.69) |
| CycleGAN | 28.14 (25.67-30.61) | 95.3 (92.9-97.7) | 25.46 (5.62-45.30) | 2.29 (1.12-3.46) | 28.79 (26.20-31.38) | 94.8 (92.4-97.2) | 25.51 (6.36-44.66) | 2.35 (1.11-3.59) |
| MAE | 26.56 (23.98-29.14) | 95.6 (93.9-97.3) | 25.53 (6.17-44.89) | 1.74 (0.92-2.56) | 29.83 (27.27-32.39) | 95.4 (93.4-97.3) | 21.22 (5.31-37.13) | 2.57 (1.31-3.83) |
| DDIM | 29.84 (27.43-32.25) | 94.8 (92.3-97.4) | 25.91 (5.73-46.09) | 1.81 (0.94-2.68) | 28.51 (25.97-31.05) | 94.5 (92.0-96.9) | 26.61 (6.67-46.55) | 1.88 (0.97-2.79) |
| SDM | 30.18 (27.81-32.55) | 95.1 (92.8-97.4) | 24.76 (5.43-44.09) | 1.91 (0.98-2.84) | 27.75 (25.13-30.37) | 94.6 (91.4-97.7) | 30.96 (4.38-57.54) | 1.97 (0.96-2.98) |
Figure 8: Modality- and sequence-conditioned brain image and segmentation mask generation by BrainGFM. Top, schematic illustration of prompt-guided generation, in which modality, MRI sequence, and mask prompts guide BrainGFM to synthesize brain images and segmentation masks. Representative generated segmentation masks are displayed in pseudocolor to distinguish different anatomical or pathological regions. Bottom, representative synthetic brain images generated under six imaging conditions: DWI, T1, T1ce, T2, FLAIR, and CT. The examples illustrate the diversity of anatomical structures, tissue contrast, and imaging appearances captured by BrainGFM across heterogeneous modalities and sequences.
Figure 9: Qualitative comparison of low-field brain MRI enhancement across different methods.
Figure 10: Qualitative comparison of low-dose brain CT enhancement across different methods.
Figure 11: Qualitative comparison of non-contrast brain MRI enhancement across different methods.
Figure 12: Qualitative comparison of DWI-to-FLAIR brain MRI transformation across different methods.
Figure 13: Representative cases illustrating diagnostic improvement and deterioration following BrainGFM assistance. a. Six ischemic stroke cases in which BrainGFM assistance improved radiologists’ stroke-subtyping accuracy. b. Two cases in which diagnostic accuracy deteriorated following AI assistance. For each case, DWI and ADC maps are shown alongside patient age, sex, large-vessel occlusion status, and atrial fibrillation status. Donut charts show the proportion of the three radiologists who correctly identified the stroke subtype before (experts alone) and after (experts assisted by AI) BrainGFM assistance, with red and blue sectors indicating correct and incorrect diagnoses. These examples illustrate the potential diagnostic benefits of BrainGFM assistance and the risk of deterioration in initially correct clinical judgments.
More in this topic
Medical Imaging Foundation Models
Medical imaging foundation models learn reusable representations for tasks across clinical images, modalities, and institutions.
See all 238 papers