A generative foundation model for computational brain imaging
Authors: Yuting He, Xu Wen, Hongxing Lan, Chengliang Liu, Yuxing Liu, Yan Xi, Wenxue Yu, Jindi Kong, Zhiqiang Zhang, Liang Jiang, Rongjun Ge, Yang Chen, Multi-Center Cooperative Clinical Study Group for "Lesion-Phenotype Brain Mapping in Acute Stroke"
Organizations: Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA. · School of Computer Science and Engineering, Southeast University, Nanjing, China. · Department of Computer and Information Science, University of Macau, Macau, China. · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China. · Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, Ohio, USA. · Department of Radiology, Affiliated Jinling Hospital, Medical School of Nanjing University, Nanjing, China. · Department of Radiology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China. · School of Instrument Science and Engineering, Southeast University, Nanjing, China.
Computational brain imaging is constrained by a mismatch between increasingly diverse tasks and scarce, heterogeneous clinical data. Although existing brain foundation models transfer pretrained representations, their knowledge remains entangled within model-specific features and fixed architectures. This paper proposes BrainGFM, a generative foundation model that extends knowledge transfer beyond representation reuse through a reusable and reconfigurable brain-imaging prior. By pretraining on a corpus of about 1.3 million MRI and CT slices, BrainGFM supported task-aligned data synthesis for low-data predictive learning and task-adapted post-training for image reconstruction. Across 48 downstream tasks, BrainGFM improved segmentation and classification under limited supervision and achieved high-fidelity image enhancement across heterogeneous imaging conditions. Multicenter clinical validation involving 4,796 stroke patients across two internal and four external centers demonstrated its applicability to three clinically relevant tasks around stroke cohorts. Blinded reader assessments showed that BrainGFM-generated FLAIR images approached acquired FLAIR in overall diagnostic quality. In a human-AI collaboration study, BrainGFM assistance yielded a 14% relative improvement in mean radiologist stroke-subtyping accuracy. These findings support generative priors as a reusable foundation for heterogeneous computational brain-imaging applications.
Figures & tables
Figure 1: Overview of BrainGFM for Computational brain imaging. a. Heterogeneity in brain diseases, patient populations, clinical status, acquisition protocols, and clinical centers produces heterogeneous brain imaging distributions and limits the generalization of DL models. b. Our BrainGFM-1.3M comprises about 1.3 million axial slices spanning major MRI sequences and CT, providing a large-scale dataset for brain generative pretraining. c. Our BrainGFM learns a reusable brain-imaging prior from generative pretraining on our BrainGFM-1.3M dataset. d. Task-specific conditioning controls the pretrained generator where class labels are used for class-conditioned synthesis, segmentation masks for spatially conditioned synthesis, and source images for brain image reconstruction. These conditions are combined with task-specific adaptation as detailed in Fig. 6 b,c. e. Evaluation spans 48 downstream tasks, comprising 20 segmentation, 8 classification, and 20 image-enhancement tasks. f. Clinical validation in a multicenter stroke cohort assesses BrainGFM for TOAST stroke subtyping, lesion segmentation, and MRI sequence transformation, as well as its clinical utility in reader studies and radiologist-AI collaboration.
Figure 2: Task-aligned data synthesis enables low-data predictive learning. a. Overview of the task-aligned data synthesis framework. BrainGFM is adapted with few brain images and synthesizes large-scale task-aligned data to train downstream segmentation and classification models. b. Task-family-level performance. BrainGFM is compared with direct low-data training, the strongest competing low-data method, and full-data training references. BrainGFM achieves strong Dice and AUC across heterogeneous prediction targets. c. Performance across 20 segmentation and 8 classification tasks. BrainGFM maintains strong performance across heterogeneous datasets, imaging modalities, and prediction targets. d. Task robustness measured by mean and worst-task performance. BrainGFM achieves a favourable balance between overall and lower-bound performance, indicating its robustness and broad applicability. e. Amplifiability of low-data supervision. The lollipop plot shows BrainGFM’s performance gain compared with direct low-data training (with − without BrainGFM), measured as Δ Dice or Δ AUC. BrainGFM achieves positive gains across all tasks, demonstrating its powerful amplifiability for the value of few supervision. Superscript ∧ denotes models trained using the full dataset, and superscript ∗ denotes brain-imaging foundation models.
Figure 3: Task-adapted post-training enables high-fidelity brain-image enhancement. a. Overview of the task-adapted post-training framework. BrainGFM is adapted with paired task-specific brain images and directly enhances non-ideal measurements to enhanced brain images. b. The radar plot shows PSNR across tasks, and the heat map reports BrainGFM’s rank for PSNR, SSIM, RMSE, and LPIPS. BrainGFM maintains strong performance across heterogeneous datasets, modalities, and enhancement targets. c. Representative qualitative comparisons. BrainGFM reduces image degradation while preserving anatomical structures and local details. d. Task-family-level performance. BrainGFM outperforms CAR and the strongest competitors across the three task families and four evaluation metrics. e. Robustness between mean and worst-task PSNR. BrainGFM achieves a favourable balance between overall and lower-bound performance. f. Amplifiability of task-adapted post-training. The lollipop plot reports BrainGFM’s PSNR gain over CAR as Δ PSNR. Positive gains across all tasks demonstrate that task-adapted post-training amplifies the enhancement capability of the model without brain-image pretraining.
Internal cohort
External cohort
Task
Method
JLH, NFH
NJH, WXPH, NJPH, CZSPH
UNet (few)
58.0
50.8
60 cases
(57.3-58.7)
(48.3-53.3)
CAR
69.6
67.2
(68.8-70.3)
(66.4-67.9)
MedSAM
72.0
70.3
Table 1: Multicenter validation across stroke cohorts. Performance was evaluated on pooled eligible cases from the two internal centers and four external centers for MRI sequence transformation, lesion segmentation, and TOAST stroke subtyping.
Figure 4: Blinded reader study and human-AI collaboration study in stroke cohorts. a-b Workflow of the blinded reader study for acquired and DWI-synthesized FLAIR images and the human-AI collaboration study for ischemic stroke subtyping. c. Radiologist ratings across five image-quality criteria: overall diagnostic quality (ODQ), anatomical fidelity (AF), FLAIR contrast fidelity (CF), artifact- and hallucination-free quality (AHQ), and diagnostic confidence (DC). d. Proportion of reader assessments meeting all five image-quality criteria at increasing Likert-score thresholds. e. F1 score, one-versus-rest accuracy, and sensitivity for experts, AI-assisted experts, and BrainGFM across LAA, CE, and SAO. f. Relationship between recorded AI recommendation acceptance and changes in diagnostic accuracy among initially high-confidence assessments (confidence score 3). Solid and dashed contours represent the 50% and 95% regions of the bootstrap distributions. Error bars in panels c-e indicate 95% CIs.
Figure 5: Ablation study and model analysis of BrainGFM. a. Synthetic-data scaling across representative segmentation and classification tasks shows consistent performance gains followed by task-dependent saturation, demonstrating that task-aligned synthesis effectively amplifies limited supervision. b. Data scaling across three enhancement tasks shows progressive improvement with increasing post-training data, with task-dependent saturation indicating efficient adaptation from limited paired data. c. t-SNE analysis of real, BrainGFM-generated, and CAR-generated features shows greater overlap between BrainGFM and real-data distributions, supporting improved distribution alignment. d. Representative BrainGFM-generated images across modalities and sequences preserve modality-specific contrast, anatomical structure, and local image details, demonstrating multimodal generation fidelity. e. Intermediate image and mask predictions across successive feature scales reveal progressive formation of global structure and fine spatial details, illustrating the coarse-to-fine next-scale generation process.
Figure 6: BrainGFM architecture, generative pretraining and downstream adaptation. a. BrainGFM pretraining consists of brain-domain adaptation of a multi-scale visual tokenizer and next-scale autoregressive generative pretraining. A shared encoder ( E ) and vector quantizer ( Q ) map brain images and segmentation masks into multi-scale discrete tokens using a shared visual codebook. The autoregressive Transformer learns the distribution of tokens through coarse-to-fine next-scale prediction under categorical modality and sequence conditions. The pretrained decoder ( D ) reconstructs brain images or segmentation masks from generated tokens. b. Task-aligned data synthesis adapts BrainGFM with limited supervision using LoRA. For classification, task-specific class conditioning enables the synthesis of class-labelled images for training ResNet50. For segmentation, BrainGFM generates task-specific masks and synthesizes corresponding images, producing image-mask pairs for training UNet. c. Task-adapted post-training adapts BrainGFM for image enhancement using paired source-target images. Source-image tokens condition autoregressive prediction of target-image tokens, while LoRA is optimized using token-level prediction and image-level reconstruction losses.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Data flow and study design of BrainGFM. a. Eleven public brain-imaging resources comprising 64,754 3D volumes and 1,295,080 axial slices were collected and partitioned into pretraining and held-out subsets. The pretraining subset was used for two-stage generative pretraining, including visual tokenizer training and autoregressive modeling.Held-out samples from nine shared datasets, together with 15 additional public datasets, supported downstream evaluation across 48 tasks, including segmentation (20 tasks), classification (8 tasks), and image enhancement (20 tasks). Task-specific training data were constructed through labeled image–mask pair generation, class-conditioned image synthesis, and source–target image pairing. b. Six clinical stroke cohorts comprising 4,796 patients and 4,829 cases from two internal centers (JLH and NFH) and four external centers (NJH, WXPH, NJPH, and CZSPH) were collected. Task-eligible cases were selected according to task-specific imaging and annotation criteria for lesion segmentation, TOAST subtyping, and DWI-to-FLAIR transformation. Internal-center data were used for task-specific training, including low-data settings, while separate internal and external test sets were used for clinical validation.
Table 2: Public or access-controlled brain imaging datasets used in this study. The datasets used for BrainGFM pretraining and downstream evaluation are detailed below. The download link or access website is provided in the second column. * indicates the datasets in the BrainGFM-1.3M.
Dataset
Modality
Total volumes
Total slices
Train volumes
Train slices
Held-out volumes
Held-out slices
ADNI
MRI
3,840
76,800
3,413
68,260
427
8,540
FOMO-60K
MRI
47,591
951,820
42,303
846,060
5,288
105,760
ABIDE II
MRI
2,164
43,280
1,924
38,480
240
4,800
TCGA
MRI
668
13,360
592
11,840
76
1,520
TCIA
MRI
2,684
53,680
2,384
47,680
300
6,000
IBSR18
MRI
18
360
16
320
2
40
Appendix
Table 3: Composition of BrainGFM-1.3M. The 11 source datasets comprise 64,754 3D volumes and 1,295,080 axial slices (20 slices per volume). Training and held-out partitions were constructed at the patient level using a ratio of 8:1, with all available volumes from the same patient assigned to the same partition. Owing to differences in the number of imaging sequences per patient, the resulting volume counts do not necessarily follow an exact 8:1 ratio. The table reports the corresponding volume and slice counts for each source.
Low-data training
Full-data training
Testing
Task-Dataset-Modality
Native
Modality
Volumes
Slices
Volumes
Slices
Volumes
Slices
Target
TuS: Brain tumour segmentation (13 tasks)
TuS-BraTS21-T1
3D
T1
-
20
-
2,240
-
440
Tumour regions (ET, TC, WT)
TuS-TCIA-T1
3D
T1
-
20
-
1,240
-
240
Brain tumour masks
TuS-TCGA-T1
3D
T1
-
20
-
300
-
60
Brain tumour masks
TuS-BraTS21-T1ce
3D
T1ce
-
20
-
2,240
-
440
Tumour regions (ET, TC, WT)
Appendix
Table 4: Training and testing data partitions for the 48 downstream BrainGFM tasks. The table summarizes the data partitions for 20 segmentation, eight classification, and 20 image-enhancement tasks, with sample counts reported separately as cases/volumes and slices/images. For segmentation and classification, the low-data training partition denotes the few annotated samples used for task-specific adaptation, and the full-data training partition denotes the full setting reference for UNet (full) and ResNet50 (full). For image enhancement, the experiments are implemented in the full-data setting. For volumetric tumour-segmentation tasks, one case comprising 20 axial slices was reserved for low-data adaptation, and the remaining cases were divided approximately 5:1 into full-data training and testing partitions. Dashes indicate quantities that are not separately reported or are not applicable to the corresponding task. All sample counts represent real image data and exclude synthetically generated samples.
Internal cohorts
External cohorts
JLH
NFH
NJH
WXPH
NJPH
CZSPH
Total
Cohort characteristics
Patients, n
2,617
1,669
116
143
125
126
4,796
Imaging examinations, n
2,617
1,702
116
143
125
126
4,829
Age, years
67.7 ±12.3 , n =2,251
61.4 ±11.3 , n =1,582
67.1 ±13.2 , n =116
67.2 ±13.3 , n =143
69.2 ±11.8 , n =125
67.2 ±12.3 , n =126
65.4 ±12.4 , n =4,343
Female sex, n/N (%)
710/2,251 (31.5%)
465/1,582 (29.4%)
31/116 (26.7%)
72/143 (50.3%)
55/125 (44.0%)
52/126 (41.3%)
1,385/4,343 (31.9%)
Appendix
Table 5: Characteristics of the multicenter clinical stroke cohorts. Data are reported as mean ± s.d., median (IQR), or n (%), as appropriate. Percentages are based on available cases. JLH, Jinling Hospital; NFH, Nanjing First Hospital; NJH, Nanjing Jiangning Hospital; WXPH, Wuxi People’s Hospital; NJPH, Northern Jiangsu People’s Hospital; CZSPH, Changzhou Second People’s Hospital; NIHSS, National Institutes of Health Stroke Scale; TOAST, Trial of Org 10172 in Acute Stroke Treatment; mRS, modified Rankin Scale.
Experts
Metric
Real
BrainGFM
MAE
CAR
ODQ
3.65
3.35
2.70
2.48
(3.44-3.86)
(3.14-3.56)
(2.49-2.91)
(2.27-2.69)
AF
3.50
3.22
2.70
2.43
(3.29-3.71)
(3.01-3.43)
(2.49-2.91)
(2.22-2.64)
CF
3.50
3.23
2.70
2.47
(3.29-3.71)
(3.02-3.44)
(2.49-2.91)
(2.26-2.68)
Appendix
Table 6: Radiologist assessment of real and generated FLAIR images. Expert-specific and pooled ratings across five image-quality domains. Values are presented as mean ratings with the reported CIs in parentheses. All images were assessed on a five-point Likert scale, with higher scores indicating better perceived quality. ODQ, overall diagnostic quality; AF, anatomical fidelity; CF, contrast fidelity; AHQ, artifact- and hallucination-free quality; DC, diagnostic confidence. The “Mean” row represents the arithmetic mean across the five quality domains. “All readers” denotes the pooled ratings from the three experts. Bold and underlined values indicate the highest and second-highest mean scores, within each reader and evaluation metric.
Expert 1 ( ≥ 15 yr)
Expert 2 ( ≥ 10 yr)
Expert 3 ( ≥ 10 yr)
Cohort
Metric
Subtype
Human
Human + AI
Human
Human + AI
Human
Human + AI
BrainGFM
LAA
67.3
63.3 (-4.0)
54.7
80.7 (+26.0)
65.1
62.3 (-2.7)
80.7
CE
84.7
86.7 (+2.0)
87.3
90.7 (+3.3)
89.7
91.1 (+1.4)
91.3
SAO
73.3
67.3 (-6.0)
60.7
82.0 (+21.3)
67.1
67.1 (0.0)
82.7
[0.8pt/2pt]
Accuracy
Macro mean
75.1
72.4 (-2.7)
67.6
84.4 (+16.9)
74.0
73.5 (-0.5)
84.9
LAA
69.7
69.7 (0.0)
90.9
90.9 (0.0)
71.4
77.8 (+6.3)
84.9
Appendix
Table 7: Human-AI collaboration study: Reader-specific diagnostic performance and response to BrainGFM assistance in internal and external stroke cohorts. Diagnostic performance was compared before and after AI assistance, with standalone BrainGFM as a reference. Values in parentheses indicate absolute changes (%) in percentage points following AI assistance. Diagnostic confidence, AI acceptance, and diagnostic corrections were summarized across both cohorts. Expert 2 exhibited the largest performance improvements, alongside the lowest baseline high-confidence rate (8.3%), highest AI acceptance rate (90.3%), and greatest proportion of incorrect diagnoses corrected (27.3%). In contrast, Experts 1 and 3 showed higher baseline confidence, lower AI acceptance, and more limited or variable improvements. These findings highlight substantial inter-reader heterogeneity in the clinical benefits of BrainGFM assistance.
Internal 2 cohorts
External 4 cohorts
Task
Method
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
RegGAN
23.95
87.1
16.31
14.02
22.92
87.3
18.28
14.57
(21.24-26.66)
(80.9-93.2)
(15.99-16.63)
(5.38-22.66)
(20.40-25.44)
(80.9-93.7)
(17.26-19.29)
(6.53-22.61)
CycleGAN
24.14
87.4
15.93
14.15
23.29
87.1
17.57
13.80
(20.48-27.80)
(80.7-94.1)
(15.62-16.23)
(5.71-22.59)
(19.34-27.24)
(80.3-93.9)
(16.59-18.55)
(5.02-22.58)
MAE
25.34
88.7
13.87
14.17
24.52
89.1
15.07
16.10
Appendix
Table 8: Performance of BrainGFM in multicenter clinical stroke-imaging validation. Performance was evaluated on pooled task-eligible cases from two internal and four external centers. Higher PSNR, SSIM, Dice, ACC, F1, and AUC indicate better performance, whereas lower LPIPS and HD95 indicate better performance.
TuS-BraTS21-T1
TuS-TCIA-T1
TuS-TCGA-T1
TuS-BraTS21-T1ce
TuS-TCIA-T1ce
Method
Dice ↑
Dice ↑
Dice ↑
Dice ↑
Dice ↑
UNet (few)
39.6 (36.1-43.1)
43.1 (39.8-46.4)
43.2 (39.6-46.8)
58.2 (55.7-60.7)
60.4 (58.1-62.7)
CAR
46.3 (42.9-49.7)
49.0 (45.8-52.2)
45.1 (41.6-48.6)
70.0 (67.9-72.1)
68.9 (66.9-70.9)
MedSAM
48.7 (45.6-51.8)
51.2 (48.2-54.2)
46.8 (43.5-50.1)
73.5 (71.6-75.4)
70.9 (69.0-72.8)
MAE
47.9 (44.6-51.2)
49.8 (46.7-52.9)
45.9 (42.4-49.4)
71.4 (69.3-73.5)
69.8 (67.7-71.9)
BrainIAC
49.6 (46.6-52.6)
50.6 (47.7-53.5)
47.6 (44.4-50.8)
72.9 (71.1-74.7)
71.4 (69.6-73.2)
Appendix
Table 9: Performance of BrainGFM in low-data segmentation across multiple datasets. Values are Dice scores (%) with 95% CIs. For datasets with multiple target regions, scores are averaged across regions. Higher Dice indicates better performance. The best low-data result is shown in bold, and the strongest competing low-data baseline is underlined. UNet (full) is reported as a full-data reference and is not included in the low-data ranking.
SeC-BraTS23-#
TuC-Br35H-Unknow
Method
ACC ↑
F1 ↑
AUC ↑
ACC ↑
F1 ↑
AUC ↑
ResNet50 (few)
83.9 (81.7-86.0)
84.0 (81.9-86.1)
91.6 (89.9-93.3)
87.3 (85.4-89.3)
87.2 (85.2-89.3)
93.9 (92.2-95.7)
CAR
85.1 (83.1-87.0)
85.1 (83.1-87.1)
92.7 (91.1-94.3)
89.4 (87.3-91.6)
88.7 (86.5-90.8)
94.8 (93.4-96.3)
MedSAM
85.3 (83.1-87.5)
85.4 (83.4-87.4)
92.8 (91.3-94.4)
89.2 (87.2-91.2)
89.0 (87.0-91.0)
94.77 (93.4-96.1)
MAE
87.1 (85.1-89.2)
86.5 (84.6-88.4)
93.76 (92.3-95.2)
89.8 (87.8-91.7)
89.7 (87.6-91.9)
95.6 (94.4-96.9)
BrainIAC
91.0 (89.1-93.0)
90.3 (88.2-92.3)
97.8 (97.0-98.7)
89.6 (87.5-91.8)
89.2 (87.1-91.2)
95.2 (94.1-96.3)
Appendix
Table 10: Performance of BrainGFM in low-data classification across multiple datasets. Values are ACC, F1, and AUC (%) with 95% CIs. Multiclass AUC is macro-averaged using one-vs-rest. Bold and underlined values indicate the best low-data result and strongest competing baseline. ResNet50 (full) is a full-data reference excluded from ranking. Dashes indicate unavailable results.
LFMR-BraTS21-T1
LFMR-BraTS21-T2
Method
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
RegGAN
33.66 (29.11-38.21)
94.4 (90.6-98.2)
29.11 (0.27-57.95)
5.17 (1.12-9.22)
34.03 (29.45-38.61)
94.0 (90.2-97.8)
37.21 (7.62-66.80)
6.46 (1.99-10.93)
CycleGAN
37.78 (34.10-41.46)
97.2 (95.4-99.0)
17.28 (2.45-32.11)
1.71 (0.30-3.12)
38.16 (34.54-41.78)
97.4 (95.9-98.9)
22.27 (5.17-39.37)
1.31 (0.36-2.26)
MAE
38.09 (34.69-41.49)
97.4 (95.8-99.0)
16.49 (1.49-31.49)
1.88 (0.67-3.09)
38.19 (34.84-41.54)
97.6 (96.2-98.9)
22.08 (5.26-38.90)
1.65 (0.69-2.61)
DDIM
37.10 (33.48-40.72)
96.7 (94.7-98.7)
18.59 (3.01-34.17)
1.28 (0.35-2.21)
37.88 (34.40-41.36)
97.4 (95.9-98.9)
22.95 (5.13-40.77)
1.08 (0.45-1.71)
SDM
37.75 (34.94-40.56)
96.9 (95.7-98.1)
16.08 (0.36-31.80)
1.50 (0.58-2.42)
37.90 (35.08-40.72)
97.0 (95.8-98.1)
22.01 (7.13-36.89)
1.65 (0.85-2.45)
Appendix
Table 11: Performance of BrainGFM in medical-image enhancement across multiple datasets. Values are reported with 95% confidence intervals. Higher PSNR and SSIM indicate better performance, whereas lower RMSE and LPIPS indicate better performance. The best result is shown in bold, and the strongest competing baseline is underlined.
NCMR-TCIA-T1,T1ce
NCMR-MSDBrain-T1,T1ce
Method
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
PSNR ↑
SSIM ↑
RMSE ↓
LPIPS ↓
RegGAN
26.91 (24.56-29.26)
91.1 (86.6-95.7)
46.07 (7.72-84.42)
7.42 (2.51-12.33)
25.98 (23.16-28.80)
90.9 (86.2-95.6)
42.58 (8.14-77.02)
7.61 (2.53-12.69)
CycleGAN
28.14 (25.67-30.61)
95.3 (92.9-97.7)
25.46 (5.62-45.30)
2.29 (1.12-3.46)
28.79 (26.20-31.38)
94.8 (92.4-97.2)
25.51 (6.36-44.66)
2.35 (1.11-3.59)
MAE
26.56 (23.98-29.14)
95.6 (93.9-97.3)
25.53 (6.17-44.89)
1.74 (0.92-2.56)
29.83 (27.27-32.39)
95.4 (93.4-97.3)
21.22 (5.31-37.13)
2.57 (1.31-3.83)
DDIM
29.84 (27.43-32.25)
94.8 (92.3-97.4)
25.91 (5.73-46.09)
1.81 (0.94-2.68)
28.51 (25.97-31.05)
94.5 (92.0-96.9)
26.61 (6.67-46.55)
1.88 (0.97-2.79)
SDM
30.18 (27.81-32.55)
95.1 (92.8-97.4)
24.76 (5.43-44.09)
1.91 (0.98-2.84)
27.75 (25.13-30.37)
94.6 (91.4-97.7)
30.96 (4.38-57.54)
1.97 (0.96-2.98)
Appendix
Table 19
Figure 8: Modality- and sequence-conditioned brain image and segmentation mask generation by BrainGFM. Top, schematic illustration of prompt-guided generation, in which modality, MRI sequence, and mask prompts guide BrainGFM to synthesize brain images and segmentation masks. Representative generated segmentation masks are displayed in pseudocolor to distinguish different anatomical or pathological regions. Bottom, representative synthetic brain images generated under six imaging conditions: DWI, T1, T1ce, T2, FLAIR, and CT. The examples illustrate the diversity of anatomical structures, tissue contrast, and imaging appearances captured by BrainGFM across heterogeneous modalities and sequences.
Figure 9: Qualitative comparison of low-field brain MRI enhancement across different methods.
Figure 10: Qualitative comparison of low-dose brain CT enhancement across different methods.
Figure 11: Qualitative comparison of non-contrast brain MRI enhancement across different methods.
Figure 12: Qualitative comparison of DWI-to-FLAIR brain MRI transformation across different methods.
Figure 13: Representative cases illustrating diagnostic improvement and deterioration following BrainGFM assistance. a. Six ischemic stroke cases in which BrainGFM assistance improved radiologists’ stroke-subtyping accuracy. b. Two cases in which diagnostic accuracy deteriorated following AI assistance. For each case, DWI and ADC maps are shown alongside patient age, sex, large-vessel occlusion status, and atrial fibrillation status. Donut charts show the proportion of the three radiologists who correctly identified the stroke subtype before (experts alone) and after (experts assisted by AI) BrainGFM assistance, with red and blue sectors indicating correct and incorrect diagnoses. These examples illustrate the potential diagnostic benefits of BrainGFM assistance and the risk of deterioration in initially correct clinical judgments.
Foundation models pretrained using self-supervised learning have transformed computer vision by learning transferable representations from large-scale unlabeled data. However, existing foundation models for neuroimaging remain limited by task-specific training, slice-based learning strategies, or relatively small pretraining datasets, restricting their generalizability across diverse brain MRI applications. In this work, we present BrainNext, a general-purpose self-supervised foundation model for volumetric brain MRI analysis. BrainNext combines masked autoencoder (MAE) pretraining with a native three-dimensional Bi-Directional xLSTM-UNet architecture to learn rich anatomical representations from 60,551 unlabeled brain MRI examinations spanning multiple MRI modalities. The pretrained model is subsequently adapted to downstream tasks through lightweight task-specific fine-tuning. We evaluate BrainNext on the Foundation Models for Medical Imaging (FOMO) 2025 Method Track, encompassing classification, segmentation, and brain-age estimation, where it achieved second place overall and ranked first in the meningioma segmentation task on the official FOMO 2025 challenge leaderboard, demonstrating strong transferability across heterogeneous neuroimaging tasks. These results highlight the potential of large-scale self-supervised pretraining to learn robust and transferable volumetric representations, establishing BrainNext as a scalable foundation model for diverse brain MRI applications.
Moona Mazher, Abdul Qayyum, Steven A. Niederer +1
UCL Hawkes Institute, Department of Computer Science, University College London, London, United Kingdom · National Heart & Lung Institute, Imperial College London, London, United Kingdom · Division of Cardiovascular Medicine, Stanford Cardiovascular Institute, Stanford University, California, USA
Brain magnetic resonance imaging (MRI) is central to neuroscience and clinical assessment, but models are commonly developed for individual diseases, populations or imaging protocols. Foundation models promise more general representations, yet they are usually pretrained once and can lose earlier capabilities when updated with new data. Here we show that Alcmaeon, a three-dimensional brain MRI foundation model pretrained without manual labels on more than 425,000 volumes and derived imaging maps, can be expanded sequentially across clinical domains. Alcmaeon combines volumetric encoding and latent diffusion generation with Graph-Blueprint Pruning (GBP), which protects network modules important to earlier domains while leaving the remaining capacity trainable. Across expansion from healthy ageing and neurodegeneration to developmental, psychiatric and tumour imaging, GBP showed less forgetting than sequential adaptation and elastic weight consolidation across voxel-level reconstruction measures, with its largest advantage after adaptation to tumour imaging. The blueprints provided an inspectable record of how model capacity was protected and reused. Representations from different model levels supported image synthesis, disease classification, survival modelling and postoperative prediction, although no single representation was optimal for every task. These findings provide a route towards brain MRI foundation models that can grow with emerging data while retaining earlier capabilities.
Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li +8
1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom. +7
Brain MRI underpins a wide range of neuroscientific and clinical applications, yet most learning-based methods remain task-specific and require substantial labeled data. Here we show that a single self-supervised representation can generalize across heterogeneous brain MRI endpoints. We trained BrainDINO, a self-distilled foundation model, on approximately 6.6 million unlabeled axial slices from 20 datasets encompassing broad variation in population, disease, and acquisition setting. Using a frozen encoder with lightweight task heads, BrainDINO supported transfer across tumor segmentation, neurodegenerative and neurodevelopmental conditions classification, brain age estimation, post-stroke temporal prediction, molecular status prediction, MRI sequence classification, and survival modeling. Across tasks and supervision regimes, BrainDINO consistently equaled or exceeded natural-image and MRI-specific self-supervised baselines, with particularly strong advantages under label scarcity. Representation analyses further showed anatomically organized and pathology-sensitive feature structure in the absence of task-specific supervision. Our findings indicate that large-scale slice-wise self-supervised learning can yield a unified brain MRI representation that supports diverse neuroimaging tasks without volumetric pretraining or full-network fine-tuning, establishing a scalable foundation for robust and data-efficient brain imaging analysis. Code is available at https://github.com/mclwu22/BrainDINO
Yizhou Wu, Shansong Wang, Yuheng Li +5
Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA. · Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA. · Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA, USA. +2