BrainRotViT: Transformer-ResNet Hybrid for Explainable Modeling of Brain Aging from 3D sMRI

Wasif Jalal; Md Nafiu Rahman; Atif Hasan Rahman; M. Sohel Rahman

Figure 1: Overview of the proposed 2.5D, slice-based representation-learning framework that approximates volumetric context. The model integrates a Vision Transformer (ViT) encoder to extract slice-level embeddings with a lightweight 2D convolutional regression head to aggregate volumetric information. By applying 2D convolutions over a (slice × embedding) matrix derived from the transformer, the framework jointly models global contextual and local spatial patterns, enabling subsequent analyses on brain age gaps and thus linking aging with neurological conditions such as Alzheimer’s disease, mild cognitive impairment, and autism spectrum disorder.

Dataset	# Total Samples	# Samples Used	Age (Mean $\pm$ SD)
Training + Validation Datasets (Total = 5,108; Train = 4,086, Val = 1,022)
ADNI [ 36 ]	5881	900	63.04 $\pm$ 3.54
IXI [ 37 ]	565	565	48.69 $\pm$ 16.46
ABIDE [ 38 ]	1114	750	17.99 $\pm$ 9.72
DLBS [ 39 , 40 ]	315	315	54.62 $\pm$ 20.09
COBRE [ 41 ]	147	146	36.98 $\pm$ 12.78

Table 1: Demographic information of datasets used for training, validation, and independent cohort evaluation.

Figure 2: Implementation details of methodology. (a): Chronological age distribution of the 4086 training samples. (b): Chronological age distribution of the 1022 validation samples. (c): Pre-processing routine converting NIfTI-format 3D T1w MRI images to 2D sagittal slices used as model inputs. (d): Heatmap of a 160 pixel

\times

768 pixel 2D feature map (ViT embedding vectors of 160 slices concatenated in order; log-scaled, min-max scaled, and rotated for better visualization) — the input the residual CNN receives for one sample. Slices change along the vertical axis; the horizontal axis is the embedding dimension. Vertical symmetry appears across slices, and adjacent slices show similar embeddings, producing vertical lines of similarly intense positions. (e): Heatmap of a 160

\times

160 cosine similarity matrix between slices of 3D samples, averaged across all samples. The grid axes both index slices 1–160. The principal diagonal represents each slice’s similarity with itself (1, most intense red). Intensity spreading from the diagonal shows the ViT produces similar embeddings for neighboring slices, while intensity along the anti-diagonal shows symmetrically antipodal slices are very similar — i.e., the transformer effectively captures sagittal brain symmetry.

Figure 3: (a): Scatter plot of predicted age of validation samples from 11 datasets (MAE = 3.43 years, Pearson

r=0.98

, Spearman

\rho=0.97

,

R^{2}=0.96

). (b): Scatter plot of predicted age of independent test samples from 4 datasets (MAE = 5.15 years, Pearson

r=0.925

, Spearman

\rho=0.901

,

R^{2}=0.854

). (c): Scatter plot of validation (1,119 scans from 276 subjects) on ADNI only (5,881 scans from 1,378 total subjects). (d): Scatter plot of validation (223 out of 1,113 subjects) on ABIDE-II only. (e): Bias-corrected proportion of AD and AD/MCI subjects with extreme positive aging (

\textit{BAG}>1\textit{std.dev.}

) and extreme negative aging (

\textit{BAG}<-1\textit{std.dev.}

) visualized. (f) Bias-corrected proportion of AD/MCI subjects with extreme positive aging (

\textit{BAG}>1\textit{std.dev.}

) visualized against other AD/MCI subjects in ADNI. (g) Bias-corrected proportion of ASD subjects with extreme positive aging (

\textit{BAG}>1\textit{std.dev.}

) visualized against other ASD subjects in ABIDE-II.

Method	MAE	Pearson $r$	Spearman $\rho$	$R^{2}$
3D-ResNet (baseline)	5.55	0.96	0.92	0.88
3D-ViT (baseline)	6.37	0.81	0.87	0.78
SFCN [ 24 ]	5.03	0.96	0.95	0.91
Two-Stage Age
Network (TSAN) [ 67 ]	3.72	0.92	0.98	0.92
TSAN 1 st -stage	4.23	0.92	0.97	0.91

Table 2: Comparison of the proposed BrainRotViT model with existing methods on validation data. See Supplementary Section 4 for further details.

Cohort	$n$	MAE	$R^{2}$	Pearson $r$	Spearman $\rho$
SALD	494	5.35	0.850	0.925	0.906
AgeRisk	186	5.12	0.880	0.940	0.936
SUDMEX	135	4.87	0.350	0.740	0.760
TrueCrime	133	4.79	0.500	0.810	0.512
Overall	948	5.15	0.854	0.925	0.901

Table 3: Independent cohort evaluation of BrainRotViT. Each external dataset was excluded from training and used only for testing.

Dataset	Ours	TSAN [ 67 ]	SFCN [ 24 ]	3DResNet
SALD	5.35	5.91	5.48	6.94
TrueCrime	4.79	5.18	6.09	12.71
AgeRisk	5.12	5.57	7.20	8.85
SUDMEX	4.87	5.29	7.12	10.26
Overall	5.15	5.65	6.14	8.59

Table 4: Independent cohort performance comparison (MAE, years) between BrainRotViT and representative baseline and state-of-the-art methods. Lower values indicate better performance.

Model	Training Time (h)	Trainable Params	GFLOPs /sample	Peak VRAM
BrainRotViT
(CNN head)	4	50.4M	1.6	0.23 GB
BrainRotViT
(Transformer)	12	0.33M	$\approx$ 1,098 $\dagger$	1.38 GB ‡
SFCN	20	2.96M	131.7	8.03 GB ‡
3D-ResNet	24	64.2M	253.4	3.98 GB ‡

Table 5: Comparison of approximate training times, model size, and computational cost across different models. All models were trained on identical hardware settings (RTX 3060, Ryzen 3700X CPU, 16 GB RAM).

Figure 4: Model Interpretation. (a): Example of per-slice attention map extracted from vision transformer. (b) Mid-slice section views of fused 3D attention map along axial, coronal, and sagittal planes (from left to right). (c): Saliency map of trained residual CNN extracted through guided back propagation. (d)–(k): 3D attention map previews by age range.

Component	Alternatives tested	Selected	Alt. MAE
ViT pretraining
ViT backbone	ViT-small, ViT-large, DeiT [ 101 ] , DeiT-tiny, RoPE ViT [ 102 ] , MaxViT [ 103 ] , Swin-tiny [ 104 ]	ViT-base	3.48–3.71
Age–sex group width	5 and 20 years	10 years	4.72–5.26
Slices per volume	16 and 64	32	3.77–3.95
Age/sex supervision	Age classes only; age classes + FiLM sex input; direct age regression (with or without FiLM sex input); no sex input in CNN head	Composite age–sex classes, sex fused in CNN head	3.56–7.06
Regression head

Table 6: Summary of ablation experiments. Each row varies one component of the proposed model; the selected configuration reaches a validation MAE of 3.43 years in every row. Alt. MAE is the range (or value) of validation MAE, in years, over the alternatives tested.

Figure 5: Ablation study results. (a) Comparison of ViT backbone architectures; (b) Impact of age–sex group granularity; (c) Effect of slice count per volume; (d) Front-end architecture comparison; (e) Sex information integration methods; (f) Optimal depth of convolutional blocks; (g) Kernel size configurations; (h) Activation function comparison; (i) Sex fusion effectiveness; (j) Learning rate sensitivity; (k) Impact of residual connections; (l) Comparison of 2D and 1D convolutional front-ends.

Figure S1 : Validation performance stratified by demographic and clinical subgroups. Bars show MAE with 95% bootstrap confidence intervals for (A) age decade, (B) sex, (C) diagnosis, and (D) acquisition site. Prediction error is lowest for cognitively normal participants and increases in the youngest and oldest age groups, consistent with regression-to-the-mean effects. Sites with fewer than 10 validation subjects were pooled into “OTHER”.

Figure S2 : Bland–Altman analysis of predicted versus chronological age on the ADNI validation cohort. The mean signed bias is close to zero, indicating little overall systematic error, while the 95% limits of agreement (approximately

\pm 8.8

years) illustrate the spread of individual prediction errors.

Figure S3 : Residual age (predicted age

-

chronological age) plotted against chronological age. A negative slope (

-0.38

years/year; correlation

r=-0.62

) indicates regression-to-the-mean behavior, with overestimation in younger participants and underestimation in older participants. This age-dependent bias motivates the age-adjusted analyses reported in the main text.

Stratum Type	Stratum	$n$	MAE (95% CI)	RMSE	Bias
Overall	All	1119	3.517 (3.356–3.683)	4.509	-0.158
Age decade	50s	32	6.085 (5.193–6.927)	6.561	6.056
Age decade	60s	208	3.629 (3.108–4.164)	5.220	3.098
Age decade	70s	620	2.921 (2.762–3.090)	3.623	0.123
Age decade	80s	230	4.027 (3.699–4.353)	4.787	-3.664
Age decade	90s	29	8.587 (7.839–9.462)	8.891	-8.587

Table S1: Stratified validation performance on the ADNI cohort. Mean absolute error (MAE) with 95% bootstrap confidence intervals, root mean squared error (RMSE), and signed bias are reported for the overall cohort and stratified by age decade, sex, diagnosis, and acquisition site. Sites with fewer than 10 validation subjects were pooled into “OTHER” to ensure stable estimates.

Analysis	Uncorrected (crude)	(a) Corrected BAG	(b) Corrected + adjusted
AD/MCI, High vs rest (threshold OR)	5.49 [3.74–8.06]	4.17 [3.04–5.74], $p=9\times 10^{-20}$	(secondary) 4.17
AD/MCI, High vs Low (threshold OR)	6.59 [4.03–10.77]	6.96 [4.04–11.99], $p=3\times 10^{-13}$	(secondary) 6.96
AD/MCI ~ BAG, adjusted (age+sex+site)	—	—	OR 1.72 per +1 SD [1.43–2.07], $p=9\times 10^{-9}$ (1.16/yr)
AD ~ BAG, adjusted (age+sex+site)	—	—	OR 2.28 per +1 SD [1.73–3.00], $p=5\times 10^{-9}$

Table 16

Figure S4 : Alignment and registration of the 3D attention volume to the AAL3 atlas. (a) and (b) show the raw attention volume prior to registration; (c) shows the reference AAL3 atlas; (d) shows the final aligned overlay used to compute per-region attention scores. All the images were rendered using the open-source NIfTI viewing tool, MRIcroGL

Rank	Region	Max Intensity	Total Voxels
1	Calcarine	0.999609	32921
2	Precentral	0.999533	27058
3	Occipital	0.999030	79832
4	Cuneus	0.998887	40388
5	Temporal	0.998687	211174
6	Fusiform	0.998079	69355

Table S3 : Ranked ROIs by maximum attention intensity from the aligned attention volume overlapped with the AAL3 atlas.

Figure S5 : Model-parameter randomization sanity check. The cascade curve shows the Spearman correlation between the original and randomized-model attention maps as ViT layers are progressively randomized from the top layer down to full randomization, alongside representative attention maps at each stage. A comparison with CNN-only randomization is also shown, illustrating that the fused saliency map is dominated by the ViT attention component.

Figure S6 : Inter-subject variability of attention maps. Panels show the mean, standard deviation, and coefficient of variation (CV) computed across per-subject unfused central-slice attention maps, along with a histogram of voxel-level CV values across the validation cohort.

Figure S7 : Split-half stability analysis. The two panels show mean attention maps computed independently from two randomly partitioned halves of the validation cohort (

n=50

each), with the corresponding voxel-wise scatter plot showing agreement between the two halves (Spearman

\rho=0.942

, Pearson

r=0.943

).

	3D-ResNet	3D-ViT	SFCN	Global–Local	Triamese-ViT	TSAN
Type	Baseline	Baseline	Reference	Reference	Reference	Reference
Backbone	ResNet-34 (3D)	ViViT tubelet	SFCN	VGG8 + transformer	MultiView ViT ( $\times 3$ )	ScaleDense (2-stage)
Key config	blocks [3,4,6,3]; ch. 64–512	patch $16^{2}$ , depth-patch 16; dim 512, 8 layers, 8 heads, MLP 2048	6 conv stages [32–256,64]; soft-label head	patch 64, step 32; 6 blocks, hidden 512, 8 heads, drop 0.5	emb 768, 10 layers, 12 heads, MLP 3072; patch $8^{2}$ , drop 0.1	$n_{f}{=}8$ , 5 blocks; scale attention; uses sex
Input	$160{\times}192{\times}160$	$160{\times}192{\times}160$	$160{\times}192{\times}160$	2D slices	3 orthogonal views	MNI vol. + sex
Loss	MSE	MSE	soft-label KL (bin 1 yr, $\sigma{=}1$ )	L1 (MAE)	L1 (MAE)	MSE + ranking ( $\lambda{=}10$ , $\beta{=}1$ )
Optimiser	Adam (amsgrad)	AdamW	SGD ( $m{=}0.9$ )	Adam	Adam (amsgrad)	Adam

Table S4 : Architecture and optimisation hyperparameters for all competing methods. The 3D-ResNet and 3D-ViT baselines were implemented and tuned in this work; the remaining methods use their authors’ reference configurations.

Baseline	Search space	Budget	Selection metric
3D-ResNet	depth $\in\{$ 18, 34, 50 $\}$ ; lr $\in\{3{\times}10^{-4},1{\times}10^{-3},3{\times}10^{-3}\}$ ; weight decay $\in\{1{\times}10^{-4},5{\times}10^{-4}\}$ ; batch size $\in\{2,4\}$	$\sim$ 10 configs, 1 seed	validation MAE (best checkpoint)
3D-ViT	dim $\in\{384,512\}$ ; depth $\in\{6,8,12\}$ ; lr $\in\{5{\times}10^{-5},1{\times}10^{-4},3{\times}10^{-4}\}$ ; weight decay $\in\{1{\times}10^{-4},1{\times}10^{-2}\}$	$\sim$ 10 configs, 1 seed	validation MAE (best checkpoint)

Table S5 : Hyperparameter search space and tuning budget for the two baselines. Selection minimises validation MAE.

Stratum		$n$	MAE [95% CI]	RMSE	bias	$r/\rho$
Overall	all	1022	3.43 [3.24, 3.63]	4.66	$-0.27$	0.98/0.97
Age decade	10s	141	1.53 [1.30, 1.81]	2.14	$+0.37$	0.86/0.79
	20s	198	2.63 [2.29, 2.97]	3.58	$+0.74$	0.64/0.66
	30s	115	4.09 [3.49, 4.67]	5.18	$-0.26$	0.41/0.42
	40s	92	5.07 [4.21, 5.97]	6.61	$-0.26$	0.41/0.42
	50s	103	4.20 [3.60, 4.83]	5.21	$+0.20$	0.48/0.49

Table S6 : Validation-set accuracy stratified by age decade (

90+

merged into

80+

), age range, sex, dataset (scanner/site cohort), and diagnosis (ABIDE, ADNI). MAE and its

95\%

bootstrap CI are in years; bias is the mean signed error;

r

is Pearson and

\rho

Spearman.

Figure S8 : Validation MAE stratified by (a) age decade, (b) age range, (c) sex, and (d) dataset/scanner-site cohort. Error bars are

95\%

bootstrap CIs; the dashed line marks the pooled MAE and per-bar labels give the sample count.

Figure S9 : Validation MAE by diagnosis for the two labelled cohorts: (a) ABIDE (ASD vs. Control) and (b) ADNI (CN, MCI, AD). Error bars are

95\%

bootstrap CIs; the dashed line marks the pooled MAE.

Figure S10 : (a) Predicted vs chronological age with identity and fitted line. (b) Binned calibration curve (mean predicted age

\pm

SD per

5

-year bin).

Figure S11 : (a) Residual distribution. (b) Residual vs chronological age with linear and LOWESS trends; the trend bends negative beyond

\sim

70 years, indicating age-dependent bias concentrated at the extremes.

Figure S12 : Bland–Altman analysis of predicted vs chronological age. Solid line: mean bias (

-0.27

yr); dashed lines:

95\%

limits of agreement (

[-9.39,+8.85]

yr).

Figure S13 : Brain-age gap versus chronological age (a) before and (b) after age-bias correction (trend fit on the training set).

Dataset	$N_{\text{original}}$	$N_{\text{preprocessed}}$	Excluded	Failure rate
Training cohorts
ADNI	5896	5881	15	0.3%
IXI	581	565	16	2.8%
ABIDE-II	1114	1114	0	0.0%
DLBS	315	315	0	0.0%
COBRE	148	146	2	1.4%

Table S7 : Per-cohort preprocessing failure rates for the training cohorts and the held-out external test cohorts.

N_{\text{original}}

is the number of raw T1-weighted volumes provided;

N_{\text{preprocessed}}

is the number retained after skull stripping, N4 correction, and metadata assembly. Excluded scans failed preprocessing; no other exclusion criteria were applied.

Backbone	MAE
ViT-base (ours)	3.43
ViT-small	3.67
ViT-large	3.71
DeiT	3.59
DeiT-tiny	3.66
RoPE ViT	3.62

Table S8 : Effect of different ViT variants used as the frozen encoder.

Group Width (years)	MAE
5	4.72
10 (ours)	3.43
20	5.26

Table S9 : Effect of age–sex group granularity used during ViT pretraining.

Slices per Volume	MAE
16	3.95
32 (ours)	3.43
64	3.77

Table S10 : Effect of the number of evenly spaced slices sampled per volume during ViT pretraining.

Front End	MAE
CNN with residual blocks (ours)	3.43
ViT	6.51
MLP only	8.12
Bidirectional GRU	3.76
Bidirectional Mamba	3.81
Sequential Transformer	3.76

Table S11 : Effect of different architectures placed after the ViT encoder.

Model Variant (ViT)	MAE
With age–sex composite classes (ours)	3.43
With age classes only (no sex)	5.07
With age classes + sex input via FiLM	5.76
With direct age regression head (no sex)	5.35
With regression head + sex input via FiLM	7.06
With sex-aware encoder but CNN without sex input	3.56

Table S12 : Effect of methodological variants using ViT-Base encoder and best CNN regression head. All results are reported on the validation set.

Model Variant	MAE
2D CNN	3.43
1D CNN	3.64

Table S13 : Comparison of 2D and 1D CNN front-ends on validation performance (seed 701; five-seed statistics in Section 7.4 ).

Number of Conv Blocks	MAE
1	4.23
2	3.76
3 (ours)	3.43
4	3.49
5	3.54
6	3.73

Table S14 : Effect of varying the number of convolutional blocks in the CNN regression head.

Kernel Sizes	MAE
(10,60),(5,15),(2,6) (ours)	3.43
(12,80),(6,20),(3,8)	3.47
(6,40),(4,12),(2,6)	3.82
(8,50),(5,15),(1,4)	3.54
Dilated variant	3.50

Table S15 : Effect of different kernel size configurations in the residual CNN.

Activation	MAE
SiLU (ours)	3.43
ReLU	3.48
Leaky ReLU	3.52
GELU	3.71

Table S16 : Effect of different activation functions in the CNN and FC layers.

Configuration	Validation MAE (years)
With Sex Fusion	3.43
Without Sex Fusion	3.56

Table S17 : Effect of incorporating biological sex information in the CNN regression head.

Learning Rate	$1\times 10^{-4}$	$5\times 10^{-4}$ (ours)	$1\times 10^{-3}$	$5\times 10^{-3}$	$1\times 10^{-2}$
MAE	4.52	3.43	3.83	5.10	6.13
r	0.96	0.98	0.97	0.94	0.92
R 2	0.90	0.96	0.92	0.86	0.80

Table S18 : Effect of learning rate on model performance (validation set).

Model Variant	MAE
With Residual Connections	3.43
Without Residual Connections	3.70

Table S19 : Impact of removing residual connections in the CNN on validation performance.

Seed	With residual	Without residual
700	3.51	3.92
701	3.43	3.70
702	3.48	3.78
703	3.56	5.01
704	3.50	4.53
Mean $\pm$ SD	$3.496\pm 0.047$	$4.188\pm 0.563$

Table S20 : MAE (months) with and without residual connections, across 5 seeds.

Seed	CNN front-end	Bidirectional GRU	Bidirectional Mamba	Sequential Transformer
700	3.51	3.83	3.805	3.744
701	3.43	3.76	3.814	3.758
702	3.48	3.81	3.599	3.774
703	3.56	3.78	3.688	3.654
704	3.50	3.73	3.642	3.880
Mean $\pm$ SD	$3.496\pm 0.047$	$3.782\pm 0.040$	$3.710\pm 0.097$	$3.762\pm 0.081$

Table S21 : MAE (years) for alternative front-end architectures across 5 seeds.

Seed	Sex incorporated	Sex not incorporated
700	3.51	3.71
701	3.43	3.56
702	3.48	3.74
Mean $\pm$ SD	$3.473\pm 0.040$	$3.670\pm 0.096$

Table S22 : MAE (months) with and without sex incorporated, across 3 seeds.

Seed	2D CNN	1D CNN
700	3.51	3.55
701	3.43	3.64
702	3.48	3.61
703	3.56	3.59
704	3.50	3.58
Mean $\pm$ SD	$3.496\pm 0.047$	$3.593\pm 0.039$

Table S23 : MAE (years) for 2D CNN vs. 1D CNN, across 5 seeds.

Cohort	Available	Used	Unique subjects	Selection rule
ADNI	5,881	900	900	1 scan/subject; youngest 900 subjects (50.5–76.4 y)
ABIDE-II	1,114	750	749	oldest 750 scans (10–64 y)
FCON1000	1,355	763	763	all processed scans with age and sex recorded
CoRR	3,357	377	256	all scans meeting the MPRAGE T1w criterion
OASIS-1	416	300	294	oldest 300 scans (25–96 y)

Table S24: Cohorts from which only part of the available data was used. “Available” is the number of scans in the public release; “Used” is the number of scans in the training pool.

Figure S14 : Age range of each cohort after subset selection, sorted by median age. Boxes show the interquartile range, whiskers extend to 1.5 IQR, and points are individual scans.

Figure S15 : Age distribution of each cohort after subset selection (5-year bins; y-axes scaled independently).

Figure S16 : Scans per age decade by sex. The decade–sex combinations are the Age_Group classes used as targets in the ViT stage. Males outnumber females in the teens because of ABIDE-II; females outnumber males above 50.

\mathcal{L}_{\mathrm{CE}}=-\frac{1}{B}\sum_{i=1}^{B}\sum_{c=1}^{C}y_{i,c}\log\hat{y}_{i,c}

Algorithm 1 Proposed BrainRotViT Framework

BrainRotViT: Transformer-ResNet Hybrid for Explainable Modeling of Brain Aging from 3D sMRI

Abstract

Figures & tables

Brain Age Prediction

Interpretable Brain Age Prediction Using White Matter Fiber Orientation Distributions with Rotation-Equivariant Deep Learning

Enhancing brain age estimation with structural MRI and synthesized cerebral blood volume maps

Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI

Abstract

Brain Age Prediction

Explore similar work

Interpretable Brain Age Prediction Using White Matter Fiber Orientation Distributions with Rotation-Equivariant Deep Learning

Enhancing brain age estimation with structural MRI and synthesized cerebral blood volume maps

Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI