Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

Dexuan Ding; Yuankai Qi; Bogong Wang; Luping Zhou; Amin Beheshti

cs.CVSep 28, 2026

Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

Authors: Dexuan Ding, Yuankai Qi, Bogong Wang, Luping Zhou, Amin Beheshti

Organizations: Macquarie University · Australian National University · University of Sydney

Abstract

Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but their use on volumetric data requires bridging the mismatch between a 2D encoder and a 3D volume input. Existing methods typically encode slices independently and integrate their features afterwards. We introduce Multiscale Volumetric Reduction (MVR), a reduce-then-encode approach that compresses each anatomical view from (D) slices into (M << D) complementary 2D components before foundation-model encoding. MVR combines an uncentered-PCA base component derived from the original through-plane intensities with residual detail components constructed from multiscale spatial descriptors. The reduction is estimated from the training volumes without diagnostic labels or gradient-based optimization and remains fixed thereafter. The resulting components are independently processed by a shared frozen 2D foundation model and concatenated for linear probing. Under this frozen-encoder setting, MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to the evaluated 2D-to-3D adaptation methods and simple input-reduction baselines, while also generalizing strongly from ADNI to AIBL.

Figures & tables

Figure 1: Comparison of volumetric representation strategies for brain MRI. (a) Medical foundation models learn volumetric representations through dedicated 3D pretraining. (b) Approaches based on pretrained 2D foundation models encode $D$ selected 2D slices independently and integrate the resulting features after encoding, with method-specific adaptation where applicable. (c) Our MVR instead performs reduction in the input space, transforming each volume into $M\ll D$ dense 2D components before encoding with a shared frozen 2D foundation model. The resulting component features are concatenated to form the volume representation. and denote trainable and frozen modules.

Figure 2: Main architecture of our Multiscale Volumetric Reduction (MVR). (a) Base component extraction summarizes the original through-plane intensities into a dense base component and sampled base scores. (b) Multiscale residual expansion incorporates in-plane variation at multiple scales and produces $M-1$ complementary detail components after accounting for the base reference. (c) Encoding and classification applies MVR to the axial, coronal, and sagittal views, independently encodes the resulting components with a shared frozen 2D foundation model, and concatenates their features for classification.

Figure 3: Visualization of the multiscale decomposition and resulting MVR components. (a) Gaussian smoothing at increasing scales produces multiscale difference components $B_{0},\ldots,B_{4}$ , capturing in-plane variation at different spatial scales. (b) The base component $b_{v}$ summarizes the original through-plane intensities, while the detail components capture complementary residual multiscale variation. The first four detail components are shown.

Method	ADNI		OASIS		ABIDE
Method	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC ( Tak et al., 2026 )	$76.96\pm 3.94$	$70.34\pm 4.66$	$63.31\pm 9.15$	$58.22\pm 6.18$	$51.40\pm 3.14$	$51.48\pm 1.35$
BrainMVP ( Rui et al., 2025 )	$74.38\pm 2.70$	$68.61\pm 3.69$	$67.76\pm 5.22$	$64.31\pm 2.95$	$50.41\pm 4.11$	$50.78\pm 4.11$
3DINO ( Xu et al., 2025 )	$85.96\pm 2.17$	$78.14\pm 2.26$	$67.05\pm 2.58$	$63.01\pm 3.30$	$59.30\pm 2.43$	$55.30\pm 1.82$
MedicalNet ( Chen et al., 2019 )	$86.60\pm 1.48$	$78.10\pm 2.00$	$68.35\pm 4.99$	$\bm{64.91\pm 3.88}$	$62.37\pm 2.65$	$58.85\pm 2.25$
RAPTOR ( An et al., 2025 )	$80.26\pm 0.94$	$72.91\pm 2.18$	$64.26\pm 5.67$	$61.47\pm 5.26$	$62.95\pm 1.79$	$57.27\pm 2.22$

Table 1: Classification performance under the frozen-encoder setting on ADNI, OASIS, and ABIDE. Results are mean

\pm

sample standard deviation across five held-out folds; best and second-best means are highlighted in bold and underlined . † denotes the use of frozen DINOv3 encoder.

Method	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC ( Tak et al., 2026 )	$69.26\pm 2.16$	$60.14\pm 1.56$
BrainMVP ( Rui et al., 2025 )	$78.43\pm 0.77$	$66.51\pm 2.77$
3DINO ( Xu et al., 2025 )	$83.73\pm 0.93$	$72.19\pm 2.71$
MedicalNet ( Chen et al., 2019 )	$86.21\pm 0.25$	$71.45\pm 2.63$
RAPTOR ( An et al., 2025 )	$77.66\pm 1.42$	$62.95\pm 2.60$
AnyMC3D † ( Liu et al., 2026 )	$80.46\pm 0.95$	$65.61\pm 3.88$

Table 2: ADNI-to-AIBL external evaluation without AIBL-specific fitting or model selection. Results are mean

\pm

sample standard deviation across five ADNI-fold-trained models. Best and second-best means are highlighted in bold and underlined . † denotes the use of Frozen DINOv3 encoder.

Multiscale construction
Variant	ADNI		OASIS		ABIDE
Variant	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
w/o Gaussian scale space	$87.01\pm 2.29$	$78.34\pm 3.97$	$69.39\pm 2.21$	$61.11\pm 4.61$	$63.38\pm 1.37$	$59.17\pm 3.82$
Component composition
w/o base component in encoding	$86.47\pm 0.99$	$80.69\pm 2.35$	$68.37\pm 2.35$	$62.42\pm 2.82$	$66.02\pm 1.33$	$59.21\pm 2.40$
w/o detail components	$85.44\pm 2.37$	$75.68\pm 3.18$	$67.29\pm 2.62$	$60.32\pm 3.81$	$61.53\pm 3.94$	$57.94\pm 3.18$

Table 3: Component ablations of MVR. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. The variant w/o base component in encoding omits the base component from encoder inputs while retaining its use in residual construction. The variant w/o detail components retains only the base component.

Variant	FM encodings	ADNI		OASIS		ABIDE
Variant	FM encodings	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
$M=3$	$9$	$86.98\pm 2.18$	$77.76\pm 1.66$	$70.72\pm 2.46$	$65.22\pm 2.67$	$62.57\pm 2.36$	$59.18\pm 3.68$
$M=5$	$15$	$87.67\pm 1.96$	$80.64\pm 0.44$	$70.65\pm 2.92$	$63.16\pm 1.79$	$64.84\pm 1.90$	$57.66\pm 3.49$
$M=8$	$24$	$87.89\pm 1.27$	$80.19\pm 1.53$	$70.53\pm 2.50$	$63.51\pm 5.32$	$64.80\pm 0.86$	$59.33\pm 2.22$
$M=16$ (default)	$48$	$88.85\pm 1.22$	$80.92\pm 1.02$	$70.30\pm 2.10$	$64.49\pm 2.53$	$66.02\pm 1.41$	$61.03\pm 3.21$
$M=32$	$96$	$88.28\pm 1.37$	$80.01\pm 0.99$	$70.41\pm 2.02$	$63.56\pm 2.88$	$65.41\pm 1.51$	$58.04\pm 3.56$

Table 4: Effect of component count using separate encoding and concatenation across all three views. FM encodings are counted per volume. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

$P$	AUC $\uparrow$	BAcc $\uparrow$
$64$	$88.33\pm 1.63$	$80.84\pm 1.51$
$128$	$88.46\pm 1.45$	$79.99\pm 1.51$
$256$	$\bm{88.85\pm 1.22}$	$80.92\pm 1.02$
$512$	$88.80\pm 1.37$	$\bm{81.68\pm 0.87}$
All	$88.62\pm 1.46$	$80.59\pm 1.52$

Appendix

Table 5: Effect of the number of sampled through-plane vectors per training subject on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Retained views	ADNI		OASIS		ABIDE
Retained views	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
Axial	$86.80$	$79.22$	$68.80$	$62.79$	$65.86$	$60.77$
Coronal	$85.36$	$77.52$	$69.94$	$64.32$	$61.76$	$58.58$
Sagittal	$85.77$	$76.60$	$67.88$	$62.43$	$62.71$	$58.52$
Axial + coronal	$87.90$	$80.05$	$69.77$	$62.17$	$65.25$	$\bm{61.47}$
Axial + sagittal	$88.48$	$80.80$	$69.47$	$63.15$	$\underline{65.95}$	$59.92$

Appendix

Table 6: Effect of anatomical-view composition on classification performance. AUC and BAcc are reported on a

0

–

100

scale. Best results are highlighted in bold and second-best results are underlined.

Method	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC	$79.65\pm 2.29$	$72.01\pm 1.20$
BrainMVP	$85.24\pm 3.38$	$75.25\pm 4.80$
3DINO	$87.01\pm 1.60$	$79.10\pm 0.94$
AnyMC3D	$\bm{92.93\pm 3.14}$	$\bm{84.56\pm 4.28}$
MVR (Ours)	$89.10\pm 1.16$	$81.15\pm 2.59$

Appendix

Table 7: Classification performance with LoRA adaptation on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Gaussian scales (mm)	AUC $\uparrow$	BAcc $\uparrow$
$\{1,2,4,8,16\}$	$88.85\pm 1.22$	$\bm{80.92\pm 1.02}$
$\{1,\sqrt{2},2,2\sqrt{2},4,4\sqrt{2},8,8\sqrt{2},16\}$	$\bm{88.93\pm 1.58}$	$\bm{80.92\pm 2.34}$

Appendix

Table 8: Sensitivity to the Gaussian scale configuration on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Representation	AUC $\uparrow$	BAcc $\uparrow$
Block 6 only	$86.83\pm 1.61$	$78.61\pm 0.62$
Block 12 only	$88.31\pm 1.32$	$80.62\pm 1.99$
Block 6 + Block 12	$\mathbf{88.85\pm 1.22}$	$\mathbf{80.92\pm 1.02}$

Appendix

Table 9: Feature-layer ablation on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best results are highlighted in bold .

Explore similar work

A continually expandable foundation model for brain MRI
Aug 8, 2026Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li +8Functional Magnetic Resonance Imaging Electroencephalography Foundation Models
1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom. +7
Scaling Vision Transformers for Functional MRI with Flat Maps
Oct 15, 2025Connor Lane, Mihir Tripathy, Leema Krishna Murali +15Functional Magnetic Resonance Imaging Vision Transformer
MedARC · Sophont · Baylor College of Medicine +3
FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI
Jun 9, 2026Mo Wang, Wenhao Ye, Junfeng Xia +3Functional Magnetic Resonance Imaging Deep Learning Framework
Southern University of Science and Technology · University of Warwick

Aug 8, 2026, cs.CVJ/K move · Enter open · S save

A continually expandable foundation model for brain MRI

Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li, Hao Chen+7

1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom.+7

Functional Magnetic Resonance Imaging Electroencephalography Foundation Models Wireless Foundation Models Medical World Model Latent Diffusion Model Voxel+2

Open paper AskPDF arXiv

Brain magnetic resonance imaging (MRI) is central to neuroscience and clinical assessment, but models are commonly developed for individual diseases, populations or imaging protocols. Foundation models promise more general representations, yet they are usually pretrained once and can lose earlier capabilities when updated with new data. Here we show that Alcmaeon, a three-dimensional brain MRI foundation model pretrained without manual labels on more than 425,000 volumes and derived imaging maps, can be expanded sequentially across clinical domains. Alcmaeon combines volumetric encoding and latent diffusion generation with Graph-Blueprint Pruning (GBP), which protects network modules important to earlier domains while leaving the remaining capacity trainable. Across expansion from healthy ageing and neurodegeneration to developmental, psychiatric and tumour imaging, GBP showed less forgetting than sequential adaptation and elastic weight consolidation across voxel-level reconstruction measures, with its largest advantage after adaptation to tumour imaging. The blueprints provided an inspectable record of how model capacity was protected and reused. Representations from different model levels supported image synthesis, disease classification, survival modelling and postoperative prediction, although no single representation was optimal for every task. These findings provide a route towards brain MRI foundation models that can grow with emerging data while retaining earlier capabilities.

Abstract

Explore similar work

A continually expandable foundation model for brain MRI

Scaling Vision Transformers for Functional MRI with Flat Maps

FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI