Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

Dexuan Ding; Yuankai Qi; Bogong Wang; Luping Zhou; Amin Beheshti

cs.CVSep 28, 2026

Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

Authors: Dexuan Ding, Yuankai Qi, Bogong Wang, Luping Zhou, Amin Beheshti

Organizations: Macquarie University · Australian National University · University of Sydney

Abstract

Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but their use on volumetric data requires bridging the mismatch between a 2D encoder and a 3D volume input. Existing methods typically encode slices independently and integrate their features afterwards. We introduce Multiscale Volumetric Reduction (MVR), a reduce-then-encode approach that compresses each anatomical view from (D) slices into (M << D) complementary 2D components before foundation-model encoding. MVR combines an uncentered-PCA base component derived from the original through-plane intensities with residual detail components constructed from multiscale spatial descriptors. The reduction is estimated from the training volumes without diagnostic labels or gradient-based optimization and remains fixed thereafter. The resulting components are independently processed by a shared frozen 2D foundation model and concatenated for linear probing. Under this frozen-encoder setting, MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to the evaluated 2D-to-3D adaptation methods and simple input-reduction baselines, while also generalizing strongly from ADNI to AIBL.

Figures & tables

Figure 1: Comparison of volumetric representation strategies for brain MRI. (a) Medical foundation models learn volumetric representations through dedicated 3D pretraining. (b) Approaches based on pretrained 2D foundation models encode $D$ selected 2D slices independently and integrate the resulting features after encoding, with method-specific adaptation where applicable. (c) Our MVR instead performs reduction in the input space, transforming each volume into $M\ll D$ dense 2D components before encoding with a shared frozen 2D foundation model. The resulting component features are concatenated to form the volume representation. and denote trainable and frozen modules.

Figure 2: Main architecture of our Multiscale Volumetric Reduction (MVR). (a) Base component extraction summarizes the original through-plane intensities into a dense base component and sampled base scores. (b) Multiscale residual expansion incorporates in-plane variation at multiple scales and produces $M-1$ complementary detail components after accounting for the base reference. (c) Encoding and classification applies MVR to the axial, coronal, and sagittal views, independently encodes the resulting components with a shared frozen 2D foundation model, and concatenates their features for classification.

Figure 3: Visualization of the multiscale decomposition and resulting MVR components. (a) Gaussian smoothing at increasing scales produces multiscale difference components $B_{0},\ldots,B_{4}$ , capturing in-plane variation at different spatial scales. (b) The base component $b_{v}$ summarizes the original through-plane intensities, while the detail components capture complementary residual multiscale variation. The first four detail components are shown.

Method	ADNI		OASIS		ABIDE
Method	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC ( Tak et al., 2026 )	$76.96\pm 3.94$	$70.34\pm 4.66$	$63.31\pm 9.15$	$58.22\pm 6.18$	$51.40\pm 3.14$	$51.48\pm 1.35$
BrainMVP ( Rui et al., 2025 )	$74.38\pm 2.70$	$68.61\pm 3.69$	$67.76\pm 5.22$	$64.31\pm 2.95$	$50.41\pm 4.11$	$50.78\pm 4.11$
3DINO ( Xu et al., 2025 )	$85.96\pm 2.17$	$78.14\pm 2.26$	$67.05\pm 2.58$	$63.01\pm 3.30$	$59.30\pm 2.43$	$55.30\pm 1.82$
MedicalNet ( Chen et al., 2019 )	$86.60\pm 1.48$	$78.10\pm 2.00$	$68.35\pm 4.99$	$\bm{64.91\pm 3.88}$	$62.37\pm 2.65$	$58.85\pm 2.25$
RAPTOR ( An et al., 2025 )	$80.26\pm 0.94$	$72.91\pm 2.18$	$64.26\pm 5.67$	$61.47\pm 5.26$	$62.95\pm 1.79$	$57.27\pm 2.22$

Table 1: Classification performance under the frozen-encoder setting on ADNI, OASIS, and ABIDE. Results are mean

\pm

sample standard deviation across five held-out folds; best and second-best means are highlighted in bold and underlined . † denotes the use of frozen DINOv3 encoder.

Method	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC ( Tak et al., 2026 )	$69.26\pm 2.16$	$60.14\pm 1.56$
BrainMVP ( Rui et al., 2025 )	$78.43\pm 0.77$	$66.51\pm 2.77$
3DINO ( Xu et al., 2025 )	$83.73\pm 0.93$	$72.19\pm 2.71$
MedicalNet ( Chen et al., 2019 )	$86.21\pm 0.25$	$71.45\pm 2.63$
RAPTOR ( An et al., 2025 )	$77.66\pm 1.42$	$62.95\pm 2.60$
AnyMC3D † ( Liu et al., 2026 )	$80.46\pm 0.95$	$65.61\pm 3.88$

Table 2: ADNI-to-AIBL external evaluation without AIBL-specific fitting or model selection. Results are mean

\pm

sample standard deviation across five ADNI-fold-trained models. Best and second-best means are highlighted in bold and underlined . † denotes the use of Frozen DINOv3 encoder.

Multiscale construction
Variant	ADNI		OASIS		ABIDE
Variant	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
w/o Gaussian scale space	$87.01\pm 2.29$	$78.34\pm 3.97$	$69.39\pm 2.21$	$61.11\pm 4.61$	$63.38\pm 1.37$	$59.17\pm 3.82$
Component composition
w/o base component in encoding	$86.47\pm 0.99$	$80.69\pm 2.35$	$68.37\pm 2.35$	$62.42\pm 2.82$	$66.02\pm 1.33$	$59.21\pm 2.40$
w/o detail components	$85.44\pm 2.37$	$75.68\pm 3.18$	$67.29\pm 2.62$	$60.32\pm 3.81$	$61.53\pm 3.94$	$57.94\pm 3.18$

Table 3: Component ablations of MVR. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. The variant w/o base component in encoding omits the base component from encoder inputs while retaining its use in residual construction. The variant w/o detail components retains only the base component.

Variant	FM encodings	ADNI		OASIS		ABIDE
Variant	FM encodings	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
$M=3$	$9$	$86.98\pm 2.18$	$77.76\pm 1.66$	$70.72\pm 2.46$	$65.22\pm 2.67$	$62.57\pm 2.36$	$59.18\pm 3.68$
$M=5$	$15$	$87.67\pm 1.96$	$80.64\pm 0.44$	$70.65\pm 2.92$	$63.16\pm 1.79$	$64.84\pm 1.90$	$57.66\pm 3.49$
$M=8$	$24$	$87.89\pm 1.27$	$80.19\pm 1.53$	$70.53\pm 2.50$	$63.51\pm 5.32$	$64.80\pm 0.86$	$59.33\pm 2.22$
$M=16$ (default)	$48$	$88.85\pm 1.22$	$80.92\pm 1.02$	$70.30\pm 2.10$	$64.49\pm 2.53$	$66.02\pm 1.41$	$61.03\pm 3.21$
$M=32$	$96$	$88.28\pm 1.37$	$80.01\pm 0.99$	$70.41\pm 2.02$	$63.56\pm 2.88$	$65.41\pm 1.51$	$58.04\pm 3.56$

Table 4: Effect of component count using separate encoding and concatenation across all three views. FM encodings are counted per volume. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

$P$	AUC $\uparrow$	BAcc $\uparrow$
$64$	$88.33\pm 1.63$	$80.84\pm 1.51$
$128$	$88.46\pm 1.45$	$79.99\pm 1.51$
$256$	$\bm{88.85\pm 1.22}$	$80.92\pm 1.02$
$512$	$88.80\pm 1.37$	$\bm{81.68\pm 0.87}$
All	$88.62\pm 1.46$	$80.59\pm 1.52$

Appendix

Table 5: Effect of the number of sampled through-plane vectors per training subject on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Retained views	ADNI		OASIS		ABIDE
Retained views	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$	AUC $\uparrow$	BAcc $\uparrow$
Axial	$86.80$	$79.22$	$68.80$	$62.79$	$65.86$	$60.77$
Coronal	$85.36$	$77.52$	$69.94$	$64.32$	$61.76$	$58.58$
Sagittal	$85.77$	$76.60$	$67.88$	$62.43$	$62.71$	$58.52$
Axial + coronal	$87.90$	$80.05$	$69.77$	$62.17$	$65.25$	$\bm{61.47}$
Axial + sagittal	$88.48$	$80.80$	$69.47$	$63.15$	$\underline{65.95}$	$59.92$

Appendix

Table 6: Effect of anatomical-view composition on classification performance. AUC and BAcc are reported on a

0

–

100

scale. Best results are highlighted in bold and second-best results are underlined.

Method	AUC $\uparrow$	BAcc $\uparrow$
BrainIAC	$79.65\pm 2.29$	$72.01\pm 1.20$
BrainMVP	$85.24\pm 3.38$	$75.25\pm 4.80$
3DINO	$87.01\pm 1.60$	$79.10\pm 0.94$
AnyMC3D	$\bm{92.93\pm 3.14}$	$\bm{84.56\pm 4.28}$
MVR (Ours)	$89.10\pm 1.16$	$81.15\pm 2.59$

Appendix

Table 7: Classification performance with LoRA adaptation on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Gaussian scales (mm)	AUC $\uparrow$	BAcc $\uparrow$
$\{1,2,4,8,16\}$	$88.85\pm 1.22$	$\bm{80.92\pm 1.02}$
$\{1,\sqrt{2},2,2\sqrt{2},4,4\sqrt{2},8,8\sqrt{2},16\}$	$\bm{88.93\pm 1.58}$	$\bm{80.92\pm 2.34}$

Appendix

Table 8: Sensitivity to the Gaussian scale configuration on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best means are in bold and second-best means are underlined .

Representation	AUC $\uparrow$	BAcc $\uparrow$
Block 6 only	$86.83\pm 1.61$	$78.61\pm 0.62$
Block 12 only	$88.31\pm 1.32$	$80.62\pm 1.99$
Block 6 + Block 12	$\mathbf{88.85\pm 1.22}$	$\mathbf{80.92\pm 1.02}$

Appendix

Table 9: Feature-layer ablation on ADNI. AUC and BAcc are reported on a

0

–

100

scale as mean

\pm

sample standard deviation across five held-out folds. Best results are highlighted in bold .

Explore similar work

Aug 8, 2026cs.CV

A continually expandable foundation model for brain MRI

Brain magnetic resonance imaging (MRI) is central to neuroscience and clinical assessment, but models are commonly developed for individual diseases, populations or imaging protocols. Foundation models promise more general representations, yet they are usually pretrained once and can lose earlier capabilities when updated with new data. Here we show that Alcmaeon, a three-dimensional brain MRI foundation model pretrained without manual labels on more than 425,000 volumes and derived imaging maps, can be expanded sequentially across clinical domains. Alcmaeon combines volumetric encoding and latent diffusion generation with Graph-Blueprint Pruning (GBP), which protects network modules important to earlier domains while leaving the remaining capacity trainable. Across expansion from healthy ageing and neurodegeneration to developmental, psychiatric and tumour imaging, GBP showed less forgetting than sequential adaptation and elastic weight consolidation across voxel-level reconstruction measures, with its largest advantage after adaptation to tumour imaging. The blueprints provided an inspectable record of how model capacity was protected and reused. Representations from different model levels supported image synthesis, disease classification, survival modelling and postoperative prediction, although no single representation was optimal for every task. These findings provide a route towards brain MRI foundation models that can grow with emerging data while retaining earlier capabilities.

Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li +8

1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom. +7

Functional Magnetic Resonance Imaging Electroencephalography Foundation Models Wireless Foundation Models Medical World Model Latent Diffusion Model Voxel+2

Ask PDF

Oct 15, 2025cs.CV

Scaling Vision Transformers for Functional MRI with Flat Maps

We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K hours of open fMRI data, and (2) we release the first open evaluation suite (Brainmarks) for fMRI foundation models. Our core innovation is simple: we adapt the Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a cortical flat map projection. We directly compare flat maps to both parcellation and volume-based representations. While each has its advantages, flat maps generally perform best. We perform the first systematic scaling analysis for fMRI and observe strict power law scaling, albeit with limits. Finally, we use Brainmarks to do controlled benchmark comparisons. On subject-level trait prediction, we report a challenging null result: no single model achieves clear state-of-the-art performance. Moreover, all models struggle to outperform a simple functional connectivity baseline. On cognitive state decoding, we observe more robust performance, and in this setting our CortexMAE family outperforms prior models by a large margin. Code, models, and datasets are available at https://github.com/MedARC-AI/CortexMAE and https://github.com/MedARC-AI/Brainmarks.

Connor Lane, Mihir Tripathy, Leema Krishna Murali +15

MedARC · Sophont · Baylor College of Medicine +3

Functional Magnetic Resonance Imaging Vision Transformer Transformer Architectures

Ask PDF

Jun 9, 2026eess.IV

FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI

The success of large-scale deep learning models in neuroscience is fundamentally constrained by severe data heterogeneity. Native fMRI data aggregated from diverse sources exhibit substantial variation in both spatial and temporal resolutions. Consequently, most existing frameworks rely on lengthy, rigid preprocessing pipelines that enforce uniformity across datasets. This practice introduces two critical limitations: (1) potential degradation of subject-specific anatomical information; (2) significant computational overhead, often requiring hours of processing per subject. Here, we propose FlexiBrain, a resolution-agnostic voxel-level encoding framework for native fMRI based on Mamba-JEPA. FlexiBrain defines patch sizes in real-world physical units and employs a dynamic patch resizing, thereby bypassing destructive spatial standardization while enabling direct ingestion of data in native space. We instantiate the framework using an efficient Mamba-JEPA backbone to model high-dimensional 4D fMRI signals. Across five diverse downstream neuroscience tasks, FlexiBrain consistently outperforms recent state-of-the-art methods, achieving gains of up to 12 percentage points without external data augmentation. Importantly, FlexiBrain functions as a seamless plug-in module, substantially reducing preprocessing costs and accelerating the development of robust voxel-level fMRI foundation models. Code is available at https://github.com/OneMore1/FlexiBrain.

Mo Wang, Wenhao Ye, Junfeng Xia +3

Southern University of Science and Technology · University of Warwick

Functional Magnetic Resonance Imaging Deep Learning Framework Voxel

Ask PDF