eess.IVFeb 27, 2026

Extending 2D foundational DINOv3 representations to 3D segmentation of neonatal brain MR images

Authors: Annayah Usman, Behraj Khan, Tahir Qasim Syed

Organizations: Institute of Business Administration Karachi

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Precise volumetric delineation of hippocampal structures is essential for quantifying neurodevelopmental trajectories in pre-term and term infants, where subtle morphological variations may carry prognostic significance. While foundation encoders trained on large-scale visual data offer discriminative representations, their 2D formulation is a limitation with respect to the 33D organization of brain anatomy. We propose a volumetric segmentation strategy that reconciles this tension through a structured window-based disassembly-reassembly mechanism: the global MRI volume is decomposed into non-overlapping 3D windows or sub-cubes, each processed via a separate decoding arm built upon frozen high-fidelity features, and subsequently reassembled prior to a ground-truth correspendence using a dense-prediction head. This architecture preserves constant a decoder memory footprint while forcing predictions to lie within an anatomically consistent geometry. Evaluated on the ALBERT dataset for hippocampal segmentation, the proposed approach achieves a Dice score of 0.65 for a single 3D window. The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.

Figures & tables

Figure 1: (a) A 3D MRI volume is decomposed into sub-cubes, encoded using a frozen 2D DINOv3 ViT with depth embedding, decoded via a lightweight head, and reassembled for global supervision. (b) Two-pass training: detached global loss computation followed by gradient-correct sub-cube backpropagation.
ExperimentALBERT (n=20)
DSCIoUVol. Error (%)
8 sub-cubes ( 64364^{3} )0.35180.214830.42
1 sub-cube ( 1283128^{3} )0.65140.485114.61
Table 1: Comparison on the ALBERT dataset. The multi-cube setting partitions the volume into eight 64364^{3} sub-cubes, while the single-cube setting processes the full 1283128^{3} crop. Results are reported for fold 1.
Figure 2: Qualitative comparison of hippocampus segmentation. The single-cube model shows improved anatomical coherence and boundary accuracy compared to the multi-cube setting.
ExperimentALBERT (n=20)
DSCIoUVol. Error (%)
Baseline0.65140.485114.61
No depth embedding0.65280.487211.71
Single-scale decoding0.35850.219831.75
Table 2: Ablation on the ALBERT dataset. These experiments are performed using the single-cube setting where a single full 1283128^{3} crop is processed. Results are reported for fold 1 only.
More in this topic

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 768 papers

Explore similar work

CardsList
Mar 24, 2026

NeuroSeg Meets DINOv3: Transferring 2D Self-Supervised Visual Priors to 3D Neuron Segmentation via DINOv3 Initialization

2D visual foundation models, such as DINOv3, a self-supervised model trained on large-scale natural images, have demonstrated strong zero-shot generalization, capturing both rich global context and fine-grained structural cues. However, an analogous 3D foundation model for downstream volumetric neuroimaging remains lacking, largely due to the challenges of 3D image acquisition and the scarcity of high-quality annotations. To address this gap, we propose to adapt the 2D visual representations learned by DINOv3 to a 3D biomedical segmentation model, enabling more data-efficient and morphologically faithful neuronal reconstruction. Specifically, we design an inflation-based adaptation strategy that inflates 2D filters into 3D operators, preserving semantic priors from DINOv3 while adapting to 3D neuronal volume patches. In addition, we introduce a topology-aware skeleton loss to explicitly enforce structural fidelity of graph-based neuronal arbor reconstruction. Extensive experiments on four neuronal imaging datasets, including two from BigNeuron and two public datasets, NeuroFly and CWMBS, demonstrate consistent improvements in reconstruction accuracy over SoTA methods, with average gains of 2.9% in Entire Structure Average, 2.8% in Different Structure Average, and 3.8% in Percentage of Different Structure. Code: https://github.com/yy0007/NeurINO.

Yik San Cheng, Runkai Zhao, Weidong Cai

The University of Sydney

3D Medical Image SegmentationNeuroimagingVision Foundation Model Adaptation3D CNNsTopology-Preserving Image Segmentation
AskPDF
Sep 28, 2026

Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but their use on volumetric data requires bridging the mismatch between a 2D encoder and a 3D volume input. Existing methods typically encode slices independently and integrate their features afterwards. We introduce Multiscale Volumetric Reduction (MVR), a reduce-then-encode approach that compresses each anatomical view from (D) slices into (M << D) complementary 2D components before foundation-model encoding. MVR combines an uncentered-PCA base component derived from the original through-plane intensities with residual detail components constructed from multiscale spatial descriptors. The reduction is estimated from the training volumes without diagnostic labels or gradient-based optimization and remains fixed thereafter. The resulting components are independently processed by a shared frozen 2D foundation model and concatenated for linear probing. Under this frozen-encoder setting, MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to the evaluated 2D-to-3D adaptation methods and simple input-reduction baselines, while also generalizing strongly from ADNI to AIBL.

Dexuan Ding, Yuankai Qi, Bogong Wang +2

Macquarie University · Australian National University · University of Sydney

Medical Imaging Foundation ModelsDimensionality ReductionBrain MRI
AskPDF
Aug 7, 2026

DINO-3DRA: Leveraging 2D Foundation Model Semantics for 3D Cerebral Aneurysm Segmentation

Accurate aneurysm segmentation in 3D rotational angiography (3DRA) is hindered by extreme class imbalance, morphological similarity to vessels, and absent large-scale 3D pretraining. 2D vision foundation models encode dense structural priors from 1.7 billion images, yet naïve slice-wise transfer fragments anatomical continuity and destabilises optimisation. We propose DINO-3DRA, a dual-path framework achieving effective cross-dimensional semantic transfer by injecting frozen DINOv3 features into a 3D U-Net backbone via Room-Lite spatial mixing and calibrated residual fusion. On multi-centre 3DRA data, DINO-3DRA achieves state-of-the-art aneurysm segmentation (Dice: 0.758; HD95: 2.75 mm; +13% over nnU-Net) with only 5.72M trainable parameters. Ablation studies confirm that gains arise from structured cross-dimensional transfer rather than loss design alone, with bridged foundation features improving anatomical continuity between aneurysms and parent vessels. Without fine-tuning on CADA and SHINY-ICARUS, DINO-3DRA eliminates all catastrophic failure cases observed in baseline architectures, demonstrating robust generalisation across heterogeneous imaging protocols.

Jiayang Lu, Fengming Lin, Alejandro F. Frangi +1

Centre for Computational Imaging and Modelling in Medicine (CIMIM), University of Manchester, Manchester, UK · Christabel Pankhurst Institute, University of Manchester, Manchester, UK · Department of Computer Science, University of Manchester, Manchester, UK +2

Medical Imaging Foundation ModelsTransfer LearningMedical Image Segmentation3D Medical Image Segmentation
AskPDF