cs.CVSep 7, 2026

CNsEMD: An Expert-Annotated Multi-Field-Strength MRI Dataset and a Hyperspherical Manifold Network for Multimodal Cranial Nerve Parcellation

Authors: Lei Xie, Junxiong Huang, Guoqiang Xie, Jiawei Zhang, Jiahao Huang, Tianling Lyu, Ye Wu, Mingchu Li, 

Organizations: Advanced Interdisciplinary Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China · College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023, China · Department of Neurosurgery, Nuclear Industry 215 Hospital of Shaanxi Province, Xianyang 712000, China · School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing 210094, China · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Cranial nerves (CNs) play essential roles in sensory, motor, and autonomic functions. Accurate CN parcellation from multimodal magnetic resonance imaging (MRI) is crucial for neuroanatomical analysis and neurosurgical planning. However, accurate CN parcellation remains extremely challenging because CNs are very small, exhibit low image contrast, and have slender tubular morphologies and complex anatomical trajectories. Moreover, the lack of publicly available, expert-annotated datasets has impeded the development and fair benchmarking of learning-based CN analysis methods. In this work, we introduce CNsEMD, an expert-annotated multimodal dataset for CN parcellation. It comprises data from 202 subjects acquired on 3T, 5T, and 7T MRI scanners. We further propose the projective hyperspherical manifold network (PHM-Net), which learns cross-modal representations by capturing angular relationships in a shared hyperspherical embedding space. Rather than performing multimodal fusion in Euclidean space, the proposed Hyperspherical cross-modal interaction (HCI) module enables bidirectional feature exchange between T1-weighted (T1w) and direction-encoded color (DEC) representations on a unit hypersphere. The Magnitude-preserving projective hyperspherical orientation representation (PHOR) captures the axial nature of DEC orientations while preserving diffusion magnitude. The hyperspherical prototype segmentation head (HPSH) further extends angular similarity to voxel-wise classification using normalized voxel embeddings and learnable class prototypes. Extensive experimental results on the CNsEMD dataset demonstrate the effectiveness of our PHM-Net against state-of-the-art methods. CNsEMD establishes a reproducible benchmark for multimodal CN imaging, while PHM-Net provides a geometry-consistent solution for CN parcellation across diverse MRI acquisitions.

Figures & tables

Fig. 1: Conceptual motivation for hyperspherical cross-modal interaction. In Euclidean space, feature similarity depends on both direction and magnitude. HCI maps multimodal representations onto a unit hypersphere through ℓ2\ell_{2} normalization and models their cross-modal relationships using angular similarity.
DatasetField StrengthMatrix SizeVoxel Size#Cases
HCP3T3T145×174×145145\times 174\times 1451.25×1.25×1.251.25\times 1.25\times 1.25102
Diff5T5T151×181×151151\times 181\times 1511.20×1.20×1.201.20\times 1.20\times 1.2050
HCP7T7T173×207×173173\times 207\times 1731.05×1.05×1.051.05\times 1.05\times 1.0550
CNsEMD3T/5T/7T–1.05–1.25202
TABLE I: Summary of CNsEMD after preprocessing. Each subject contains one-channel T1w images and three-channel DEC images.
Fig. 2: Representative CNsEMD cases grouped by the field strength of the diffusion acquisition (3T, 5T, or 7T). From left to right, each row shows the voxel-level annotations rendered as surface meshes, the T1w image, an enlarged view of the annotated CNs, and the corresponding DEC image.
Fig. 3: Overview of the proposed PHM-Net. PHOR first converts DEC vectors into sign-invariant, magnitude-preserving orientation representations. T1w and DEC features are then extracted by modality-specific encoders and progressively interacted through HCI in a shared hyperspherical space. The fused multiscale features are decoded, and HPSH produces the final voxel-wise parcellation based on angular similarities to learnable class prototypes.
 MethodsMixed ( n=40n=40 )3T subset ( n=20n=20 )
 Dice (%) ↑\uparrowJac (%) ↑\uparrowASD (mm) ↓\downarrowAHD (mm) ↓\downarrowDice (%) ↑\uparrowJac (%) ↑\uparrowASD (mm) ↓\downarrowAHD (mm) ↓\downarrow
Segmentation MethodsCSNet75.16 ±6.08\scriptstyle\pm\,\scriptstyle{6.08}61.13 ±7.20\scriptstyle\pm\,\scriptstyle{7.20}0.488 ±0.197\scriptstyle\pm\,\scriptstyle{0.197}0.383 ±0.164\scriptstyle\pm\,\scriptstyle{0.164}76.76 ±2.97\scriptstyle\pm\,\scriptstyle{2.97}63.05 ±3.72\scriptstyle\pm\,\scriptstyle{3.72}0.445 ±0.092\scriptstyle\pm\,\scriptstyle{0.092}0.355 ±0.083\scriptstyle\pm\,\scriptstyle{0.083}
MSUNet75.64 ±5.90\scriptstyle\pm\,\scriptstyle{5.90}61.77 ±7.08\scriptstyle\pm\,\scriptstyle{7.08}0.475 ±0.190\scriptstyle\pm\,\scriptstyle{0.190}0.376 ±0.162\scriptstyle\pm\,\scriptstyle{0.162}77.42 ±3.06\scriptstyle\pm\,\scriptstyle{3.06}63.95 ±3.83\scriptstyle\pm\,\scriptstyle{3.83}0.436 ±0.096\scriptstyle\pm\,\scriptstyle{0.096}0.352 ±0.083\scriptstyle\pm\,\scriptstyle{0.083}
SwinUNet67.82 ±6.90\scriptstyle\pm\,\scriptstyle{6.90}52.44 ±7.26\scriptstyle\pm\,\scriptstyle{7.26}0.922 ±0.587\scriptstyle\pm\,\scriptstyle{0.587}0.742 ±0.432\scriptstyle\pm\,\scriptstyle{0.432}70.73 ±4.15\scriptstyle\pm\,\scriptstyle{4.15}55.74 ±4.68\scriptstyle\pm\,\scriptstyle{4.68}0.693 ±0.201\scriptstyle\pm\,\scriptstyle{0.201}0.572 ±0.173\scriptstyle\pm\,\scriptstyle{0.173}
TransUNet74.64 ±5.65\scriptstyle\pm\,\scriptstyle{5.65}60.44 ±6.80\scriptstyle\pm\,\scriptstyle{6.80}0.486 ±0.150\scriptstyle\pm\,\scriptstyle{0.150}0.386 ±0.131\scriptstyle\pm\,\scriptstyle{0.131}75.94 ±3.14\scriptstyle\pm\,\scriptstyle{3.14}62.05 ±3.85\scriptstyle\pm\,\scriptstyle{3.85}0.461 ±0.093\scriptstyle\pm\,\scriptstyle{0.093}0.373 ±0.080\scriptstyle\pm\,\scriptstyle{0.080}
Multimodal MethodsFECCNet75.45 ±5.49\scriptstyle\pm\,\scriptstyle{5.49}61.50 ±6.63\scriptstyle\pm\,\scriptstyle{6.63}0.507 ±0.184\scriptstyle\pm\,\scriptstyle{0.184}0.407 ±0.171\scriptstyle\pm\,\scriptstyle{0.171}77.06 ±2.97\scriptstyle\pm\,\scriptstyle{2.97}63.46 ±3.73\scriptstyle\pm\,\scriptstyle{3.73}0.481 ±0.118\scriptstyle\pm\,\scriptstyle{0.118}0.388 ±0.103\scriptstyle\pm\,\scriptstyle{0.103}
TABLE II: Comparative results on the mixed-field-strength CNsEMD subject-independent evaluation set and the 3T, 5T, and 7T subsets. Dice and Jac are reported in percentage (%), while ASD and AHD are reported in millimeters (mm). Results are presented as mean ±\pm standard deviation. The best and second-best results are highlighted in red and blue, respectively.
Fig. 4: Qualitative comparison on representative 3T, 5T, and 7T CNsEMD cases. For each case, the rows show three-dimensional reconstructions, slice-wise results, and enlarged views of challenging regions. In the slice-wise views, colored voxel overlays indicate the expert reference, while red contours delineate each method’s predictions.
BasePHORHCIHPSHDice (%) ↑\uparrowJac (%) ↑\uparrowASD (mm) ↓\downarrowAHD (mm) ↓\downarrow
✓–––75.63 ±5.47\scriptstyle\pm\,\scriptstyle{5.47}61.74 ±6.68\scriptstyle\pm\,\scriptstyle{6.68}0.733 ±0.374\scriptstyle\pm\,\scriptstyle{0.374}0.608 ±0.353\scriptstyle\pm\,\scriptstyle{0.353}
✓–✓–75.87 ±5.50\scriptstyle\pm\,\scriptstyle{5.50}62.05 ±6.67\scriptstyle\pm\,\scriptstyle{6.67}0.531 ±0.226\scriptstyle\pm\,\scriptstyle{0.226}0.430 ±0.207\scriptstyle\pm\,\scriptstyle{0.207}
✓✓––76.15 ±5.42\scriptstyle\pm\,\scriptstyle{5.42}62.36 ±6.63\scriptstyle\pm\,\scriptstyle{6.63}0.488 ±0.161\scriptstyle\pm\,\scriptstyle{0.161}0.391 ±0.149\scriptstyle\pm\,\scriptstyle{0.149}
✓✓✓–76.08 ±5.50\scriptstyle\pm\,\scriptstyle{5.50}62.26 ±6.77\scriptstyle\pm\,\scriptstyle{6.77}0.595 ±0.312\scriptstyle\pm\,\scriptstyle{0.312}0.489 ±0.298\scriptstyle\pm\,\scriptstyle{0.298}
✓✓✓✓76.58 ±5.33\scriptstyle\pm\,\scriptstyle{5.33}62.92 ±6.55\scriptstyle\pm\,\scriptstyle{6.55}0.453 ±0.134\scriptstyle\pm\,\scriptstyle{0.134}0.358 ±0.121\scriptstyle\pm\,\scriptstyle{0.121}
TABLE III: Ablation study of the proposed components on CN parcellation performance.
Fig. 5: Modality-specific feature geometry before and after HCI at the third interaction stage. Each panel corresponds to one CN category, and its annotation reports the mean cosine similarity between spatially paired T1w and DEC embeddings.
Fig. 6: Parameter sensitivity analysis using five-fold out-of-fold validation results. (a) Effects of the HCI projection dimension Dh∈{32,64,128,256,512,1024}D_{h}\in\{32,64,128,256,512,1024\} on Dice, Jac, ASD, and AHD. (b) Effects of the number of hyperspherical prototypes K∈{1,2,3,4}K\in\{1,2,3,4\} per class in HPSH. Red circles indicate the configurations adopted in the final model.
Model variantParams (M)FLOPs/slice (G)
Baseline81.3455.43
Baseline + HCI85.0357.80
Baseline + HCI + PHOR85.0457.89
Baseline + HCI + PHOR + HPSH85.0658.68
TABLE IV: Computational complexity of the model variants. FLOPs are defined as twice the MACs reported by THOP for one 128×160128\times 160 axial T1w–DEC input pair.
T1wT2wFADECPeaksDice (%) ↑\uparrowJac (%) ↑\uparrowASD (mm) ↓\downarrowAHD (mm) ↓\downarrow
✓    71.83 ±5.56\scriptstyle\pm\,\scriptstyle{5.56}57.36 ±6.22\scriptstyle\pm\,\scriptstyle{6.22}0.587 ±0.203\scriptstyle\pm\,\scriptstyle{0.203}0.477 ±0.183\scriptstyle\pm\,\scriptstyle{0.183}
 ✓   64.62 ±6.62\scriptstyle\pm\,\scriptstyle{6.62}48.89 ±6.26\scriptstyle\pm\,\scriptstyle{6.26}0.746 ±0.313\scriptstyle\pm\,\scriptstyle{0.313}0.610 ±0.301\scriptstyle\pm\,\scriptstyle{0.301}
  ✓  58.82 ±7.70\scriptstyle\pm\,\scriptstyle{7.70}42.78 ±7.25\scriptstyle\pm\,\scriptstyle{7.25}1.030 ±0.731\scriptstyle\pm\,\scriptstyle{0.731}0.836 ±0.490\scriptstyle\pm\,\scriptstyle{0.490}
   ✓ 60.52 ±6.76\scriptstyle\pm\,\scriptstyle{6.76}44.41 ±6.53\scriptstyle\pm\,\scriptstyle{6.53}0.995 ±0.561\scriptstyle\pm\,\scriptstyle{0.561}0.811 ±0.396\scriptstyle\pm\,\scriptstyle{0.396}
    ✓55.95 ±11.58\scriptstyle\pm\,\scriptstyle{11.58}40.38 ±10.31\scriptstyle\pm\,\scriptstyle{10.31}1.064 ±0.492\scriptstyle\pm\,\scriptstyle{0.492}0.890 ±0.417\scriptstyle\pm\,\scriptstyle{0.417}
✓✓   72.75 ±6.05\scriptstyle\pm\,\scriptstyle{6.05}58.40 ±6.57\scriptstyle\pm\,\scriptstyle{6.57}0.602 ±0.493\scriptstyle\pm\,\scriptstyle{0.493}0.485 ±0.402\scriptstyle\pm\,\scriptstyle{0.402}
TABLE V: Performance comparison of different modality combinations for CN parcellation.
More in this topic

Cross-Modality Medical Image Segmentation

Cross-modality medical image segmentation labels anatomy and disease across imaging types despite differences in appearance, resolution, and available annotations.

See all 72 papers

Explore similar work

CardsList
Sep 7, 2026

Spatial Feature-wise Linear Modulation (SpFiLM) for Contrast Agent-Aware Brain Parcellation

Most automated brain parcellation tools are developed and validated on T1-weighted (T1w) MRI. Yet, some clinical workflows for which parcellation is relevant only use contrast-enhanced T1w (T1ce) MRI, on which T1w-trained models are less accurate. We present a unified network that parcellates both pre- and post-contrast agent T1w MRI reliably, trained on a combination of the two with conditioning that spatially modulates its response differently for each. Feature-wise Linear Modulation (FiLM) is a known approach for input-based modulation in networks. It applies a per-channel scale and shift uniformly across the input. However, the appearance change between pre- and post-contrast varies locally across the brain, making FiLM suboptimal for our use case. In this work, we introduce Spatial FiLM (SpFiLM), a conditioning layer whose modulation varies spatially, assembling a voxel-wise scale and shift from image-derived spatial patterns. Using a cohort of 134 patients with paired T1w and T1ce MRI parcellated into 106 classes, the addition of SpFiLM layers in a UNet increased the mean Dice on the test set of 25 patients from 80.2% to 84.1%, a 4.9% relative improvement. Adding SpFiLM layers led to the best performance on both pre- and post-contrast MRI, even when controlling for network parameter counts.

Pushpendra Singh, Joshua R. Astley, Roman Rodionov +3

School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK · Department of Epilepsy, Queen Square Institute of Neurology, University College London, London, UK

Medical Image SegmentationNeuroimagingBrain MRI
AskPDF
May 7, 2026

Learning Cross-Atlas Consistent Brain Disorder Representations via Disentangled Multi-Atlas Functional Connectivity Learning

Functional connectivity (FC) derived from resting-state fMRI is widely used to characterize large-scale brain network alterations in neurological and psychiatric disorders. However, FC construction critically depends on the choice of brain atlas, and different parcellations may emphasize distinct organizational features, leading to heterogeneous and sometimes inconsistent representations. Existing multi-atlas approaches partially alleviate this issue but often fuse atlas-derived features or predictions at a relatively shallow level, while single-atlas disentanglement methods do not explicitly address cross-atlas heterogeneity. We propose Multi-Atlas Disentangled Connectivity LEarning (MADCLE), a multi-branch representation learning framework that jointly encodes FC matrices derived from different brain atlases. Rather than introducing a single explicitly shared latent variable across parcellations, MADCLE learns atlas-wise disease-related representations and encourages them to be cross-atlas consistent through distributional alignment. Meanwhile, covariate-related and atlas-dependent residual factors are modeled separately using covariate similarity supervision, atlas-specific reconstruction, and decorrelation constraints, thereby reducing the leakage of non-disease and parcellation-dependent information into the disease-related embeddings. Experiments on the ADNI and ADHD-200 datasets suggest that MADCLE achieves competitive or improved performance compared with single-atlas baselines, multi-atlas GNN/Transformer models, and recent multi-atlas consistency frameworks. These results support the potential value of structured disentanglement for FC-based disorder identification under heterogeneous parcellation schemes.

Minheng Chen, Chao Cao, Jing Zhang +2

Department of Computer Science and Engineering, University of Texas at Arlington, United States · School of Computing, University of Georgia, United States

Multi-View ConsistencyDisentangled Representation LearningFunctional ConnectivityMedical DiagnosisNeuroimagingBrain Disorder Classification
AskPDF
Aug 26, 2026

PANDA - Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimers MRI and TCGA Pathology

Multimodal medical prediction often faces incomplete pairing: auxiliary modalities with complementary signal are available for only a subset of subjects (or none) and cannot be assumed at deployment. We introduce PANDA (Prototype Anchored Data Alignment), a two-stage framework that transfers auxiliary information to a primary-modality model without auxiliary inputs at inference. Stage 1 learns a shared embedding from the paired subset and estimates class prototypes from auxiliary modalities; Stage 2 trains the primary encoder on all subjects using cross-entropy plus alignment to the frozen prototypes. Because supervision is defined at the class-prototype level, PANDA accommodates arbitrary pairing rates, including zero subject overlap. We evaluate PANDA on two applications. On a 1,021-subject multi-scanner ADNI cohort, we perform AD/CN classification with three auxiliary modalities at distinct pairing rates: tabular scores (44.8%), FDG-PET (18.7%), and external handwriting kinematics (0% overlap). Relative to the same-backbone MRI-only baseline, PANDA attains AUC 0.868 +-0.020 (+7.9pp) and reduces 1.5T CN false positives by 24.3pp; on a fully trainable Conv5-FC3 backbone it reaches AUC 0.893 (best overall). A pairing-rate ablation shows that the joint anchor remains within seed noise from 75% to 5% pairing. On TCGA-Lung survival prediction from whole-slide images with RNA-seq as auxiliary data, PANDA improves over WSI-only on 2-year OS (AUC +3.5pp) and Cox PH (C-index +9.0pts) and outperforms full-fusion training, which underperforms WSI-only, while requiring no RNA at inference; wide confidence intervals on this smaller cohort keep the gains below conventional significance. Overall, PANDA provides a deployment-oriented mechanism for leveraging incomplete auxiliary modalities to improve primary-modality prediction.

Sheethal Bhat, Mahfuzur Rahman Chowdhury, Paula Andrea Perez-Toro +4

Pattern Recognition Lab, Friedrich-Alexander-Universität, Erlangen-Nürnberg, Germany. · Department of Neurology, Klinikum Nürnberg, Paracelsus Medical University, Nürnberg, Germany. · Department of Radiology, LMU University Hospital, LMU Medizin, Ludwig-Maximilians-Universität München, Munich, Germany. +1

Multimodal LearningMissing-Modality LearningCross-Modal AlignmentAlzheimer's Disease Classification
AskPDF