Joint Imaging-ROI Representation Learning via Cross-View Contrastive Alignment for Brain Disorder Classification

Wei Liang; Lifang He

cs.CVMar 10, 2026

Joint Imaging-ROI Representation Learning via Cross-View Contrastive Alignment for Brain Disorder Classification

Authors: Wei Liang, Lifang He

Organizations: Lehigh University, Bethlehem, PA, USA

Abstract

Brain imaging classification is commonly approached from two perspectives: modeling the full image volume to capture global anatomical context, or constructing ROI-based graphs to encode localized and topological interactions. Although both representations have demonstrated independent efficacy, their relative contributions and potential complementarity remain insufficiently understood. Existing fusion approaches are typically task-specific and do not enable controlled evaluation of each representation under consistent training settings. To address this gap, we propose a unified cross-view contrastive framework for joint imaging-ROI representation learning. Our method learns subject-level global (imaging) and local (ROI-graph) embeddings and aligns them in a shared latent space using a bidirectional contrastive objective, encouraging representations from the same subject to converge while separating those from different subjects. This alignment produces comparable embeddings suitable for downstream fusion and enables systematic evaluation of imaging-only, ROI-only, and joint configurations within a unified training protocol. Extensive experiments on the ADHD-200 and ABIDE datasets demonstrate that joint learning consistently improves classification performance over either branch alone across multiple backbone choices. Moreover, interpretability analyses reveal that imaging-based and ROI-based branches emphasize distinct yet complementary discriminative patterns, explaining the observed performance gains. These findings provide principled evidence that explicitly integrating global volumetric and ROI-level representations is a promising direction for neuroimaging-based brain disorder classification. The source code is available at https://anonymous.4open.science/r/imaging-roi-contrastive-152C/.

Figures & tables

Figure 1: Overview of the proposed joint imaging–ROI representation learning framework. (A) Joint representation learning via cross-view contrastive alignment. Subject-level global imaging and local ROI-graph embeddings are extracted by modular encoders and aligned in a shared latent space to enable downstream fusion and classification. (B) Post-hoc contribution analysis. Branch-specific attribution maps quantify complementary discriminative patterns captured by the imaging and ROI-graph representations.

Model	Acc	AUC	F1
	ADHD
ROI-only
DNN	$64.77\pm\pm 4.88$	$67.73\pm\pm 5.55$	$63.40\pm\pm 5.60$
NeuroGraph	$63.48\pm\pm 2.74$	$65.51\pm\pm 3.23$	$50.25\pm\pm 6.62$
Imaging-only
ViT3D*	$66.84\pm\pm 2.63$	$66.57\pm\pm 6.19$	$64.62\pm\pm 4.21$

Table 1: Single-branch vs. joint learning performance on ADHD-200 and ABIDE.

Imaging Encoder	ROI Encoder	Fusion	Acc	AUC	F1
			ADHD
3DSC-TF	DNN	Concat	$59.87\pm\pm 4.07$	$66.73\pm\pm 3.56$	$58.62\pm\pm 3.94$
		Cross-attn	$60.65\pm\pm 7.63$	$66.49\pm\pm 4.62$	$59.11\pm\pm 8.77$
		Contra	$62.84\pm\pm 4.20$	$68.69\pm\pm 4.62$	$61.06\pm\pm 5.79$
	NeuroGraph	Concat	$66.32\pm\pm 4.53$	$65.94\pm\pm 7.88$	$64.17\pm\pm 6.23$
		Cross-attn	$68.00\pm\pm 4.23$	$69.53\pm\pm 5.94$	$67.71\pm\pm 4.32$

Table 2: Encoder and joint learning ablation on ADHD-200 and ABIDE.

Figure 2: Missing-view robustness on ADHD-200. Shaded regions denote $\pm 1$ std over five-fold cross-validation.

Figure 3: Contribution maps for ADHD and TDC. The AAL parcellation (first column) provides anatomical reference; remaining columns show normalized contribution scores (0–1) for Joint Learning, 3DSC-TF Only, and NeuroGraph Only.

Explore similar work

Jul 12, 2026

Contrastive Joint-Embedding Prediction for Representation Learning in Structural MRI

Self-supervised learning offers a compelling approach for medical imaging, where labeled data are scarce and acquisition costs are high. We present COJEPA, a self-supervised framework for volumetric brain MRI that combines a joint-embedding predictive architecture (JEPA) with a contrastive loss (CO), targeting two complementary properties: local predictivity and global discriminability. The model is trained without labels on T1-weighted structural MRI from two cohorts (HCP-YA and AABC,

N{=}2286

, ages 22 to 90), extending I-JEPA to 3D with foreground-aware block masking, a hierarchical convolutional patch embedding, and world-space sinusoidal positional encodings. We evaluate all three objectives across zero-shot twin retrieval, brain tumor segmentation (BraTS 2024), and age regression (OpenBHB). COJEPA achieves the best monozygotic twin recall at rank@1 (0.84), the best finetuning age MAE (2.55 years on OpenBHB 3.0T), and matches CO on BraTS whole-tumor Dice, demonstrating that the combined objective yields representations that are simultaneously discriminative and locally structured.

Fabian Mager, Lars Kai Hansen

Department of Applied Mathematics and Computer Science Technical University of Denmark

Contrastive Learning Brain Age Prediction Joint-Embedding Predictive Architecture Medical Image Representation Learning

Ask PDF

Sep 17, 2026

HyperAMS-Net: Adaptive Multi-Scale Spatial Hypergraph Network for Brain Disorder Classification

Accurate classification of brain disorders from neuroimaging data remains challenging because of substantial inter-subject heterogeneity and the complex multi-scale patterns present in functional connectivity and morphological representations. To address these challenges, we propose HyperAMS-Net, a deep learning framework for brain disorder classification using neuroimaging representations derived from resting-state functional MRI or structural MRI. HyperAMS-Net integrates adaptive multi-scale convolution, hypergraph attention, spatial-channel attention, and adaptive feature fusion. Specifically, adaptive multi-scale convolution learns data-driven weights over multiple receptive fields to capture complementary patterns at different scales. Hypergraph attention models higher-order dependencies among learned feature representations through node--hyperedge--node message passing, while spatial-channel attention enhances discriminative feature learning. Adaptive feature fusion further aggregates complementary information across parallel network branches. HyperAMS-Net is evaluated on three benchmark datasets spanning distinct brain disorders: ABIDE for autism spectrum disorder, REST-meta-MDD for major depressive disorder, and ADNI for Alzheimer's disease, using 5-fold stratified cross-validation. HyperAMS-Net achieves state-of-the-art performance across all evaluated datasets, attaining the highest accuracy and AUC among the compared methods. Ablation studies further demonstrate the contribution of each proposed component, with the largest performance degradation observed when hypergraph attention is removed.

Proloy Kumar Mondal, Md Kamran Hussin Chowdhury, Hoi Leong Lee

Department of Computer Science and Engineering, University of North Texas, Denton, Texas, USA · Inje University, Gimhae-si, Republic of Korea · Faculty of Electronic Engineering and Technology, Universiti Malaysia Perlis, Arau, Malaysia

Medical Diagnosis Multi-Scale Feature Fusion Hypergraph Neural Networks Alzheimer's Disease Classification Neuroimaging

Ask PDF

Sep 14, 2026

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. We study both issues with a deliberately lightweight slice-based encoder (ResNet18 with a one-layer Transformer over slices) on 1,075 baseline T1-weighted scans from ADNI-1. First, we use FastSurfer segmentations as an anatomical reference: YOLOv8 models trained on segmentation-derived labels localize Alzheimer-relevant structures with mAP_50 above 0.96, and a Grad-CAM comparison shows that the image-only classifier frequently attends to the skull, orbits and background. Second, we adapt a CLIP-style image - tabular contrastive framework and organize ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores yields 87.3% three-way accuracy, which we treat as a leakage-driven upper bound rather than an imaging result; fusion with regional volumes yields 73.0%. We observe that the choice of contrastive target changes what the image encoder learns: on MCI vs. CN, the image-only head reaches 52.4% when the encoder is aligned to cognitive scores and 73.8% when aligned to volumes, although no tabular input is used at inference. Third, restricting the input to a per-subject crop of the medial temporal lobe raises image-only three-way accuracy from 58.7% to 65.1%. All results come from single runs on a small balanced test set, and we report confidence intervals and the protocol differences that prevent direct comparison with published numbers.

Paul-Gabriel Nicolae, Irina Georgiana Mocanu

Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology POLITEHNICA Bucharest, Romania

Data Leakage Alzheimer's Disease Classification Medical Image Grounding Explainable Medical Image Analysis Multimodal Contrastive Learning

Ask PDF