Decoding Functional Networks for Visual Categories via GNNs

Shira Karmi; Galia Avidan; Tammy Riklin Raviv

cs.CVMar 30, 2026

Decoding Functional Networks for Visual Categories via GNNs

Authors: Shira Karmi, Galia Avidan, Tammy Riklin Raviv

Organizations: The School of Electrical and Computer Engineering, Ben-Gurion University of the Negev · The School of Brain Sciences and Cognition, Ben-Gurion University of the Negev · Psychology Department, Ben-Gurion University of the Negev

Abstract

Understanding how large-scale brain networks represent visual categories is fundamental to linking perception and cortical organization. Using high-resolution 7T fMRI from the Natural Scenes Dataset, we construct parcel-level functional graphs and train a signed Graph Neural Network that models both positive and negative interactions, with a sparse edge mask and class-specific saliency. The model accurately decodes category-specific functional connectivity states (sports, food, vehicles) and reveals reproducible, biologically meaningful subnetworks along the ventral and dorsal visual pathways. This framework bridges machine learning and neuroscience by extending voxel-level category selectivity to a connectivity-based representation of visual processing.

Figures & tables

Panel of Figure 1 — Figure 1: Example NSD stimulus with COCO instance masks and caption-derived category evidence for sports , food and vehicle

Panel of Figure 2 — Figure 2: t-SNE progression of graph embeddings across training. Clusters emerge for sports (blue), food (red), and vehicle (green), showing the signed GNN gradually disentangles category-selective connectivity patterns.

Subj.	Acc.	Sports	Food	Veh.	Macro-AP
1	0.790	0.916	0.939	0.829	0.895
2	0.769	0.860	0.930	0.837	0.876
3	0.640	0.739	0.843	0.643	0.742
4	0.870	0.923	0.942	0.880	0.915
5	0.810	0.929	0.949	0.918	0.932
6	0.818	0.950	0.959	0.910	0.940

Table 1: Per-subject test accuracy and Average Precision (AP). Macro-AP is the mean AP across the three classes. Final rows show mean

\pm

standard deviation.

Figure 3: Top-weighted parcels for sports, food, and vehicle. Warmer colors denote higher relevance from the learned mask and saliency.

Method	Acc.	Sports	Food	Veh.	Macro-AP
CL	0.637	0.728	0.752	0.613	0.698
Our	0.780	0.880	0.920	0.840	0.880

Table 2: Mean performance of contrastive learning vs. signed GNN.

Explore similar work

FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding
May 19, 2026Yudan Ren, Pengcheng Shi, Zihan Ma +2Neural Decoding Mixture-of-Experts Inference
School of Electronic Information (School of Artificial Intelligence), Northwest University, Xi’an 710127, China
Spiking Neural Networks for fMRI-Based Visual Semantic Decoding
Jul 21, 2026Jiahong Zhang, Jinning Zhao, Sijun Shen +3Visual Representation Learning Neural Decoding
Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China
BrainLinear: A Linear Model for Brain Network Analysis in Sparse Tangent Subspaces
Aug 15, 2026Sijing Wu, Dongyuan Li, Miaoting Huang +4Brain Network Analysis Functional Connectivity
Guangdong University of Foreign Studies · The University of Tokyo · RIKEN AIP +1

May 19, 2026J/K move · Enter open · S save

FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding

Yudan Ren, Pengcheng Shi, Zihan Ma, Xiaowei He+1

School of Electronic Information (School of Artificial Intelligence), Northwest University, Xi’an 710127, China

Neural Decoding Mixture-of-Experts Inference Neural Network Interpretability Neuroimaging

Open paper AskPDF arXiv

Visual image reconstruction from functional Magnetic Resonance Imaging (fMRI) is a fundamental task in brain decoding, providing a crucial pathway for understanding human perceptual mechanisms and developing advanced brain-computer interfaces (BCIs). However, most current methods simply flatten fMRI signals from localized visual cortices into one-dimensional (1D) vectors, mapping them directly into latent spaces such as that of Contrastive Language-Image Pre-training (CLIP). This paradigm not only disrupts the inherent network topology of the brain-leading to limited neuroscientific interpretability-but also overlooks the synergistic contributions of other distributed functional networks in processing high-level visual semantics. To address these limitations, we propose FPED, a Functional-Network Prior-Guided Mixture of Experts (MoE) framework for interpretable brain decoding. FPED explicitly models different functional brain networks as specialized experts and employs adaptive routing to capture their complementary contributions to visual semantic understanding. Unlike conventional homogeneous decoding paradigms, our framework incorporates neurobiologically grounded priors to enable structured and interpretable network-level representation learning. Experimental results demonstrate that FPED achieves highly competitive semantic reconstruction performance with only 0.68B parameters. The learned routing dynamics reveal biologically meaningful correspondence between functional brain networks and modality-specific semantic processing, providing transparent neuroscientific interpretability. This suggests that brain network-aware expert modeling is a promising direction for bridging neural decoding and biologically inspired artificial intelligence.

Abstract

Explore similar work

FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding

Spiking Neural Networks for fMRI-Based Visual Semantic Decoding

BrainLinear: A Linear Model for Brain Network Analysis in Sparse Tangent Subspaces