cs.CVNov 18, 2026

SIRAZ: Sparse Inter-Layer Residual Aggregation for Zero-Shot Visual Neural Decoding

Authors: Neha, Mudasir Ganaie

Organizations: Department of Computer Science and Engineering, Indian Institute of Technology Ropar, Rupnagar, 140001, Punjab, India

Abstract

Brain-computer interfaces aim to establish direct communication between neural activity and external systems, making interpretation of brain signals an important problem. In this context, neural visual decoding seeks to infer visual information from neural activity by learning a mapping between brain responses and the corresponding visual stimuli. It investigates how visual information emerges from neural signals and relates to perceptual processing while enabling the reconstruction or retrieval of visual stimuli from neural signals. Our brain primarily follows a hierarchical pathway to process visual information captured by the eye. However, neural representations learned by existing methods differ fundamentally from those produced by the biological visual system. To bridge this gap and mimic the human vision hierarchy, we introduce Sparse Inter-layer Residual Aggregation for Zero-shot (SIRAZ). SIRAZ is a framework that aligns neural visual representations by jointly modeling early visual processes via fovea- and magnocellular-pathway-inspired Difference-of-Gaussians transformations. To align the semantic information with neural information, it extracts multiple intermediate representations of the image at multiple depths and passes them through the Residual Gated Adapter Mechanism. To study the temporal dynamics of neural signals, the Direction-aware Multi-Scale Temporal Attention Encoder is introduced to align the learned neural representation with the final visual representation. The proposed framework achieves 92.5% top-1 and 99.27% top-5 accuracy in the intra-subject setting on the THINGS-EEG dataset, and 50.25% and 75% top-1 and top-5 accuracy in the THINGS-MEG dataset. The results indicate that combining biologically motivated visual representations with the structured modalities of neural dynamics improves cross-modal alignment and zero-shot image retrieval.

Figures & tables

Explore similar work

CardsList
  1. Brain-Inspired Capture: Evidence-Driven Neuromimetic Perceptual Simulation for Visual Decoding

    Apr 20, 2026Feixue Shao, Guangze Shi, Xueyu Liu +6Representation LearningBrain-Computer Interfaces

  2. What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment

    Jun 24, 2026Salini Yadav, Taveena Lotey, Pravendra Singh +1Representation LearningEEG-Based Visual Decoding

  3. Aligning What EEG Can See: Structural Representations for Brain-Vision Matching

    Mar 7, 2026Jingyi Tang, Shuai Jiang, Fei Su +1EEG-Based Visual DecodingCross-Modal Alignment