cs.CVNov 9, 2026

Complementarity-Driven Dual-Teacher Hierarchical Alignment for Zero-Shot EEG-to-Image Retrieval

Authors: Lukun Wu, Jie Li, Ziqi Ren, Ruiyang Xia, Yan Gao, Xinbo Gao

Organizations: Visual Information Processing Laboratory, School of Electronic Engineering, Xidian University, Xi’an 710071, China · Center for Biomedical-photonics and Molecular Imaging (BMI), School of Life Science and Technology, Xidian University, Xi’an 710126, China · School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China · School of Electronic Information (School of Artificial Intelligence), Northwest University, Xi’an 710127, China

Abstract

Decoding visual information from EEG is central to brain--computer interfaces and brain-inspired intelligence, and contrastive alignment with pretrained vision foundation models (VFMs) has become the dominant paradigm. However, existing methods align EEG solely with CLIP-family VFMs, leaving hierarchical visual information in neural responses under-exploited. To address this gap, we formulate a mechanism-paradigm correspondence view linking hierarchical visual processing (from appearance to semantics) to the complementary biases of cross-paradigm VFMs, with image-only models (e.g., DINO) and image--text models (e.g., CLIP) capturing distinct facets of this hierarchy. Guided by this view, we propose the Dual-Teacher Hierarchical Alignment (DTHA) framework. Concretely, a shared EEG encoder is supervised by frozen multi-blur CLIP and shallow DINOv2 teachers via dual-branch contrastive alignment, so each branch fully supervises the visual cues it emphasizes. A fused-retrieval objective learns adaptive weights to combine their complementary visual representations for zero-shot EEG-to-image retrieval. For the shared EEG encoder, we introduce the Attentive Modulated Scalp-Pattern Bank Encoder (AMSPBE), combining temporo-spatial attention with a learnable scalp-pattern bank for accurate, interpretable visual feature extraction, while modeling regional temporal dynamics and filter-bank topographies. On THINGS-EEG zero-shot retrieval, DTHA achieves 92.4% Top-1 accuracy, exceeding prior state of the art by 12.4 percentage points. Spatial, spectral, and representational-similarity analyses, together with THINGS-MEG evaluation, validate DTHA's effectiveness, biological plausibility, and cross-modality generalization. This complementarity-driven perspective shows that supervision from cross-paradigm VFMs enables more complete extraction of hierarchical visual information from brain activity.

Figures & tables

Explore similar work

CardsList
  1. Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval

    Apr 20, 2026Lin Jiang, Qingshan She, Jiale Xu +3Zero-Shot LearningEEG-Based Visual Decoding

  2. Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval

    Sep 21, 2026Ye Wang, Haokun Ren, Wei Wu +4ElectroencephalographyEEG-Based Visual Decoding

  3. What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment

    Jun 24, 2026Salini Yadav, Taveena Lotey, Pravendra Singh +1Representation LearningEEG-Based Visual Decoding