SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval

Zhenyao Cui; Siyuan Kan; Siyang Li; Ziwei Wang; Dongrui Wu

cs.LGAug 19, 2026

SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval

Authors: Zhenyao Cui, Siyuan Kan, Siyang Li, Ziwei Wang, Dongrui Wu

Organizations: School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China

Abstract

Accurate visual decoding can reveal how the brain represents visual information and recover perceived content from neural signals such as electroencephalography (EEG), with potential for neural communication. However, current EEG-to-image retrieval methods perform far below their within-subject counterparts for new users without labeled calibration, limiting real-world deployment. To understand this gap, we analyze EEG features across subjects and find that different subjects preserve similar relationships among concepts but express them along different coordinate directions. We therefore propose Subject Coordinate Recovery (SCORE), a target label-free framework combining recovery-aware source training with coordinate alignment at deployment. During training, SCORE aligns source subject EEG with a common image space and simulates unseen-subject recovery through source-only episodes. At deployment, with both encoders frozen, SCORE selects reliable EEG-image landmarks through hubness-corrected matching and estimates an orthogonal transformation to recover target EEG coordinates without source data or target labels. In 200-way retrieval on two public benchmarks, SCORE outperforms the unadapted baseline for every target subject and achieves the best overall accuracy. It reaches 53.23%/83.55% and 12.01%/32.16% Top-1/Top-5 on THINGS-EEG2 and Alljoined-1.6M, respectively, surpassing the strongest baselines by 17.45/15.70 and 3.08/4.62 percentage points. Without target labels or encoder updates, SCORE brings brain-based visual decoding closer to robust, practical, low-latency deployment across users.

Figures & tables

Figure 1: Cross-subject EEG-to-image retrieval and the idea behind SCORE . (a) Source subjects provide labeled EEG–image pairs, whereas an unseen target subject provides unlabeled EEG and a candidate gallery. (b) Each subject expresses the same concept relationships along its own coordinate directions, which SCORE recovers in the image space.

Figure 2: Source and target EEG RSMs on encoder-unseen concepts (THINGS-EEG2), for one LOSO fold. (a)-(c) Two of the nine source subjects and the held-out target subject, under a shared fixed z-score scale. (d) All held-out concept pairs for the displayed source–target pair, with the target subject on the horizontal axis and source B on the vertical axis ( $r=0.713$ ).

Alignment	Top-1 $\uparrow$	Top-5 $\uparrow$
Direct	$16.89$	$41.95$
Ridge map	$20.01$	$47.15$
Orthogonal map	$28.22$	$59.16$

Table 1: Target-to-source EEG retrieval before and after coordinate mapping on THINGS-EEG2.

Figure 3: Orthogonal coordinate recovery for one target–source pair, projected onto a two-dimensional plane chosen from the fitting concepts.

Figure 4: Overview of SCORE . Left: recovery-aware source training aligns source subjects and simulates target recovery. Right: at deployment, coordinate recovery recovers the coordinates of unlabeled target EEG features before image retrieval.

Source training	Deployment method	THINGS-EEG2		Alljoined-1.6M
Source training	Deployment method	Top-1 $\uparrow$	Top-5 $\uparrow$	Top-1 $\uparrow$	Top-5 $\uparrow$
Original	None	$26.22\pm 1.08$	$57.98\pm 0.88$	$6.08\pm 0.33$	$20.55\pm 0.79$
	EA	$27.45\pm 0.82$	$59.37\pm 1.07$	$6.31\pm 0.19$	$20.47\pm 0.67$
	SATTC (SAW)	$30.98\pm 0.35$	$60.85\pm 1.00$	$8.93\pm 0.34$	$27.54\pm 1.19$
	SATTC	$26.12\pm 0.45$	$52.73\pm 1.82$	$6.78\pm 0.15$	$20.70\pm 0.98$
	CSLS	$35.78\pm 1.62$	$67.85\pm 1.21$	$8.04\pm 0.35$	$25.39\pm 0.54$

Table 2: Cross-subject retrieval on two EEG-to-image benchmarks, with the SAMGA encoder throughout. EA is Euclidean Alignment, which normalizes each subject before source training and is then deployed without further adaptation. SATTC (SAW) denotes SATTC’s subject-adaptive whitening operator, applied on its own. Values are means and standard deviations across all subjects and three seeds.

Figure 5: Retrieval accuracy for every held-out subject and the mean (THINGS-EEG2), in (a) Top-1 and (b) Top-5.

Figure 6: Retrieval examples from one held-out subject. Each method shows its top three images, the green border marks the viewed image, and the rightmost column its rank in the 200-image gallery before and after recovery.

EEG encoder	Standard		Recovery-Aware		Standard + Recovery		Complete SCORE
EEG encoder	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
NICE ( Song et al. 2024 )	$27.60$	$60.38$	$31.08$	$63.50$	$50.27$	$81.05$	$\mathbf{54.85}$	$\mathbf{83.75}$
ShallowFBCSPNet ( Schirrmeister et al. 2017 )	$10.63$	$32.67$	$11.37$	$33.83$	$28.97$	$61.45$	$\mathbf{30.73}$	$\mathbf{62.92}$
EEGNet ( Lawhern et al. 2018 )	$15.95$	$42.80$	$17.22$	$45.68$	$44.28$	$76.72$	$\mathbf{46.95}$	$\mathbf{78.65}$

Table 3: Generalization across EEG encoders on THINGS-EEG2. Values are mean Top-1 and Top-5 accuracy across three seeds.

Phase	Configuration	Top-1 $\uparrow$	Top-5 $\uparrow$
Train	SAMGA objective	$26.22\pm 1.08$	$57.98\pm 0.88$
	+ multi-positive	$28.63\pm 1.75$	$62.02\pm 1.73$
	+ simulated recovery	$29.33\pm 0.57$	$62.42\pm 1.60$
Test	CSLS ranking	$39.08\pm 0.81$	$71.23\pm 0.70$
	+ mean and scale	$43.80\pm 1.57$	$75.68\pm 0.58$
	+ recovery	$50.98\pm 1.53$	$80.38\pm 0.78$

Table 4: Cumulative ablation of recovery-aware training and coordinate recovery. Each row adds one component to the row above it.

Gallery size	Raw	SATTC (SAW)	CSLS	SCORE
200	$29.33$	$35.28$	$39.08$	$\mathbf{53.23}$
700	$15.72$	$17.98$	$21.75$	$\mathbf{29.47}$
1,200	$12.25$	$12.80$	$17.03$	$\mathbf{22.27}$
1,854	$9.47$	$9.92$	$13.33$	$\mathbf{17.03}$

Table 5: Top-1 retrieval as training-concept distractors are added to the 200 matching images. Gallery size counts the 200 matching images plus the added distractors. Every column uses the same recovery-aware representation.

Update	Ranking	Top-1 $\uparrow$	Top-5 $\uparrow$
None	Cosine	$39.53\pm 0.97$	$75.59\pm 0.81$
None	CSLS	$50.41\pm 0.60$	$83.16\pm 0.77$
Frozen map	Cosine	$56.35\pm 1.14$	$87.38\pm 0.78$
Frozen map	CSLS	$\mathbf{64.16\pm 1.59}$	$\mathbf{91.51\pm 0.91}$

Table 6: Transfer of a frozen map between disjoint 100-concept galleries.

Cross-Subject EEG Decoding

Cross-subject EEG decoding develops methods that classify or predict brain activity for people not seen during training.

See all 77 papers

Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval
Apr 20, 2026Lin Jiang, Qingshan She, Jiale Xu +3Zero-Shot Learning EEG-Based Visual Decoding
School of Automation, Hangzhou Dianzi University, and also with Zhejiang Provincial Key Laboratory of Brain Computer Collaborative Intelligence Technology and Applications, Hangzhou, Zhejiang 310018, China
Structured Visual Target Learning For Cross-Subject eeg-to-image retrieval
Sep 29, 2026Salini Yadav, Taveena Lotey, Mickaël Coustaty +2Cross-Subject EEG Decoding Visual Representation Learning
Indian Institute of Technology Roorkee, India · University of La Rochelle, France · Indian Institute of Technology (ISM) Dhanbad, India
SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval
Mar 21, 2026Qunjie Huang, Weina ZhuCross-Subject EEG Decoding EEG-Based Visual Decoding
Yunnan University, China

Apr 20, 2026J/K move · Enter open · S save

Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval

Lin Jiang, Qingshan She, Jiale Xu, Haiqi Xu+2

School of Automation, Hangzhou Dianzi University, and also with Zhejiang Provincial Key Laboratory of Brain Computer Collaborative Intelligence Technology and Applications, Hangzhou, Zhejiang 310018, China

Zero-Shot Learning EEG-Based Visual Decoding Cross-Modal Alignment

Open paper AskPDF arXiv

Zero-shot EEG-to-image retrieval aims to decode perceived visual content from electroencephalography (EEG) by aligning neural responses with pretrained visual representations, providing a promising route toward scalable visual neural decoding and practical brain-computer interfaces. However, robust EEG-to-image retrieval remains challenging, because prior methods usually rely on either a single fixed visual target or a subject-invariant target construction scheme. Such designs overlook two important properties of visually evoked EEG signals: they preserve information across multiple representational scales, and the visual granularity best matched to EEG may vary across subjects. To address these issues, subject-aware multi-granularity alignment (SAMGA) framework is proposed for zero-shot EEG-to-image retrieval. SAMGA first constructs a subject-aware visual supervision target by adaptively aggregating multiple intermediate representations from a pretrained vision encoder, allowing the model to absorb subject-dependent granularity deviations during training while preserving subject-agnostic inference. Building on this adaptive target construction, a coarse-to-fine cross-modal alignment strategy is further designed with a shared encoder wherein the coarse stage stabilizes the shared semantic geometry and reduces subject-induced distribution shift, and the fine stage further improves instance-level retrieval discrimination. Extensive experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves 91.3% Top-1 and 98.8% Top-5 accuracy in the intra-subject setting, and 34.4% Top-1 and 64.8% Top-5 accuracy in the inter-subject setting, outperforming recent state-of-the-art methods.

Abstract

Cross-Subject EEG Decoding

Explore similar work

Subject-Aware Multi-Granularity Alignment for Zero-Shot EEG-to-Image Retrieval

Structured Visual Target Learning For Cross-Subject eeg-to-image retrieval

SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval