Decoding visual information from brain signals probes neural representations and enables neuro-rehabilitation and dream decoding. Recent brain-to-image retrieval approaches have achieved promising performance, typically by averaging many (up to 80) neural trials per image, requiring repeated stimulus presentation that increases latency, cost, and user burden. When only one or a few repetitions are available, the retrieval accuracy drops sharply. This drop is commonly attributed to query noise because averaging suppresses noise and increases signal stability. However, we find a non-transitive alignment pattern: the low-repetition query signal and the image representation each align with the high-repetition center, but not directly with each other. This pattern shows that query noise is only part of the problem and that gallery placement also affects retrieval. We therefore propose a neural-anchor-based retrieval (NEAR) framework that treats the high-repetition center as an anchor and approaches it from both sides: a denoiser pulls the noisy query toward the true anchor, and a small network predicts each candidate's pseudo anchor from its image and pulls the image toward it. Across four datasets spanning EEG, MEG and fMRI, NEAR consistently improved retrieval in the few-repetition regime. On THINGS-EEG2, it improved 200-way Top-1 accuracy by 5.7 and 9.3 percentage points respectively, when averaging one and four repetitions. By anchoring neural and visual representations, NEAR reduces reliance on repeated acquisition and brings neural retrieval closer to real-world deployment.
Figures & tables
Figure 1: Non-transitive few-repetition retrieval. (a) Averaging repeated measurements produces a stable neural representation that retrieves the viewed image. (b) The few-repetition query and image each align with this representation, but not reliably with each other.
Figure 2: Accuracy and acquisition return across repetitions. (a) Accuracy rises quickly and then approaches a plateau. (b) Stimulus-time-normalized ITR peaks at k=2 because later gains no longer offset added stimulus time.
Figure 3: Target swap exposes non-transitive retrieval. With qc(k) and qˉc estimated from disjoint repetitions, qc(k) retrieves qˉc more accurately than its image vc (top). Direct retrieval can still fail when q(k)→qˉ and qˉ→v both succeed (bottom).
Figure 4: Training and inference in NEAR. Query Anchoring is learned from k -repetition averages and then frozen; Gallery Anchoring is learned from image–anchor pairs, using all nc training repetitions of concept c to form the anchor qˉc . At inference the two branches give a denoised query and a pseudo anchor per candidate, and ⊗ combines the two cosine similarities, not the two representations, as in ( 6 ). The right panels illustrate both effects.
Table 1: Top-1 accuracy (%) on THINGS-EEG2 (10 participants) and Alljoined (20 participants). Bold marks the best result at each repetition count.
Dataset
Query estimator
k=1
2
4
8
16
32
64
80
THINGS-EEG2
xDAWN
16.8/ 17.9
28.7/ 31.4
43.7/ 48.5
57.6/ 64.6
67.1/ 75.7
72.8/ 81.5
75.1/ 84.3
75.9/ 84.7
DSS
17.9/ 18.9
30.6/ 33.0
46.6/ 50.9
61.2/ 67.3
70.6/ 78.0
75.9/ 83.3
78.8/ 85.6
79.7/ 86.2
N2N
18.4/ 19.8
31.3/ 34.3
47.5/ 52.5
62.1/ 68.5
71.7/ 79.0
77.2/ 84.4
79.9/ 87.1
80.5/ 87.6
Wavelet
9.9/ 10.5
17.0/ 18.5
26.6/ 29.7
36.6/ 41.8
44.4/ 51.8
49.7/ 58.2
53.1/ 62.3
54.0/ 63.5
Alljoined
xDAWN
1.46/ 1.59
1.99/ 2.22
2.96/ 3.36
4.62/ 5.34
6.83/ 7.89
9.46/ 11.08
12.82/ 14.34
13.69/ 15.15
DSS
1.41/ 1.53
1.91/ 2.15
2.80/ 3.21
4.35/ 5.04
6.63/ 7.68
9.35/ 10.37
11.78/ 13.34
12.83/ 14.18
Table 2: Gallery Anchoring complements four query estimators. Each cell reports fixed-query / + G Top-1 accuracy (%), averaged over three seeds.
THINGS-MEG (200-way)
Split
Method
k=1
k=4
k=12
Within-subject
Base
7.5 ± 0.40
19.8 ± 0.10
30.9 ± 0.30
+ G
8.4 ± 0.10
21.9 ± 0.40
32.7 ± 1.10
NSD 7T fMRI (982-way)
Visual target
Method
k=1
k=2
k=3
CLIP ViT-L/14
Base
27.5 ± 0.20
37.8 ± 0.10
43.9 ± 0.40
Table 3: Gallery Anchoring improves MEG and fMRI Top-1 accuracy (%) with each query pathway fixed.
Figure 6: Gallery Anchoring improves NSD gallery placement. Each point is one held-out image for one participant and seed; the upper-left region marks corrected errors.
Candidate reference
k=1
4
16
80
Base (visual)
14.05
43.23
72.46
83.35
Single-repetition target
15.78
45.57
72.41
81.98
Stable target, InfoNCE
14.51
45.43
76.25
86.90
Stable target, ridge
14.85
46.73
77.59
87.67
Stable target, MLP ( +\textscG )
14.91
46.99
77.93
87.97
Table 4: Gallery controls on THINGS-EEG2. Three-seed mean Top-1 accuracy (%) with all components except the candidate reference fixed.
Brain-to-image retrieval seeks to identify the visual stimulus that elicited a non-invasive neural response. Candidate images are typically represented by pretrained vision models, whose internal representations vary in abstraction across depth. Existing methods usually train the neural encoder to recover a fixed final-layer visual target. Under this formulation, the visual hierarchy is reduced to a single prescribed endpoint, preventing representations at other depths from directly shaping the visual target. This limitation motivates learning how information across visual depths should contribute to the retrieval target. To this end, we introduce NeuroGlyph, which learns a trial-independent visual target from multiple depths of a frozen visual backbone. NeuroGlyph decomposes the target into factor-specific subspaces. Each subspace learns an image-conditioned allocation over visual depth. The resulting subspaces are fused into a single embedding for retrieval. Across THINGS-EEG and THINGS-MEG, NeuroGlyph outperforms final-layer supervision in all controlled comparisons. It also surpasses the post hoc best fixed-layer oracle in three of four comparisons. Parameter-matched ablations support both factorized target construction and image-conditioned depth allocation. Under comparable 200-way retrieval protocols, NeuroGlyph achieves the strongest system-level performance in six of eight reported metrics. These results support learning retrieval targets across the visual hierarchy rather than prescribing one visual depth.
Ye Wang, Haokun Ren, Hong Yu +4
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing, China · College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, China · Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong, China +1
Decoding visual experience from non-invasive brain activity is central to neuroscience and brain-computer interfaces. Functional magnetic resonance imaging (fMRI) offers fine spatial detail, but its slow hemodynamics and burdensome acquisition limit temporally resolved decoding. Electroencephalography (EEG) and magnetoencephalography (MEG) provide millisecond resolution, making image retrieval compelling: identify the viewed image from neural responses and a fixed candidate bank. Contrastive alignment to pretrained visual representations enables zero-shot retrieval from EEG and MEG, but most systems collapse heterogeneous visual supervision into a single embedding before ranking. This early consolidation imposes one similarity geometry on every candidate order and removes encoder-specific disagreements from the final ranking. We propose CORTIVA, a candidate-score fusion method that retains this complementary evidence. Three decoding routes are aligned to heterogeneous visual targets, score the same indexed candidates independently, and combine only their temperature-scaled score vectors before ranking. On the 200-way THINGS-EEG2 benchmark, CORTIVA reaches 73.5% Top-1 and 95.3% Top-5 across ten participants, exceeding the strongest reported baseline by 10.3 and 5.4 percentage points. With a modality-specific neural encoder, the same fusion principle reaches 42.4% Top-1 on THINGS-MEG. Matched route-removal retraining and four weight controls show that the gain comes from combining complementary route scores and persists under uniform weighting, without depending on a specialized weighting rule. Analyses in an independent DINOv2 space recover the same local error-neighborhood enrichment and posterior neural-visual correspondence. These results support candidate-score fusion as a simple, testable alternative to embedding-level consolidation for neural image retrieval.
Junhan Wang, Kani Chen
The Hong Kong University of Science and Technology, Hong Kong SAR, China
We present a brain-to-image system that decodes visual stimuli from EEG signals recorded during natural image viewing. Our system addresses two tasks: (1) EEG-to-image retrieval, which ranks the correct stimulus image among 200 candidates given an EEG segment, and (2) EEG-to-image reconstruction, which generates an image consistent with the perceived stimulus. For retrieval, we implement a multi-level blurring approach improved with biologically inspired EVNet features and trained with the InfoNCE loss. Evaluated over 10 random seeds for a single subject, the retrieval model achieves a mean final-epoch Top-1 accuracy of 86.30% and Top-5 accuracy of 98.55%. For reconstruction, we implement CognitionCapturerPro, which aligns EEG representations to multi-modal CLIP embeddings, including image, text, depth, and edge embeddings, and synthesizes images with SDXL-Turbo conditioned via IP-Adapter. Averaged over 10 seeds, the reconstruction model achieves a CLIP score of 0.903 using ViT-H-14, a CLIP score of 0.870 using ViT-L/14, and an SSIM of 0.409. These results demonstrate the feasibility of decoding rich visual representations from EEG signals using modern multi-modal alignment and generative modeling techniques.