BrainBind: Bridging Synchronized EEG and fMRI through Long-Form Naturalistic Video Representations

Shixiao Wang; Rining Wu; Jian K. Liu

cs.AINov 3, 2026

BrainBind: Bridging Synchronized EEG and fMRI through Long-Form Naturalistic Video Representations

Authors: Shixiao Wang, Rining Wu, Jian K. Liu

Organizations: School of Computer Science, Center for Human Brain Health, University of Birmingham, UK · School of Computer Science, University of Leeds, UK

Abstract

Real-world vision is dynamic and scene-rich, while EEG and fMRI resolve it at complementary temporal and spatial scales. Most neural decoding systems, however, align one modality in isolation and mostly against static images or short clips. We study the synchronized case: EEG and fMRI are recorded during the same long-form naturalistic viewing, giving a shared stimulus but also recurring narrative context and many semantically similar candidates. We present BrainBind, a framework that maps synchronized EEG and fMRI into frozen long-form video embedding spaces and treats those spaces as common visual-semantic references, not literal neural codes. On CineBrain, a five-seed Recall@K 2 x 4 matrix across two video anchors and four system configurations shows distinct retrieval profiles. With Omni, fMRI organizes the full candidate list most strongly (MRR = 23.82, nDCG@10 = 25.31), while Joint-Dual gives the highest mean label-aware R@1 (13.52%) with a distinct rank-wide profile. The complementary episode-disjoint evaluation quantifies the contribution of repeated stimulus identity and narrative context to the random split. Across these experiments, proxy choice and system configuration produce distinct top-rank and rank-wide retrieval profiles within the present synchronized dataset. The code is available at https://github.com/Forainest789/Brainbind.

Figures & tables

Figure 1: Overview of BrainBind. Synchronized EEG and fMRI responses are mapped into a frozen long-form video proxy space, which provides a shared visual-semantic geometry for comparing modality-specific and fused neural–video retrieval. The dashed reconstruction path denotes the optional $L_{1}$ reconstruction loss $\mathcal{L}_{\mathrm{rec}}$ used only in designated diagnostic configurations; the core systems use contrastive-only training. The rightmost branch is retrieval-conditioned text-to-video visualization rather than direct neural reconstruction.

Experiment family	Backbone	Total parameters
EEG / fMRI / Both, Omni	Transformer [ 36 ]	49,907,712
EEG / fMRI / Both, ImageBind	Transformer [ 36 ]	47,809,536
Joint-Dual, Omni	2 $\times$ S4AE (structured SSM) [ 15 ]	293,835,134
Joint-Dual, ImageBind	2 $\times$ S4AE (structured SSM) [ 15 ]	191,595,902
EEG S4AE-matched, Omni	S4AE (structured SSM) [ 15 ]	118,541,167
fMRI S4AE-matched, Omni	S4AE (structured SSM) [ 15 ]	118,541,167

Table 1: Model parameter counts for the reported configurations.

Experiment	Runs	Avg. epochs	Speed (it/s)	Duration (min/run)	Best val mean R@1 (%)
EEG–Omni Transformer	5	50.0	0.067	103.50	5.17
fMRI–Omni Transformer	5	50.0	0.082	83.14	22.33
Both–Omni Transformer	5	50.0	0.063	111.30	20.69
Joint-Dual–Omni 2 $\times$ S4AE	5	50.0	0.012	585.22	16.54
EEG–Omni S4AE-matched	5	24.4	0.041	82.44	1.10
fMRI–Omni S4AE-matched	5	46.8	0.027	357.98	16.76

Table 2: Training duration and throughput for runs with recorded timing.

Anchor	Input	R@1	R@5	R@10	R@20
Omni	EEG	1.51 $\pm$ 0.13	6.09 $\pm$ 0.90	11.26 $\pm$ 1.55	20.16 $\pm$ 1.82
Omni	fMRI	9.81 $\pm$ 0.16	27.70 $\pm$ 0.64	42.71 $\pm$ 0.31	59.85 $\pm$ 0.53
Omni	Both	8.41 $\pm$ 0.30	25.08 $\pm$ 0.40	39.34 $\pm$ 0.41	56.04 $\pm$ 0.69
Omni	J-Dual	10.28 $\pm$ 0.24	17.47 $\pm$ 0.54	24.20 $\pm$ 1.17	33.35 $\pm$ 1.78
ImageBind	EEG	0.65 $\pm$ 0.05	2.61 $\pm$ 0.11	4.77 $\pm$ 0.13	9.23 $\pm$ 0.26
ImageBind	fMRI	3.90 $\pm$ 0.42	12.90 $\pm$ 0.82	20.97 $\pm$ 1.03	32.68 $\pm$ 0.86

Table 3: Label-aware CineBrain bidirectional retrieval matrix. Values are percentages, reported as mean

\pm

std across five seeds. Both directions use the same evaluation pair pool with query and gallery roles reversed. EEG and fMRI use single-modality Transformers; Both uses cross-attention fusion with a Dual-Transformer; J-Dual uses mean fusion with two S4AE branches. Best R@1 values within each anchor and direction are bold.

Anchor	Input	Fusion	Backbone	R@1	MRR	mAP	nDCG@10
Omni	EEG	single	Transformer	1.91 $\pm$ 0.28	6.60 $\pm$ 0.58	6.03 $\pm$ 0.49	6.06 $\pm$ 0.75
Omni	fMRI	single	Transformer	12.71 $\pm$ 0.24	23.82 $\pm$ 0.29	21.37 $\pm$ 0.19	25.31 $\pm$ 0.20
Omni	Both	cross-attn	Dual-Transformer	11.23 $\pm$ 0.30	21.91 $\pm$ 0.18	19.61 $\pm$ 0.11	23.22 $\pm$ 0.20
Omni	J-Dual	mean	2 $\times$ S4AE	13.52 $\pm$ 0.11	18.69 $\pm$ 0.35	15.09 $\pm$ 0.39	16.52 $\pm$ 0.51
ImageBind	EEG	single	Transformer	0.61 $\pm$ 0.05	2.83 $\pm$ 0.06 †	2.71 $\pm$ 0.06 †	2.20 $\pm$ 0.06 †
ImageBind	fMRI	single	Transformer	6.12 $\pm$ 0.52	13.27 $\pm$ 0.67	11.41 $\pm$ 0.53	12.99 $\pm$ 0.72

Table 4: Ranking-aware evaluation for the controlled retrieval matrix. Mean R@1, MRR, mAP, and nDCG@10 are averaged across the s2v and v2s directions. † EEG–ImageBind ranking metrics are averaged over four seeds because one seed lacks ranking-metric outputs.

Comparison	Metric	$\Delta$ pp	$p$	Interpretation
Both vs. fMRI (Omni)	mean R@1	-1.48	0.0012	System-level difference
Both vs. fMRI (Omni)	nDCG@10	-2.09	$3.3{\times}10^{-5}$	Rank-structure difference
J-Dual vs. fMRI (Omni)	mean R@1	+0.81	0.0008	Top-rank difference
J-Dual vs. fMRI (Omni)	MRR	-5.13	$8.5{\times}10^{-7}$	Rank-organization difference
J-Dual vs. fMRI (Omni)	mAP	-6.28	$9.7{\times}10^{-7}$	Candidate-space difference
J-Dual vs. fMRI (Omni)	nDCG@10	-8.79	$1.2{\times}10^{-6}$	Rank-structure difference

Table 5: Paired seed-level comparisons. Positive

\Delta

indicates the first configuration is higher. The Joint-Dual rows quantify the difference between precise matching and rank-wide organization; the anchor row quantifies the consistent Omni advantage.

Panel of Figure 2 — Figure 2: Embedding-geometry diagnostics for Omni-aligned brain representations. The top block compares retrieval-colored 2D projections (t-SNE and UMAP) for the four Omni-aligned brain embeddings on a 3,000-sample diagnostic subset. These subset metrics are distinct from the full-pool, multi-seed aggregates in Tables 3 and 4 . The bottom block reports rank-space and high-dimensional neighborhood statistics: Top-K retrieval curve, median first-positive rank, 20-NN cosine distance, density Gini coefficient, and Top-5 retrieval within the densest 1% neighborhoods. Projections are qualitative; metric panels are computed in retrieval rank space or the original high-dimensional embedding space. Higher Gini values indicate stronger local density non-uniformity rather than a quality ranking.

Panel of Figure 3 — Figure 3: Cross-anchor geometry of the two frozen video proxies. ImageBind concentrates more variance in fewer principal components; Omni has broader spectral support and lower redundancy. Independently fitted PCA followed by in-sample CCA characterizes their linear correspondence.

Objective	R@1	R@5	R@10	R@20	MRR	mAP
InfoNCE, contrastive-only	0.080 $\pm$ 0.025	0.345 $\pm$ 0.047	0.532 $\pm$ 0.078	0.977 $\pm$ 0.105	0.393 $\pm$ 0.028	0.343 $\pm$ 0.009
SupCon, contrastive-only	0.111 $\pm$ 0.038	0.448 $\pm$ 0.008	0.782 $\pm$ 0.047	1.258 $\pm$ 0.088	0.477 $\pm$ 0.023	0.376 $\pm$ 0.030
InfoNCE, reconstruction-assisted	0.080 $\pm$ 0.025	0.316 $\pm$ 0.074	0.689 $\pm$ 0.103	1.175 $\pm$ 0.171	0.435 $\pm$ 0.057	0.391 $\pm$ 0.020
SupCon, reconstruction-assisted	0.136 $\pm$ 0.072	0.399 $\pm$ 0.098	0.648 $\pm$ 0.167	1.052 $\pm$ 0.161	0.464 $\pm$ 0.081	0.375 $\pm$ 0.028

Table 6: EEG-only S4AE loss ablation on the episode-disjoint split (percentages; mean

\pm

std over three seeds).

$\tau$	R@1	R@5	R@10	R@20	MRR	mAP
0.01	0.087 $\pm$ 0.009	0.370 $\pm$ 0.027	0.777 $\pm$ 0.062	1.350 $\pm$ 0.124	0.465 $\pm$ 0.007	0.387 $\pm$ 0.024
0.05	0.180 $\pm$ 0.058	0.486 $\pm$ 0.034	0.936 $\pm$ 0.085	1.674 $\pm$ 0.115	0.614 $\pm$ 0.025	0.486 $\pm$ 0.036
0.10	0.098 $\pm$ 0.035	0.322 $\pm$ 0.066	0.746 $\pm$ 0.103	1.394 $\pm$ 0.180	0.465 $\pm$ 0.090	0.370 $\pm$ 0.066
0.20	0.123 $\pm$ 0.041	0.396 $\pm$ 0.085	0.671 $\pm$ 0.027	1.168 $\pm$ 0.120	0.471 $\pm$ 0.003	0.367 $\pm$ 0.063

Table 7: EEG SupCon reconstruction-assisted Transformer temperature sweep on the episode-disjoint split (percentages; mean

\pm

std over three seeds).

Setting	Split / pool	s2v@1	v2s@1	Mean R@1	MRR	nDCG@10
Label-aware theoretical chance	6480	0.026	0.026	0.026	0.222	0.081
Random brain, Omni	6480	0.015	0.015	0.015	0.149	0.052
Shuffled video, Omni	6480	0.000	0.000	0.000	0.125	0.045
Random brain, ImageBind	6480	0.015	0.000	0.008	0.151	0.052
Shuffled video, ImageBind	6480	0.000	0.000	0.000	0.129	0.054

Table 8: Retrieval controls (percentages). Theoretical chance and random-brain rows are label-aware; shuffled-video rows use one-to-one row correspondence after permuting the video gallery.

Model	Subject	R@1	R@5	R@10	R@20	MRR	mAP
fMRI–Omni	sub-0001	5.234 $\pm$ 5.239	16.501 $\pm$ 13.148	24.103 $\pm$ 15.745	31.956 $\pm$ 14.967	11.083 $\pm$ 8.145	11.085 $\pm$ 8.146
fMRI–Omni	sub-0002	0.050 $\pm$ 0.037	0.197 $\pm$ 0.053	0.367 $\pm$ 0.093	0.768 $\pm$ 0.157	0.288 $\pm$ 0.061	0.288 $\pm$ 0.061
J-Dual–Omni	sub-0001	24.150 $\pm$ 1.944	42.934 $\pm$ 1.379	47.909 $\pm$ 0.725	49.792 $\pm$ 0.241	32.203 $\pm$ 1.652	32.203 $\pm$ 1.653
J-Dual–Omni	sub-0002	0.069 $\pm$ 0.012	0.274 $\pm$ 0.059	0.505 $\pm$ 0.120	0.860 $\pm$ 0.194	0.320 $\pm$ 0.019	0.320 $\pm$ 0.019

Table 9: Selective three-seed held-out-training checks (percentages; mean

\pm

std over seeds). Twenty percent of each held-out subject’s samples are used for checkpoint selection; the remaining samples form the test gallery.

Anchor	Input	Probe	R@1	s2v@1	v2s@1	s2v@5	v2s@5	s2v@10	v2s@10
ImageBind	EEG	Linear	0.55	0.50	0.60	1.15	1.90	2.05	3.80
Omni	EEG	Linear	1.05	0.95	1.15	3.20	3.65	6.20	5.60
ImageBind	EEG	MLP	0.80	0.80	0.80	2.45	2.30	3.00	4.00
Omni	EEG	MLP	1.23	1.15	1.30	4.05	4.55	6.80	7.35
ImageBind	fMRI	MLP	18.58	17.20	19.95	27.75	35.45	34.30	43.15
Omni	fMRI	MLP	22.85	20.60	25.10	47.25	52.30	58.40	64.95

Table 10: Low-capacity probe results on a 2,000-candidate CineBrain random-split gallery (percentages). Linear and MLP probes use shallow projections into the frozen video proxy space and the same label-aware bidirectional retrieval protocol as BrainBind.

Model	$n$	R@1	R@5	R@10	R@20
Chance floor	–	0.069	0.347	0.693	1.381
EEG	3	0.075 $\pm$ 0.029	0.363 $\pm$ 0.109	0.579 $\pm$ 0.182	1.224 $\pm$ 0.323
fMRI	3	0.129 $\pm$ 0.073	0.417 $\pm$ 0.250	0.908 $\pm$ 0.244	1.641 $\pm$ 0.373
Both	3	0.126 $\pm$ 0.093	0.453 $\pm$ 0.200	0.893 $\pm$ 0.462	1.402 $\pm$ 0.585
J-Dual	3	0.216 $\pm$ 0.051	0.615 $\pm$ 0.083	1.026 $\pm$ 0.109	1.988 $\pm$ 0.054

Table 11: Episode-disjoint label-aware retrieval (percentages; mean

\pm

std over three coupled model/split-seed runs). No season–episode group appears in more than one split; the chance row reflects the repeated-positive structure.

Anchor / input	Direction	$n$	R@1	R@5	R@10	R@20
Omni / J-Dual	s2v	3	10.08 $\pm$ 0.32	17.44 $\pm$ 0.51	22.96 $\pm$ 1.13	30.92 $\pm$ 1.25
Omni / J-Dual	v2s	3	18.68 $\pm$ 0.42	27.14 $\pm$ 0.81	35.06 $\pm$ 1.47	44.98 $\pm$ 2.04
ImageBind / EEG	s2v	5	0.09 $\pm$ 0.06	0.30 $\pm$ 0.09	0.51 $\pm$ 0.10	0.95 $\pm$ 0.18
ImageBind / EEG	v2s	5	0.20 $\pm$ 0.03	0.82 $\pm$ 0.07	1.49 $\pm$ 0.11	2.69 $\pm$ 0.30

Table 12: Targeted S4AE configurations on CineBrain bidirectional retrieval. Values are percentages;

n

is the number of seeds. The reduced-width J-Dual configuration uses mean fusion and contrastive-only training. The single-modality EEG–ImageBind configuration jointly optimizes contrastive alignment and reconstruction. Widths, objectives, anchors, and seed sets differ, so these are configuration diagnostics rather than a factorial backbone ablation.

Anchor	Input	Fusion	Configuration	$n$	Mean R@1	MRR	mAP	nDCG@10
Omni	J-Dual	mean	reduced S4AE, contrastive	3	14.38 $\pm$ 0.35	19.14 $\pm$ 0.56	15.28 $\pm$ 0.52	16.89 $\pm$ 0.66
ImageBind	EEG	single	S4AE + reconstruction	5	0.15 $\pm$ 0.03	0.59 $\pm$ 0.05	0.50 $\pm$ 0.04	0.39 $\pm$ 0.04

Table 13: Ranking-aware metrics for the targeted S4AE configurations. Values are percentages, mean

\pm

std over

n

seeds.

Control	s2v@1	s2v@5	s2v@10	s2v@20	v2s@1	v2s@5	v2s@10	v2s@20
One-to-one theoretical chance	0.015	0.077	0.154	0.309	0.015	0.077	0.154	0.309
Label-aware theoretical chance	0.026	0.131	0.262	0.523	0.026	0.131	0.262	0.523
Label-aware random brain	0.015	0.046	0.077	0.123	0.015	0.093	0.139	0.401
Shuffled video features	0.000	0.062	0.108	0.262	0.000	0.046	0.123	0.355

Table 14: Omni chance and shuffled-feature controls. Label-aware rows use same- stim_label positives; the shuffled-video row reports the one-to-one shuffled-feature control.

Control	s2v@1	s2v@5	s2v@10	s2v@20	v2s@1	v2s@5	v2s@10	v2s@20
One-to-one theoretical chance	0.015	0.077	0.154	0.309	0.015	0.077	0.154	0.309
Label-aware theoretical chance	0.026	0.131	0.262	0.523	0.026	0.131	0.262	0.523
Label-aware random brain	0.015	0.062	0.139	0.278	0.000	0.077	0.108	0.231
Shuffled video features	0.000	0.031	0.077	0.293	0.000	0.108	0.231	0.309

Table 15: ImageBind chance and shuffled-feature controls. Label-aware rows use same- stim_label positives; the shuffled-video row reports the one-to-one shuffled-feature control.

Anchor	Control	Mean R@1	Mean MRR	Mean mAP	Mean nDCG@10
Omni	Label-aware theoretical chance	0.026	0.222	0.155	0.081
Omni	Label-aware random brain	0.015	0.149	0.148	0.052
Omni	Shuffled video features	0.000	0.125	0.125	0.045
ImageBind	Label-aware theoretical chance	0.026	0.222	0.155	0.081
ImageBind	Label-aware random brain	0.008	0.151	0.154	0.052
ImageBind	Shuffled video features	0.000	0.129	0.129	0.054

Table 16: Ranking-aware chance and control metrics. The label-aware theoretical chance accounts for multiple same-label positives per query. The shuffled-video rows use the one-to-one shuffled-feature control.

Input	Probe	Mean R@1	s2v@1	v2s@1	s2v@5	v2s@5	s2v@10	v2s@10
EEG	Linear	0.32	0.29	0.35	1.02	1.68	1.87	2.87
EEG	MLP	0.40	0.29	0.51	1.36	2.28	2.27	3.60

Table 17: Held-out test results for completed EEG–Omni probe runs. Values are percentages. The probes and full EEG–Omni encoder use the same 6,480-candidate test gallery and label-aware retrieval protocol.

Figure 4: Qualitative retrieval-conditioned visualization via the Omni pipeline. The fMRI embedding retrieves text in Omni space, and the fixed Wan2.1-T2V-1.3B generator [ 37 ] produces the displayed frames. The panel shows three of ten retained outputs from the 16-query generation pass. It visualizes a neural $\rightarrow$ proxy $\rightarrow$ text $\rightarrow$ video pathway rather than direct neural reconstruction.

Study / family	Signal	Stimulus / supervision	Output target	Scale interpretation
MindEye2 [ 32 ]	fMRI	NSD static natural images	Image retrieval and diffusion reconstruction	Strong low-data static-image reference; the target is an image embedding/generator, not video dynamics.
MinD-Video [ 7 ]	fMRI	Dynamic visual stimuli	Video reconstruction	Direct fMRI-to-video generation reference; it does not study synchronized EEG–fMRI proxy retrieval.
NeuroClips [ 14 ]	fMRI	Natural video clips	Smooth fMRI-to-video reconstruction	High-fidelity video-generation reference optimized for perceptual synthesis, not full-split ranking.
Brain Netflix [ 9 ]	fMRI	Scaled natural video viewing	Video reconstruction from brain activity	Closest large-scale video-generation reference; it focuses on fMRI synthesis and data scaling.
Animate Your Thoughts / Mind-Animator [ 29 ]	fMRI	Dynamic natural vision	Decoupled video reconstruction	Recent fMRI-to-video generation reference; its target is reconstruction, not shared EEG–fMRI retrieval geometry.
BrainBind (ours)	EEG + fMRI	CineBrain long-form naturalistic video	Bidirectional retrieval, proxy diagnostics, and retrieval-conditioned visualization	Studies whether synchronized EEG and fMRI can be organized in one frozen long-form video reference geometry.

Table 18: Video-generation and static-image external references as scale context. These studies are not direct leaderboard comparisons: they differ in stimulus regime, signal modality, and output target. They define the landscape against which BrainBind’s distinct contribution is positioned.

Model	Held-out subject	Seed	Pairs	Mean R@1	R@5	R@10	R@20	MRR	mAP
fMRI–Omni	sub-0001	42	4321	11.247	31.486	41.865	48.554	20.368	20.372
fMRI–Omni	sub-0001	123	4321	2.800	11.120	18.584	27.829	7.737	7.738
fMRI–Omni	sub-0001	456	4321	1.655	6.897	11.861	19.486	5.145	5.145
fMRI–Omni	sub-0002	42	4320	0.035	0.185	0.359	0.671	0.270	0.270
fMRI–Omni	sub-0002	123	4320	0.093	0.255	0.463	0.949	0.356	0.356
fMRI–Omni	sub-0002	456	4320	0.023	0.150	0.278	0.683	0.237	0.237

Table 19: Selective held-out-training checks (percentages). Mean R@1, R@5, R@10, R@20, MRR, and mAP average the neural-to-video and video-to-neural directions.

Property	Value
GPUs	2 $\times$ NVIDIA A100 80GB PCIe
GPU memory	80 GB per GPU
CUDA	12.1
cuDNN	90100
PyTorch	2.5.1+cu121
Launch style	accelerate launch --num_processes=2

Table 20: Hardware and software environment.

Cross-Modal Alignment

Cross-modal alignment maps representations from different data types into compatible spaces so models can compare, transfer, and combine information.

See all 458 papers

What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment
Jun 24, 2026Salini Yadav, Taveena Lotey, Pravendra Singh +1Representation Learning EEG-Based Visual Decoding
Department of CSE IIT Roorkee, India · Department of CSE IIT (ISM) Dhanbad, India
CORTIVA: Candidate-Score Fusion of Complementary Visual Teachers for EEG- and MEG-to-Image Retrieval
Aug 2, 2026Junhan Wang, Kani ChenEEG-Based Visual Decoding Cross-Modal Alignment
The Hong Kong University of Science and Technology, Hong Kong SAR, China
FSDBN: Foreground-Aware EEG-Visual Alignment via Dynamic Brain Networks
Jul 20, 2026Yiheng Liu, Chuhang Zheng, Peiliang Gong +3EEG-Based Visual Decoding Cross-Modal Alignment
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The International Joint Institute of Tianjin University, Fuzhou, Tianjin University, Tianjin, China

Jun 24, 2026J/K move · Enter open · S save

What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment

Salini Yadav, Taveena Lotey, Pravendra Singh, Partha Pratim Roy

Department of CSE IIT Roorkee, India · Department of CSE IIT (ISM) Dhanbad, India

Representation Learning EEG-Based Visual Decoding Cross-Modal Alignment Multi-View Learning

Open paper AskPDF arXiv

Zero-shot visual decoding from electroencephalography (EEG) aims to infer visual semantics from non-invasive neural recordings, but remains challenging due to the low signal-to-noise ratio, non-stationarity, and limited spatial resolution of EEG. Existing EEG-vision alignment methods often rely on holistic EEG embeddings, which can obscure the complementary temporal, spectral, and spatial structure underlying visual perception. We introduce a unified multiview EEG representation learning framework for aligning brain responses with visual semantic embeddings. Our method builds an EEG encoder that jointly models three complementary views: input-conditioned state-space temporal dynamics, learnable wavelet-based spectral decomposition for sample-adaptive frequency modeling, and attention-modulated graph learning for structured electrode interactions. The resulting multiview EEG embeddings are fused and aligned with pretrained visual representations in a shared semantic space using contrastive learning with EEG-specific regularization, enabling 200-way zero-shot visual classification. Experiments on THINGS-EEG benchmark show that our method achieves state-of-the-art performance, with 54.8% Top-1 and 85.6% Top-5 accuracy in the within-subject setting and 15.3% Top-1 and 45.4% Top-5 accuracy in the cross-subject setting. We further present the first systematic cross-session EEG-image decoding evaluation, achieving 40.8% Top-1 and 78.0% Top-5 accuracy. These results suggest that explicitly modeling multiview neural structure improves both semantic alignment and generalization in EEG-based visual decoding.

Abstract

Cross-Modal Alignment

Explore similar work

What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment

CORTIVA: Candidate-Score Fusion of Complementary Visual Teachers for EEG- and MEG-to-Image Retrieval

FSDBN: Foreground-Aware EEG-Visual Alignment via Dynamic Brain Networks