ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding

Kanglei Zhou; Chunyan Lan; Dongyang Li; Jun Zhu; Liyuan Wang

cs.AISep 4, 2026

ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding

Authors: Kanglei Zhou, Chunyan Lan, Dongyang Li, Jun Zhu, Liyuan Wang

Organizations: Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China · School of Life Sciences, Tsinghua University, Beijing, China · Department of Computer Science and Technology, Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint Center for ML, Tsinghua University, Beijing, China

Abstract

Electroencephalogram (EEG) visual decoding aims to recover visual semantics from non-invasive neural time-series signals, for which robust alignment between noisy neural responses and stable semantic representations is key to achieving high-performance decoding. Despite recent advances in contrastive learning, robust EEG decoding remains challenging because existing methods rely on fixed visual or textual anchors whose semantic relations may become misaligned with EEG representations that vary across trials, subjects, and learning stages. Our empirical evidence shows that this instability appears across both standard EEG decoding protocols and more challenging robustness settings, including strict cross-subject transfer and realistic personalized continual adaptation. We provide a formal analysis showing that fixed semantic supervision can bias optimization when EEG-specific relations evolve, and that structure-agnostic perturbations may distort semantically important EEG components. To address these issues, we propose Progressive Contrastive Alignment (ProCA), a unified and model-agnostic framework for adaptive neural-semantic alignment. ProCA progressively refines class-level contrastive supervision from frozen vision-language priors to EEG-aware semantic relations, and introduces structure-consistent interpolation to constrain feature mixing according to channel-wise and temporal importance. Across subject-dependent, subject-independent, strict cross-subject transfer, and continual adaptation settings, ProCA achieves average relative Top-1/Top-5 gains of 7.4%/3.9%, 10.0%/4.6%, 28.1%/17.8%, and 16.8%/11.6%, respectively.

Figures & tables

Panel of Figure 1 — Fig. 1 : Motivation and method comparison. (a) EEG visual decoding aims to align neural responses with visual semantics, but is challenged by subject variability and non-stationary data distributions. (b) Under realistic cross-subject generalization and continual adaptation settings, the baseline method [ 1 ] exhibits substantial performance degradation, whereas our method consistently improves decoding robustness, especially in continual adaptation. (c) The baseline loss landscape contains dense and sharp contours, reflecting unstable optimization and poor robustness to distribution shifts. (d) Our method produces a smoother and flatter loss landscape with lower loss values, indicating more stable and generalizable optimization. (e) Representative image reconstruction results, including successful cases ( ) and failure cases ( ).

Fig. 2 : Overview of ProCA . ProCA learns EEG representations by progressively aligning them with frozen vision–language embeddings through confusion-guided contrastive supervision (see Sect. IV-B ), while applying structure-consistent interpolation to preserve important channel-wise and temporal EEG components (see Sect. IV-C ).

Method	S01		S02		S03		S04		S05		S06		S07		S08		S09		S10		Average
Method	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5
	200-Way
EEGNetV4 [ 3 ]	$16.0$	$40.2$	$15.3$	$40.8$	$19.6$	$48.3$	$21.2$	$49.3$	$15.8$	$32.6$	$18.3$	$44.6$	$18.3$	$41.0$	$29.6$	$60.2$	$15.6$	$44.7$	$23.5$	$52.6$	$19.3$	$45.4$
EEGNetV4 + Ours	$16.8$	$43.3$	$16.6$	$44.5$	$23.2$	$52.6$	$22.5$	$50.6$	$16.0$	$35.3$	$20.5$	$47.5$	$18.0$	$45.5$	$32.3$	$64.1$	$19.3$	$46.5$	$24.4$	$54.0$	$21.0$	$48.4$
Improvement ( $\Delta$ , %)	+5.0	+7.7	+8.5	+9.1	+18.4	+8.9	+6.1	+2.6	+1.3	+8.3	+12.0	+6.5	-1.6	+11.0	+9.1	+6.5	+23.7	+4.0	+3.8	+2.7	+8.8	+6.6
NICE [ 10 ]	$21.3$	$50.9$	$20.1$	$42.9$	$26.2$	$54.8$	$27.8$	$61.3$	$18.0$	$42.0$	$28.1$	$52.0$	$22.5$	$54.3$	$37.2$	$68.6$	$19.1$	$49.0$	$29.0$	$61.3$	$24.9$	$54.1$

TABLE I : Overall subject-dependent Top-1 (T1) and Top-5 (T5) accuracy (%) comparison on the THINGS-EEG2 dataset under the 200-, 150-, and 100-way zero-shot object recognition settings.

Method	S01		S02		S03		S04		S05		S06		S07		S08		S09		S10		S11		S12		S13		S14		S15		S16		S17		S18		S19		S20		Average
Method	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5
	200-Way
CognitionCapturer [ 13 ]	$9.2$	$30.8$	$12.8$	$37.2$	$10.7$	$35.8$	$8.2$	$25.5$	$12.2$	$34.0$	$12.3$	$38.2$	$7.2$	$22.3$	$6.2$	$19.8$	$5.2$	$15.7$	$11.2$	$35.8$	$4.0$	$14.2$	$8.5$	$35.2$	$8.8$	$23.0$	$11.3$	$36.3$	$5.5$	$23.3$	$7.7$	$20.8$	$5.0$	$18.2$	$11.0$	$37.8$	$4.7$	$16.0$	$9.2$	$31.5$	$8.5$	$27.6$
CognitionCapturer + Ours	$10.8$	$33.7$	$14.0$	$39.8$	$12.3$	$38.0$	$9.0$	$27.3$	$13.3$	$35.2$	$14.3$	$40.0$	$7.7$	$23.7$	$7.0$	$21.0$	$6.7$	$18.0$	$12.3$	$36.3$	$5.2$	$15.8$	$10.2$	$37.0$	$10.3$	$23.8$	$12.2$	$37.3$	$7.2$	$25.5$	$8.0$	$22.3$	$6.7$	$20.5$	$12.7$	$39.3$	$5.0$	$18.2$	$10.3$	$32.5$	$9.8$	$29.3$
Improvement ( $\Delta$ , %)	+17.4	+9.4	+9.4	+7.0	+15.0	+6.1	+9.8	+7.1	+9.0	+3.5	+16.3	+4.7	+7.0	+6.3	+12.9	+6.1	+28.8	+14.6	+9.8	+1.4	+30.0	+11.3	+20.0	+5.1	+17.0	+3.5	+8.0	+2.8	+30.9	+9.4	+3.9	+7.2	+34.0	+12.6	+15.5	+4.0	+6.4	+13.8	+12.0	+3.2	+15.3	+6.2
ATMS [ 1 ]	$4.5$	$18.3$	$12.5$	$33.7$	$11.3$	$31.5$	$5.5$	$21.0$	$8.7$	$24.2$	$11.3$	$28.3$	$8.7$	$23.8$	$5.2$	$13.8$	$3.8$	$11.2$	$11.5$	$30.5$	$4.8$	$16.7$	$9.5$	$31.5$	$8.0$	$25.3$	$16.7$	$38.8$	$10.5$	$24.2$	$5.2$	$15.8$	$5.3$	$17.0$	$11.3$	$30.7$	$4.5$	$9.8$	$13.8$	$32.7$	$8.6$	$23.9$

TABLE II : Overall subject-dependent Top-1 (T1) and Top-5 (T5) accuracy (%) comparison on the Alljoined-1.6M dataset under the 200-, 150-, and 100-way settings.

Method	S01		S02		S03		S04		S05		S06		S07		S08		S09		S10		Average
Method	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5
	200-Way
ATMS [ 1 ]	$10.3$	$26.0$	$10.8$	$29.7$	$8.0$	$22.3$	$13.2$	$29.5$	$9.2$	$25.0$	$10.3$	$31.2$	$7.3$	$23.8$	$9.3$	$33.0$	$8.8$	$24.8$	$15.2$	$35.5$	$10.3$	$28.1$
ATMS + Ours	$12.2$	$28.0$	$12.3$	$31.8$	$8.8$	$24.5$	$14.3$	$32.2$	$10.5$	$26.2$	$12.2$	$33.2$	$8.3$	$25.7$	$10.5$	$35.2$	$10.2$	$26.0$	$16.2$	$38.5$	$11.6$	$30.1$
Improvement ( $\Delta$ , %)	+17.8	+7.7	+13.8	+7.3	+10.4	+9.7	+8.8	+9.0	+14.5	+4.7	+17.7	+6.4	+13.6	+7.7	+12.5	+6.6	+15.2	+4.7	+6.6	+8.5	+12.7	+7.3
CognitionCapturer [ 13 ]	$16.2$	$42.3$	$11.7$	$41.7$	$8.0$	$30.8$	$14.3$	$44.7$	$10.5$	$30.2$	$10.7$	$40.5$	$11.8$	$43.0$	$13.0$	$37.2$	$10.7$	$37.8$	$19.3$	$53.5$	$12.6$	$40.2$

TABLE III : Leave-one-subject-out subject-independent generalization comparison on the THINGS-EEG2 dataset under 200-, 150-, and 100-way retrieval settings. Top-1 (T1) and Top-5 (T5) accuracy (%) are reported.

Method	S1		S2		S3		S4		S5		S6		S7		S8		S9		S10		Average
Method	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5
	200-Way
ATMS [ 1 ]	$12.27$	$33.21$	$8.67$	$27.92$	$14.67$	$39.50$	$20.08$	$46.33$	$11.67$	$33.50$	$17.17$	$45.08$	$15.83$	$40.83$	$26.42$	$56.75$	$17.08$	$39.25$	$22.92$	$53.92$	$16.68$	$41.63$
LwF [ 33 ]	$14.80$	$36.60$	$10.30$	$31.70$	$15.00$	$41.10$	$21.40$	$48.90$	$12.20$	$32.70$	$18.00$	$45.00$	$18.30$	$46.20$	$28.40$	$60.90$	$16.70$	$41.40$	$25.60$	$54.70$	$18.07$	$43.92$
EWC [ 32 ]	$15.10$	$37.10$	$12.20$	$33.80$	$17.40$	$40.30$	$23.80$	$49.50$	$12.00$	$34.50$	$18.40$	$46.50$	$16.80$	$44.60$	$27.00$	$59.40$	$17.92$	$40.75$	$26.50$	$57.10$	$18.71$	$44.36$
ER [ 34 ]	$13.71$	$36.42$	$12.07$	$35.57$	$16.50$	$43.14$	$22.86$	$49.21$	$12.36$	$33.93$	$17.93$	$48.14$	$18.57$	$45.57$	$29.57$	65.07	$17.64$	$43.21$	$25.29$	$58.07$	$18.65$	$45.83$

TABLE IV: Continual EEG visual decoding comparison on the THINGS-EEG2 dataset under 200-, 150-, and 100-way retrieval settings. Top-1 (T1) and Top-5 (T5) accuracy (%) are reported.

Method	S01		S02		S03		S04		S05		S06		S07		S08		S09		S10		S11		S12		S13		S14		S15		S16		S17		S18		S19		S20		Avg
Method	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5	T1	T5
200-Way Setting
Naive	$2.33$	$8.83$	$3.67$	$13.17$	$4.17$	$13.67$	$3.17$	$11.83$	$6.67$	$20.67$	$5.00$	$19.50$	$3.50$	$11.33$	$0.83$	$3.17$	$1.17$	$4.83$	$2.67$	$10.83$	$2.33$	$7.50$	$4.67$	$17.17$	$1.67$	$11.33$	$8.33$	$25.17$	$3.00$	$13.17$	$2.00$	$7.33$	$2.00$	$10.33$	$4.67$	$21.67$	$1.33$	$5.67$	$3.17$	$9.67$	$3.40$	$13.58$
ER [ 34 ]	$3.83$	$12.00$	$6.83$	$19.83$	$4.17$	$16.00$	$3.00$	$11.17$	7.33	$19.50$	$6.50$	$19.67$	$5.33$	$13.67$	$1.17$	$3.33$	$1.67$	$5.00$	$4.00$	$17.17$	$1.67$	$6.50$	$5.83$	$22.17$	$2.17$	$9.33$	$9.17$	$28.00$	$4.50$	$14.67$	$2.50$	$8.00$	$1.33$	$8.17$	$7.67$	$24.17$	$1.67$	$3.83$	$5.17$	$16.50$	$4.40$	$15.06$
DER [ 35 ]	$3.00$	$12.33$	$5.33$	$19.17$	$5.83$	$17.83$	$1.83$	$11.67$	$5.67$	$19.83$	$6.33$	$21.00$	$4.50$	$14.50$	$1.00$	$3.83$	$1.33$	$6.50$	$3.33$	$15.83$	$1.33$	$8.50$	$4.67$	$21.33$	$1.50$	$8.33$	$9.83$	$28.17$	$4.00$	$14.67$	$2.17$	$8.67$	$2.00$	$11.67$	8.33	26.50	$1.17$	$5.50$	$3.00$	$14.50$	$3.98$	$15.42$
ASAL [ 38 ]	$2.67$	$12.83$	$6.17$	$18.33$	$4.50$	$17.33$	$4.83$	$14.83$	$6.17$	$18.17$	$6.83$	$20.67$	$3.33$	$13.33$	$1.83$	$5.00$	$1.17$	$3.83$	$4.67$	$18.33$	$3.83$	$8.50$	$5.17$	$21.67$	$2.17$	$11.83$	$9.17$	$29.83$	$3.83$	$14.17$	$2.00$	$8.50$	$1.33$	$6.33$	$6.00$	$23.67$	$0.83$	$5.00$	$3.50$	$16.33$	$3.77$	$15.41$

TABLE V : Continual EEG visual decoding comparison on the Alljoined-1.6M dataset under 200-, 150-, and 100-way retrieval settings. Top-1 (T1) and Top-5 (T5) accuracy (%) are reported over 20 subjects, with the best result in each column highlighted in bold.

Fig. 3 : Heatmaps of strict cross-subject transfer on the THINGS-EEG2 dataset. 3 to 3 visualize subject-wise transfer results under different metrics and methods, where rows denote source subjects used for training and columns denote unseen target subjects used for evaluation without adaptation. 3 summarizes the average transfer performance across all off-diagonal source–target pairs.

Source	Top-1 Accuracy (%)					Top-5 Accuracy (%)
Source	Baseline		ProCA		$\Delta$	Baseline		ProCA		$\Delta$
S01	$3.00$ $\pm$	$0.34$	$3.83$ $\pm$	$0.40$	$+0.83$	$10.33$ $\pm$	$0.59$	$11.83$ $\pm$	$0.57$	$+1.50$
S02	$2.83$ $\pm$	$0.30$	$4.06$ $\pm$	$0.33$	$+1.23$	$10.61$ $\pm$	$0.58$	$13.06$ $\pm$	$0.62$	$+2.45$
S03	$2.67$ $\pm$	$0.22$	$3.11$ $\pm$	$0.32$	$+0.44$	$10.67$ $\pm$	$0.55$	$12.44$ $\pm$	$0.60$	$+1.77$
S04	$3.56$ $\pm$	$0.32$	$4.44$ $\pm$	$0.32$	$+0.88$	$11.39$ $\pm$	$0.55$	$13.44$ $\pm$	$0.61$	$+2.05$
S05	$2.72$ $\pm$	$0.22$	$3.50$ $\pm$	$0.22$	$+0.78$	$9.50$ $\pm$	$0.55$	$10.72$ $\pm$	$0.54$	$+1.22$

TABLE VI : Per-source-subject stability analysis under strict cross-subject transfer over three independent runs. Each entry reports the average accuracy over all target subjects except the source subject.

Setting	S01	S02	S03	S04	S05	S06	S07	S08	S09	S10	Avg
Top-1 Accuracy (%)
Baseline	23.0	20.6	27.8	30.9	20.8	28.3	27.3	41.1	29.9	35.6	28.5
Ours	23.8	26.2	33.1	32.4	22.0	29.4	29.0	42.1	30.9	37.1	30.6
Ours w/o PCA	21.8	22.8	30.4	31.4	21.1	29.1	28.3	40.5	30.0	36.3	29.2
Ours w/o Prog. Update	23.5	24.0	31.7	31.3	21.0	28.0	27.5	40.8	28.5	35.5	29.2
Ours w/o SCI	24.6	25.3	32.0	32.0	21.6	28.6	27.4	41.3	29.3	35.8	29.8

TABLE VII: Ablation study of ATMS on the 200-Way setting.

Fig. 4 : Sensitivity analysis of ProCA on the 200-way setting. ProCA shows stable Top-1 and Top-5 accuracy over a broad range of

\alpha_{\max}

and stage transition epochs, with only modest variations around the optimum. Results on the 150-way and 100-way settings are provided in Appendix Figs. A11 and A10 , respectively.

Fig. 5 : Comparison with Euclidean Alignment (EA) on subject-independent EEG visual decoding. 5 and 5 show that ProCA achieves the best average Top-1 and Top-5 accuracy across 200-, 150-, and 100-way retrieval settings. 5 and 5 further visualize subject-wise gains of ProCA over NeuroBridge and NeuroBridge with EA, respectively, indicating that ProCA provides consistent improvements beyond conventional alignment preprocessing.

Fig. 6 : Loss landscape sharpness comparison across ten subjects and three EEG visual decoding baselines. The left column reports the maximum Hessian eigenvalue

\lambda_{\max}

, and the right column reports the Hessian trace. Lower values indicate flatter local minima. ProCA generally reduces both measures across baselines, indicating smoother optimization landscapes and improved representation robustness.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

\mathbf{C}^{(0,\mathrm{img})}_{ij}=\mathrm{sim}(\mathbf{s}^{\mathrm{img}}_{i},\mathbf{s}^{\mathrm{img}}_{j}),\quad\mathbf{C}^{(0,\mathrm{txt})}_{ij}=\mathrm{sim}(\mathbf{s}^{\mathrm{txt}}_{i},\mathbf{s}^{\mathrm{txt}}_{j})

Appendix

Algorithm 1 Training Procedure of ProCA

Fig. A7 : t-SNE visualization of EEG feature representations. The left panel shows the baseline method, while the right panel shows our proposed ProCA. ProCA produces more compact and better-separated clusters, indicating improved representation discriminability and robustness under challenging cross-subject shifts and semantic ambiguity.

Fig. A8 : Qualitative comparison of image reconstruction results. For each example, we show the ground-truth stimulus (Seen), the reconstruction produced by the baseline method [ 1 ] (Baseline), and the reconstruction obtained by our method (Ours). The results illustrate that the baseline often generates semantically ambiguous or incorrect reconstructions, while our method recovers more discriminative and semantically consistent visual content across a diverse range of visual categories.

Fig. A9 : Representative failure cases of EEG-based image reconstruction. Compared with the baseline, our method generally yields more semantically consistent reconstructions, but still fails for visually complex or ambiguous stimuli, reflecting inherent limitations that remain challenging in real-world EEG reconstruction and deployment settings.

Fig. A10 : Hyperparameter sensitivity analysis on the 100-way setting. Top: Top-1 accuracy; Bottom: Top-5 accuracy. Left: effect of the maximum fusion coefficient

\alpha_{\max}

with the transition epoch fixed to 25. Right: effect of the stage transition epoch with

\alpha_{\max}=0.8

. ProCA exhibits stable performance across a wide range of hyperparameters, with optimal results at

\alpha_{\max}=0.8

and a transition epoch around 25.

Fig. A11 : Hyperparameter sensitivity analysis on the 150-way setting. Top: Top-1 accuracy; Bottom: Top-5 accuracy. Left: performance under different values of

\alpha_{\max}

with the transition epoch fixed to 25. Right: performance under different transition epochs with

\alpha_{\max}=0.8

. Consistent with the 100-way setting, ProCA remains robust to hyperparameter variations and achieves its best performance at an intermediate transition stage across both evaluated accuracy metrics and parameter ranges.

EEG-Based Visual Decoding

EEG-based visual decoding maps brain signals recorded during visual perception to image identities, attributes, or reconstructed scenes.

See all 44 papers

MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding
May 23, 2026Zexuan Chen, Sichao Liu, Runhao Lu +4EEG-Based Visual Decoding Cross-Modal Alignment
KTH, SWeden · University of Cambridge, UK · EPFL, Switzerland +2
ProtoEEG: Structured Prototype Alignment for EEG Visual Decoding
Oct 20, 2026Kieren Yu, Chang Huang, Ziyang Liu +1EEG-Based Visual Decoding Cross-Modal Alignment
What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment
Jun 24, 2026Salini Yadav, Taveena Lotey, Pravendra Singh +1Representation Learning EEG-Based Visual Decoding
Department of CSE IIT Roorkee, India · Department of CSE IIT (ISM) Dhanbad, India

May 23, 2026J/K move · Enter open · S save

MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding

Zexuan Chen, Sichao Liu, Runhao Lu, Huichao Qi+3

KTH, SWeden · University of Cambridge, UK · EPFL, Switzerland+2

EEG-Based Visual Decoding Cross-Modal Alignment Multimodal Contrastive Learning

Open paper AskPDF arXiv

Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience, requiring methods that bridge neural representations and computational models of vision. We introduce a tri-modal contrastive framework for EEG-based visual decoding that aligns EEG, visual, and textual representations within a unified latent space. Our approach follows a two-stage design. First, we pre-train an EEG encoder via masked reconstruction on unlabeled trials, learning spatio-temporal regularities that transfer robustly to downstream tasks. Second, we jointly align EEG, image, and LLM-generated textual descriptions through contrastive learning, where text supervision acts as a semantic regularizer that injects linguistic structure into the shared space without overwhelming the primary EEG-image signal. The encoder integrates subject-specific adaptation, graph-attention over channels, and temporal-spatial convolutional embeddings. On the Things-EEG2 200-way zero-shot benchmark, our framework achieves 54.1% Top-1 and 83.4% Top-5 accuracy, substantially exceeding the strongest prior baseline (32.4% / 64.0%), with paired Wilcoxon tests confirming significance (p < 0.01) over all in-subject baselines. We validate generalization on Things-MEG. Analysis reveals that compact embedding geometries (CN-CLIP) outperform much larger backbones, and that decoding aligns with established neurophysiology of visual processing. This work is a critical step towards robust, semantically-grounded visual decoding from non-invasive temporal neural signals. The source code is publicly available in https://github.com/anon-eeg/eeg_image_decoding.

Abstract

EEG-Based Visual Decoding

Explore similar work

MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding

ProtoEEG: Structured Prototype Alignment for EEG Visual Decoding

What Does the Brain See? Multiview Neural Representations to Demystify the Brain-Visual Alignment