Imagined speech decoding from EEG offers a promising route toward naturalistic brain-computer interfaces. Traditional closed-set classifiers treat semantic categories as independent labels and do not explicitly exploit relationships among linguistic intents, limiting output flexibility. We adapt a multimodal contrastive learning framework to imagined speech, aligning EEG trials with language representations in the embedding space of a frozen pretrained text encoder. An EEG encoder is trained with a composite objective combining semantically reweighted EEG-sentence alignment and text-prototype-based class discrimination, encouraging semantic structure while preserving separability among 39 intent categories. We evaluate the approach on the Chisco imagined speech dataset across five subjects, reporting Top-k accuracy, macro-F1, and balanced pairwise matching accuracy (2v2). We also investigate maximum EEG-text cosine similarity (MaxCosine) as an intrinsic indicator of decoding reliability and probe semantic structure beyond category labels using five evaluation-time semantic axes and a qualitative Semantic Gallery Probe (SGP) with previously unseen candidate sentences. The contrastive framework achieved decoding performance comparable to a task-optimized closed-set classifier while mapping EEG into a frozen language embedding space. MaxCosine identified subsets of held-out trials with higher Top-1 and Top-5 accuracy across subjects. Evaluation-time semantic axes showed mean agreement above the 50% chance reference across all five dimensions, and the SGP qualitatively illustrated direct comparison with unseen candidate sentences. Together, these findings suggest that EEG-language alignment can retain comparable fixed-intent decoding while supporting similarity-based retrieval, intrinsic reliability ranking, and evaluation of broader semantic structure.
Figures & tables
Fig. 1: Semantic classes and their distribution for a representative participant (Subject 1), which is almost consistent across all subjects.
Fig. 2: Overview of the proposed Semantic Alignment Framework.
Model
Dim.
Params
BAAI/bge-small-zh-v1.5 [ 30 ]
512
≈ 24M
BAAI/bge-base-zh-v1.5 [ 30 ]
768
≈ 102M
BAAI/bge-large-zh-v1.5 [ 30 ]
1024
≈ 326M
intfloat/multilingual-e5-base [ 31 ]
768
≈ 278M
intfloat/multilingual-e5-large [ 31 ]
1024
≈ 560M
paraphrase-multilingual-mpnet-base-v2 [ 32 , 33 ]
768
≈ 278M
TABLE I: Text Encoder Candidates
Axis
Left pole
Right pole
A1
Practical/Transactional
Social/Emotional
A2
Physical/Embodied
Informational/Cognitive
A3
Goal-directed/Obligation
Leisure/Enjoyment
A4
Mobility/Outside-world
Home/Personal-life
A5
Individual/Self-related
Interpersonal/Social
TABLE II: Semantic axes used in the evaluation-time generalization analysis.
Fig. 3: Empirical comparison of different Chinese language embedding models on the Chisco dataset. Bars represent classification Top-1 accuracy of probes (LR, MLP, XGBoost) trained on the text embeddings, and a zero-shot baseline computed without supervision. Error bars indicate ± std across folds.
Fig. 4: Classification performance of the task-optimized closed-set classifier and the proposed contrastive EEG–language framework across Subjects 01–05. Results are reported as the mean ± std across the five held-out cross-validation folds.
Fig. 5: Validation-to-test transfer of MaxCosine thresholds for Subject 01. Fold-specific thresholds corresponding to the target coverage levels were determined exclusively from the validation-score distribution and subsequently applied unchanged to the corresponding held-out test fold. Points and error bars represent the mean ± std across the five folds.
Fig. 6: Top-1 and Top-5 accuracy among the held-out test trials retained by validation-derived MaxCosine thresholds for Subject 01. The horizontal axis reports the achieved held-out test coverage. Points and error bars represent the mean ± standard deviation across the five folds.
Fig. 7: Top-1 accuracy across five non-overlapping, validation-referenced MaxCosine quintiles for Subject 01. Each held-out test score is expressed relative to its fold-specific validation-score distribution and assigned to one of five quintiles, from the lowest (Q1) to the highest (Q5). Points and error bars represent the mean ± standard deviation across the five folds.
Validation-referenced percentile
Held-out test example
P100 target
“I am interested in the field of marketing.” True category: Job Hunting and Career Development Top-5 predictions: Job Hunting and Career Development; Work; Learning; Personal Information; Education
P90 target
“Deposits, Withdrawals and Checking Accounts.” True category: Finance Top-5 predictions: Finance; Prices and Costs; Personal Information; Inquiries and Personal Matters; Providing and Requesting Assistance
P80 target
“Books borrowed from the library can be renewed.” True category: Internet and Information Technology Top-5 predictions: Education; Learning; Products and Quality Assurance; Food and Dining; Welcoming and Thanking
TABLE III: Illustrative held-out Top-5 decoding examples across validation-referenced MaxCosine percentiles for Subject 01. Percentiles express each test score relative to its fold-specific validation-score distribution. A higher percentile therefore indicates a higher relative MaxCosine score, not a calibrated probability of correctness.
Fig. 8: Fold-specific ROC curves for discrimination of correct and incorrect Top-1 predictions using raw MaxCosine for Subject 01.
Subject
AUROC mean ± std
Coverage at 20%
Top-1 accuracy 100% → 20%
Top-5 accuracy 100% → 20%
S1
0.64 ± 0.027
20.4
8.3 → 17.2
29.2 → 48.6
S2
0.61 ± 0.027
19.3
10 → 19.2
32.7 → 50.5
S3
0.60 ± 0.020
19.7
9 → 15.6
30.1 → 42.9
S4
0.62 ± 0.027
20
9.6 → 18.3
31.1 → 46.2
S5
0.61 ± 0.058
19.8
9.1 → 16.9
30.9 → 45.9
TABLE IV: Across-subject summary of the MaxCosine reliability analysis. AUROC is reported as the mean ± standard deviation across the five folds. “Coverage at 20%” denotes the mean achieved held-out test coverage after applying the fold-specific threshold targeting 20% validation coverage. The Top-1 and Top-5 columns report mean test accuracy at approximately full coverage and at the 20% validation-target operating point, respectively. Coverage and accuracy values are expressed as percentages.
Fig. 9: Balanced accuracy measures agreement between the semantic-axis assignment derived from the held-out EEG embedding and that derived from the corresponding sentence embedding. Bars show the mean across the five subjects and error bars indicate the standard deviation across subjects. The dashed line denotes the 50% chance level.
Fig. 10: SGP example for Subject 1. A frozen gallery of held-out sentence embeddings is projected to 2D via PCA for visualization. Each point corresponds to a gallery sentence and is colored by its cosine similarity to the EEG trial.