Inner-Speech Decoding from EEG Across Three Corpora Under a Fixed Architecture and Protocol

Md Wahiduzzaman Suva; Md Imtiaj Alam Sajin; Esm E Moula Chowdhury Abha; Susham Moula Choudhury Akash; Md Tauhidul Islam; Abhijit Bhowmik; Md Manirul Islam; Debajyoti Karmaker

cs.AINov 1, 2026

Inner-Speech Decoding from EEG Across Three Corpora Under a Fixed Architecture and Protocol

Authors: Md Wahiduzzaman Suva, Md Imtiaj Alam Sajin, Esm E Moula Chowdhury Abha, Susham Moula Choudhury Akash, Md Tauhidul Islam, Abhijit Bhowmik, Md Manirul Islam, Debajyoti Karmaker

Organizations: Computer Science Department, American International University-Bangladesh, Dhaka, Bangladesh · Department of Software Engineering, N.I. Lobachevsky National Research Nizhny Novgorod State University, Nizhny Novgorod, Russia · School of Computing, RMIT University, Melbourne, Australia

Abstract

Inner-speech decoding from electroencephalography could support non-invasive communication for people with severe motor impairment, but comparison across studies is hindered by dataset- and protocol-specific evaluation. We present a controlled cross-dataset study using one preprocessing pipeline, one within-subject five-fold cross-validation protocol, and a compact dual-attention Transformer (VC-DATNet; approximately 131,000 parameters) across three public corpora: Thinking Out Loud (10 participants), BISpeech (4), and SyncSpeech (3). VC-DATNet achieved per-participant sensor/source hybrid accuracies of 39.48% on the four-class Thinking Out Loud task, 20.60% on the eight-word BISpeech task, and 25.53% on the eight-word SyncSpeech task, for which we report the first decoding result. Sensor-space and cortical source-space representations were complementary, with each performing better for different participants; this pattern replicated across all three corpora and nine architectures. Moreover, ten decoders spanning convolutional, recurrent, temporal-convolutional, hybrid Transformer, and pretrained foundation-model families converged within a narrow accuracy range on every dataset. These findings indicate that representation choice should be individualized and suggest that performance on these public corpora is constrained more by signal quality and dataset scale than by model capacity. The released pipeline provides a reproducible benchmark for future inner-speech decoding studies

Figures & tables

Study	Method(s)	Original evaluation	Highlights / relevance
CNN-based decoders
Schirrmeister et al. [ 50 ]	ShallowConvNet / DeepConvNet	BCI IV-2a, HGD	End-to-end CNNs match filter-bank pipelines
Lawhern et al. [ 27 ]	EEGNet (depthwise-separable conv)	BCI IV-2a, others	Compact, cross-paradigm default baseline
Ingolfsson et al. [ 22 ]	EEG-TCNet (temporal conv blocks)	BCI IV-2a	Low-latency convolutional decoder
Riyad et al. [ 48 ]	Incep-EEGNet (inception filters)	BCI IV-2a	Multi-scale temporal receptive fields
Mane et al. [ 38 ]	FBCNet (filter-bank + variance layer)	BCI IV-2a/2b	Explicit spectral-variance layer

Table 1: Comparative summary of representative EEG decoding architectures (2017–2025) surveyed in Related Work , included as a literature reference for the EEG decoder families discussed below. Accuracies are not reported in this table because nearly every architecture listed was developed and evaluated on motor-imagery benchmarks; retraining them under our pipeline on the three inner-speech corpora used in this paper is a direct extension ( Limitations and Scope ). Inner-speech / imagined-speech studies are summarised separately in Table 2 . Architectures are grouped by family. MI = motor imagery; HGD = High-Gamma Dataset.

Study	Dataset	Method	Task (classes)	Acc. (%)	Note
Thinking Out Loud (ds003626): the primary benchmark of this paper
Nieto et al. 2022 [ 43 ]	ToL	Dataset release; RF / CNN baselines	4-cls directional	$\approx$ 25 (chance)	De facto open ToL benchmark
van den Berg et al. 2021 [ 57 ]	ToL	EEGNet (2D CNN)	4-cls directional	29.7	First deep-learning benchmark on ToL
Gasparini et al. 2022 [ 17 ]	ToL	SVM / XGBoost / LSTM / BiLSTM	4-cls directional	26.2–36.1	BiLSTM (36.1) is the strongest prior ToL result
Bimodal Inner Speech corpus / BISpeech (ds004196)
Wilson et al. 2022 [ 62 ]	BISpeech	SVM / RF (EEG, fMRI)	8-word; binary	22.2 (8-w, EEG); 62.8 (bin., EEG)	First classification study on BISpeech

Table 2: Summary of EEG-based inner-speech / imagined-speech decoding studies. “S-D” = subject-dependent; “LOSO” = leave-one-subject-out (subject-independent); “–” = not reported. Honest, reproducible accuracies on these tasks cluster a few points above chance (

\approx

30–36 % for 4-class,

\approx

15–30 % for 6–8-class). † Result is anomalously high relative to the field and, on the available evidence, attributable to within-subject over-fitting or train/test leakage (e.g. CSP/PLV or normalisation statistics fitted across the cross-validation split); not used as a comparison target. ‡ Competition test-set or single-subject (

n{=}1

) result, not comparable to cross-validated multi-subject accuracies.

Dataset	Year	Subj.	Ch.	Classes	Trials	Stimuli
Thinking Out Loud (ToL; ds003626) [ 43 ]	2022	10	128	4	$\sim$ 50/class ( $\sim$ 200/subject)	Directional words: up, down, left, and right
BISpeech (ds004196) [ 35 , 51 ]	2023	4	64	8	40/word	Semantic words: four social and four numerical
SyncSpeech (ds006033) [ 52 ]	2025	3	64	8	40/word	Semantic words: four social and four numerical

Table 3: The three public inner-speech EEG datasets used in this study, all obtained from OpenNeuro. Only the EEG modality is used; the simultaneously recorded fMRI data in BISpeech and SyncSpeech are not analysed. All datasets are evaluated using within-subject five-fold cross-validation. Channels denotes the number of recorded scalp electrodes; the decoded sensor- and source-space configurations are described in Data Preprocessing .

Figure 1: End-to-end pipeline and architecture of VC-DATNet. Panel (a) covers preprocessing ( Data Preprocessing ): raw-EEG loading and event annotation, channel configuration and average re-referencing,

4

–

40

Hz band-pass filtering, ICA artefact removal, channel selection, and epoching to a

2.5

s window at

256

Hz. The output is a preprocessed epoch

\mathbf{X}\in\mathbb{R}^{B\times C\times T}

, where

C

is the dataset-determined sensor-channel count or, for the source-space configuration ( Source-Space Projection ),

20

language-network regions of interest. Panel (b) shows the shared front-end. A multi-scale temporal Conv2D with four parallel kernels of

\{32,64,96,128\}

samples (Eqs. 2 – 3 ) is followed by batch normalisation and a depthwise spatial Conv2D (

C\times 1

, BatchNorm, ELU; Eq. 4 ), producing the feature map

\mathbf{X}_{\mathrm{spat}}\in\mathbb{R}^{B\times F\times T}

. Panel (c) shows the dual-stream encoder. The feature map

\mathbf{X}_{\mathrm{spat}}

is pooled in parallel into a mean stream (

\mathcal{P}_{\mu}

, Eq. 5 ) and a clamped log-variance stream (

\mathcal{P}_{\sigma^{2}}

, Eq. 6 ); the latter is a per-window estimator of band-limited log-power (Lemma 3 ). Each stream is processed independently by a

4

-layer dual-attention Transformer encoder (Eqs. 10 – 13 ). Panel (d) is a zoomed view of one dual-attention block. The spatial and channel gates are derived from the value tensor at full sequence resolution and from a sequence-mean summary respectively. They are added to form

\mathbf{V}_{\mathrm{dual}}

before the scaled dot-product attention

\mathrm{softmax}(\mathbf{Q}\mathbf{K}^{\top}/\sqrt{d_{k}})

, which leaves the query–key similarity structure intact (Proposition 4 ); the result is projected by

\mathbf{W}_{O}

. Panel (e) is the classifier head and outputs. The two transformed streams are concatenated (Eq. 19 ), integrated by a small convolutional encoder (Conv2D, BatchNorm, ELU), flattened, and projected to

K

class logits (Eq. 21 ). Softmax and arg-max produce the class-probability vector

\mathrm{softmax}(\mathbf{y})\in\Delta^{K-1}

and the predicted label

\arg\max_{k}y_{k}

. Training is end-to-end with cross-entropy (Eq. 22 ). The network corresponds to the operator composition of Definition 1 .

Symbol	Shape	Meaning
$\mathbf{X}$	$\mathbb{R}^{B\times C\times T}$	Preprocessed EEG epoch
$\mathbf{X}_{\text{temp}}$	$\mathbb{R}^{B\times F_{t}\times C\times T}$	Multi-scale temporal features
$\mathbf{X}_{\text{spat}}$	$\mathbb{R}^{B\times F\times 1\times T}$	Depthwise-spatial features
$\mathbf{P}_{\mu},\mathbf{P}_{\sigma^{2}}$	$\mathbb{R}^{B\times F\times L}$	Mean / log-variance pooled streams
$\mathbf{Q},\mathbf{K},\mathbf{V}$	$\mathbb{R}^{B\times N\times D}$	Query / key / value tensors ( $N{=}L$ )
$\mathbf{g}_{t}\in[0,1]^{B\times N\times D}$	per-token	Temporal gate (per-timestep)

Table 4: Principal symbols and shapes used throughout the architecture description.

Preprocessing
Component / hyperparameter	Value
Sampling rate	$256$ Hz
Epoch length	$2.5$ s ( $\approx$ 640 samples)
Band-pass filter	$4$ – $40$ Hz (4th-order Butterworth)
Normalisation	per-trial $z$ -score
Architecture

Table 5: Complete VC-DATNet configuration, identical across all three datasets and both signal representations. The values are the defaults of the released training script. The input dimension (sensor-channel or source-ROI count) and the output-class count are fixed by the dataset and the task; every other quantity is held constant.

Dataset / task	Classes	Chance (%)	VC-DATNet (%)	Strongest prior (%)	Reference
ToL, directional	4	25.0	39.48 (hybrid)	36.1 (BiLSTM)	Gasparini et al. [ 17 ]
BISpeech, 8-word	8	12.5	20.60 (sensor)	14.78 (RF, EEG-only)	Wilson et al. [ 63 ]
BISpeech, social vs. numerical	2	50.0	59.37 (source)	62.81 (SVM)	Wilson et al. [ 62 ]
BISpeech, social 4-word	4	25.0	38.68 (hybrid)	— (none EEG-only)	—
BISpeech, numerical 4-word	4	25.0	36.78 (sensor)	— (none EEG-only)	—
SyncSpeech, 8-word	8	12.5	25.53 (hybrid)	— (none published)	—

Table 6: Headline inner-speech decoding accuracy of VC-DATNet on the three datasets, against the strongest envelope-consistent peer-reviewed baseline on each. All VC-DATNet results are within-subject five-fold cross-validation, EEG-only. “Sensor” and “source” denote the two signal representations ( Data Preprocessing ); “hybrid” selects, per subject, the better of the two. Chance is

100/K

for a

K

-class task. SyncSpeech has no prior published decoding result.

(a) Bench protocol: full $128$ -channel BioSemi montage, vanilla single-condition training.
Representation	S01	S02	S03	S04	S05	S06	S07	S08	S09	S10	Mean
Sensor-space	30.0	38.8	45.0	35.0	32.5	37.0	37.1	40.0	40.4	40.0	37.58
Source-space	40.0	31.2	32.8	38.8	36.7	38.0	36.2	32.0	30.0	35.4	35.11
Hybrid (best)	40.0	38.8	45.0	38.8	36.7	38.0	37.1	40.0	40.4	40.0	39.48
Winner	src	sens	sens	src	src	src	sens	sens	sens	sens	—
(b) A priori subset: $22$ inner-speech-relevant channels, cross-condition (Pron+Inner+Vis) multi-task pretrain $\to$ inner-only fine-tune.

Table 7: Per-subject 4-class accuracy (%) on Thinking Out Loud (10 subjects, within-subject five-fold CV) under two configurations of VC-DATNet. Block (a) uses the cross-architecture bench protocol (full

128

-channel BioSemi montage, vanilla single-condition training, no cross-condition pretrain), matching every other architecture in Table 11 . Block (b) uses the a priori

22

-channel inner-speech-relevant subset with cross-condition (Pron+Inner+Vis) multi-task pretrain followed by inner-only fine-tune. In both blocks the source-space row uses the same

20

-ROI Desikan–Killiany projection. “Hybrid” is the per-subject better of sensor and source within each block. The two-regime effect is present under both configurations (sens/src wins re-balance slightly across configs but both regimes co-exist), and the headline result is robust to either choice of channel subset and pretrain.

Task	Classes	Chance (%)	Sensor (%)	Source (%)	Hybrid (%)
8-word	8	12.5	20.60	17.46	20.60
Social vs. numerical	2	50.0	57.64	59.37	59.37
Social, 4-word	4	25.0	37.44	35.72	38.68
Numerical, 4-word	4	25.0	36.78	35.67	36.78

Table 8: BISpeech (ds004196) decoding accuracy (%): 4 subjects, within-subject five-fold CV, EEG-only. “Hybrid” is the per-subject better representation. The two-regime effect recurs: source-space wins the binary task, sensor-space the finer-grained multi-class tasks.

Method	S01	S02	S03	S05	Mean
EEGNet [ 27 ]	17.85	18.18	18.82	19.69	18.64
ShallowConvNet [ 50 ]	19.73	20.38	17.57	19.38	19.27
DeepConvNet [ 50 ]	17.87	17.88	17.87	17.81	17.86
EEG-TCNet [ 22 ]	18.80	16.29	18.50	17.81	17.85
EEGPT [ 58 ] (linear probe)	21.00	18.20	17.90	19.10	19.05
TransNet [ 37 ]	18.81	20.37	18.80	20.31	19.57

Table 9: Per-subject hybrid accuracy (%) on BISpeech 8-word task across all benched architectures (4 subjects, within-subject five-fold CV, EEG-only). Hybrid is per-subject best of sensor- and source-space. Bold = top accuracy per subject. Underline = VC-DATNet. “—” denotes a configuration not run.

Representation	Sub-01	Sub-02	Sub-03	Mean
Sensor-space	21.45	27.42	24.65	24.51
Source-space	24.52	26.33	18.22	23.02
Hybrid (best)	24.52	27.42	24.65	25.53

Table 10: SyncSpeech (ds006033) decoding accuracy (%): 8-word inner speech, 3 subjects, within-subject five-fold CV, EEG-only. To our knowledge these are the first published decoding results on this corpus.

				ToL (4c)			BISpeech (8c)			SyncSpeech (8c)
Method	Family	Domain	Params	S	Sr	H	S	Sr	H	S	Sr	H
EEGNet (Lawhern et al.) [ 27 ]	CNN	Multi	4.4 K	29.7 †	—	29.7 †	18.40	17.93	18.64	21.90	19.74	22.34
ShallowConvNet (Schirrmeister et al.) [ 50 ]	CNN	MI	211.8 K	34.70	32.96	35.21	19.04	18.40	19.27	22.53	23.23	26.41
DeepConvNet (Schirrmeister et al.) [ 50 ]	CNN	MI	345.9 K	35.57	31.62	35.57	17.47	16.84	17.86	21.47	23.69	23.70
EEGNeX (Chen et al.) [ 8 ]	CNN	Multi	62.9 K	35.08	35.44	36.68	19.10	18.95	19.66	26.75	24.20	26.75
EEG-TCNet (Ingolfsson et al.) [ 22 ]	TCN	MI	5.9 K	31.28	30.34	31.99	17.70	15.19	17.85	15.23	14.61	16.81

Table 11: Ten EEG decoders evaluated on the same three inner-speech datasets in both signal representations under the same fixed protocol (within-subject five-fold CV, bandpass 4–40 Hz, per-trial

z

-score, AdamW + cosine schedule, mixup

\alpha=0.3

, label smoothing

0.1

, 120 epochs, EEG-only). The “Family” column gives the architectural family (CNN, TCN = temporal convolutional network, RNN, Foundation = self-supervised pretraining, CNN-T = CNN–Transformer hybrid). The “Domain” column gives the task each architecture was originally developed and validated for: MI = motor imagery; IS = inner speech; Multi = multiple EEG tasks. Eight of ten trained-from-scratch baselines were originally validated on motor imagery; only Gasparini’s BiLSTM was developed on inner speech. “Params” is the trainable-parameter count for the canonical ToL configuration (

C\!=\!128

T\!=\!641

K\!=\!4

); counts differ slightly with

C

and

K

on the other datasets. EEGPT’s parameter count is the frozen pretrained encoder (Wang et al. [ 58 ] ). For each (model, dataset) we report grand-mean accuracy in sensor space (“S”), source space (“Sr”), and per-subject hybrid (“H”, better of the two). All values % accuracy. Despite spanning over three orders of magnitude in parameter count, five architectural families, and three task domains, the methods converge to the same narrow accuracy band on each dataset. “—” denotes a configuration not run; EEGPT (linear-probe) and the reproduced Gasparini BiLSTM are sensor-only by design.

	ToL ( $n\!=\!10$ )			BISpeech ( $n\!=\!4$ )			SyncSpeech ( $n\!=\!3$ )
Method	Sens $\uparrow$	Src $\uparrow$	Tie	Sens $\uparrow$	Src $\uparrow$	Tie	Sens $\uparrow$	Src $\uparrow$	Tie
ShallowConvNet	7	2	1	1	1	2	1	2	0
DeepConvNet	10	0	0	3	1	0	0	2	1
EEG-TCNet	7	2	1	3	1	0	2	1	0
EEGNet	—	—	—	3	1	0	2	1	0
TransNet	6	4	0	1	2	1	2	0	1

Table 12: Cross-architecture replication of the sensor/source two-regime effect. For each (model, dataset) cell of Table 11 , we count the number of subjects on which sensor-space decoding beat source-space (“Sens

\uparrow

”), source-space beat sensor-space (“Src

\uparrow

”), or the two tied to within 0.5 pp (“Tie”). The total per cell is the number of subjects in that dataset (ToL: 10, BISpeech: 4, SyncSpeech: 3). Both regimes appear in every (model, dataset) cell except DeepConvNet on ToL, confirming the split is a data property and not specific to VC-DATNet.

Panel of Figure 2 — Figure 2: Class-conditional alpha-band ( $8$ – $13$ Hz) log-power signature on the two corpora with sufficient cohort to support a dedicated sensor + source panel: ToL (a) and BISpeech (b). In each subfigure, the top row shows per-class sensor-space scalp topomaps of the cross-subject grand-mean $z$ -scored alpha power, centred on the across-class mean; the bottom row shows the matching source-space signature as a diverging horizontal-bar chart per class over the $20$ Desikan–Killiany language-network ROIs, where each bar extends left of the zero-line for negative deviations and right for positive deviations, on the same “class $-$ mean” colour scale. The class-differentiated fronto-temporal-parietal alpha pattern visible in (a) recurs in (b) despite the smaller cohort and the different (semantic) stimulus structure, with the social and numerical sub-vocabularies recruiting overlapping but distinguishable regions. The third corpus (SyncSpeech) is rendered together with the other two on the cortex in Figure 3 .

Panel of Figure 3 — Figure 3: Cortical projection of the class-conditional alpha-band signature onto the fsaverage pial surface for all three inner-speech corpora (left and right hemispheres, lateral views per class). The $20$ Desikan–Killiany language-network ROIs are coloured by their per-class deviation from the across-class mean, on the same scale used by the matching source-space diverging-bar charts for ToL and BISpeech in Figure 2 (panels 2(a) and 2(b) , bottom row of each); SyncSpeech is shown only on the cortex here. Reading down the figure makes the cross-dataset replication direct: in every corpus the class-conditional structure is carried by bilateral inferior frontal, superior temporal, and parietal regions of the cortical language network, despite differences in cohort size ( $10$ / $4$ / $3$ ), task ( $4$ -class directional vs. $8$ -word semantic), and acquisition hardware.

Figure 4: Peak-frequency distribution of VC-DATNet’s learned temporal-conv filters. For each of the four kernel-length branches (

k\in\{32,64,96,128\}

samples at

256

Hz;

\approx 125

250

375

500

ms), the

8

filters from the best-fold checkpoint of every trained target (ToL sub-01 sensor, plus sub-01/04/06/10 source;

40

filters per branch,

160

total) are zero-padded, FFT’d, and assigned their peak frequency over

1

–

60

Hz. Coloured bands shade the canonical EEG ranges (theta

4

–

8

, alpha

8

–

13

, beta

13

–

30

, low gamma

30

–

40

Hz); triangles mark the per-branch medians. The mass of the distribution lies in beta. The

{>}40

Hz tail is outside the

4

–

40

Hz training bandpass (dashed lines) and is not interpretable as a learned high-frequency feature.

EEG Decoding

Electroencephalography (EEG) decoding maps brain-signal patterns to mental states, stimuli, speech, or intended actions.

See all 218 papers

Decoding silent reading from non-invasive EEG
Aug 20, 2026Ingo Marquardt, Anthilia Alchanat, Priyanka JainEEG Decoding EEG-to-Text Decoding
nubrain
Cross-subject decoding of imagined speech from scalp EEG: a leakage-controlled evaluation on three public corpora
Oct 14, 2026Blessing Isoyiza Adeika, Onyema OsuagwuCross-Subject EEG Decoding Brain-Computer Interfaces
Morgan State University, Baltimore, Maryland, United States of America
Do EEG Foundation Models Transfer to Speech? A Benchmark on Overt and Imagined Speech Decoding
Jul 29, 2026Owais Mujtaba Khanday, Mohamed Baha Ben Ticha, Sanae Belfrouh +2EEG Decoding EEG Foundation Models
Dept. Signal Theory, Telematics and Communications, University of Granada, Spain · Research Centre for Information and Communication Technologies (CITIC-UGR), Spain · Laboratory of Information Technologies, University of Chouaib Doukkali, Morocco +1

Aug 20, 2026J/K move · Enter open · S save

Decoding silent reading from non-invasive EEG

Ingo Marquardt, Anthilia Alchanat, Priyanka Jain

nubrain

EEG Decoding EEG-to-Text Decoding

Open paper AskPDF arXiv

Non-invasive decoding of inner speech faces a fundamental data problem: a corpus pairing brain activity with a person's spontaneous inner monologue cannot be collected, and the available proxy paradigms (cued repetitive and retrospectively reported generative inner speech) are slow to acquire, poorly time-locked, and subject compliance is unverifiable. We therefore treat silent reading as a scalable proxy task and ask how much lexical and semantic information a contrastive decoder can extract from it. We report an open-vocabulary analysis of approximately 240,000 word presentations recorded from a single densely-sampled participant across 393 runs (ca. 49 h) of 19-channel dry-electrode EEG. Words from continuous narrative text were presented in rapid serial visual presentation, with typography randomised on every trial to partially decorrelate word identity from low-level visual form. A convolutional EEG encoder, optionally followed by a causal transformer, was trained with a CLIP-style contrastive objective to align short EEG windows with hidden-state embeddings of the presented word taken from a large language model. Decoding, evaluated as word-grouped top-10 retrieval against permutation baselines, was reliably above chance, extended to mid-frequency and rare words, and scaled log-linearly with training-data volume with no sign of saturation. Removing occipital and posterior-temporal electrodes reduced the word-level gain by roughly one third but left context tracking unchanged. Control analyses separate word-level decoding from narrative context tracking and from a non-neural positional prior introduced by the transformer's positional embedding. These results establish that open-vocabulary word-level information is recoverable from EEG during silent reading, and that decoding is data-limited rather than saturated.

Abstract

EEG Decoding

Explore similar work

Decoding silent reading from non-invasive EEG

Cross-subject decoding of imagined speech from scalp EEG: a leakage-controlled evaluation on three public corpora

Do EEG Foundation Models Transfer to Speech? A Benchmark on Overt and Imagined Speech Decoding