CIPHER: Conformer-based Inference of Phonemes from High-density EEG

Varshith Madishetty

cs.CLMar 22, 2026

CIPHER: Conformer-based Inference of Phonemes from High-density EEG

Authors: Varshith Madishetty

Abstract

Decoding speech information from scalp EEG remains difficult due to low SNR and spatial blurring. We present CIPHER (Conformer-based Inference of Phonemes from High-density EEG Representations), a dual-pathway model using (i) ERP features and (ii) broadband DDA coefficients. On OpenNeuro ds006104 (24 participants, two studies with concurrent TMS), binary articulatory tasks reach near-ceiling performance but are highly confound-vulnerable (acoustic onset separability and TMS-target blocking). On the primary 11-class CVC phoneme task under full Study 2 LOSO (16 held-out subjects), performance is substantially lower (real-word WER: ERP 0.671 +/- 0.080, DDA 0.688 +/- 0.096, indicating limited fine-grained discriminability. We therefore position this work as a benchmark and feature-comparison study rather than an EEG-to-text system, and we constrain neural-representation claims to confound-controlled evidence.

Figures & tables

	Study 1 (2019)	Study 2 (2021)
Participants	8 (sub-P01–P08)	16 (sub-S01–S16)
Session	ses-01	ses-02
Tasks	CV and VC phoneme pairs	Single phonemes, CV/VC, CVC
Phonemes	6 consonants + 5 vowels	Same
EEG System	64-ch BioSemi ActiveTwo	Same
Sampling Rate	2048 Hz (raw)	2048 Hz (raw)

Table 1: Dataset overview.

Panel of Figure 1 — Figure 1: CIPHER dual-pathway architecture. Raw EEG is processed in parallel through an ERP pathway (narrowband, phase-locked dynamics) and a DDA pathway (broadband nonlinear dynamical coefficients), fused by a shared Conformer encoder, and decoded by four task-specific classification heads.

	Study 2 Validation	Study 1 Test (Cross-Study)
Best Top-1 Accuracy	1.000	1.000
Best F1 (Macro)	1.000	1.000
Best Condition	ERP / Manner / LipTMS	ERP / Manner / LipTMS
Evidence Tier	Secondary (confound-vulnerable)	Secondary (confound-vulnerable)

Table 2: Best cross-dataset classification performance.

Binary Task	ERP EEG	DDA EEG	Acoustic-only	Best EEG $-$ Acoustic
Category	0.552	0.656	1.000	$-0.344$
Manner	0.633	0.708	1.000	$-0.292$
Place	0.518	0.522	1.000	$-0.478$
Voicing	0.504	0.482	1.000	$-0.496$

Table 3: NULL-only LOSO binary-task accuracy: EEG vs. acoustic-only baseline.

Feature	Word Type	Eval	WER (mean $\pm$ std)	Folds	Samples
ERP	Real Words	Study 2 LOSO	$\mathbf{0.671\pm 0.080}$	16	616
ERP	Pseudowords	Study 2 LOSO	$0.780\pm 0.029$	16	616
DDA	Real Words	Study 2 LOSO	$0.688\pm 0.096$	16	616
DDA	Pseudowords	Study 2 LOSO	$\mathbf{0.772\pm 0.050}$	16	616

Table 4: Phoneme-level WER on CVC triphones under Study 2 LOSO (16 folds).

Figure 2: WER comparison across ERP/DDA and real/pseudoword conditions.

Model	ERP Ph.	ERP Man.	ERP Pl.	DDA Ph.	DDA Man.	DDA Pl.
Chance	0.091	0.500	0.500	0.091	0.500	0.500
LR (mean+std)	0.089	0.590	0.507	0.139	0.711	0.513
LDA (mean+std)	0.132	0.769	0.519	0.153	0.800	0.518
ShallowConvNet [ 9 ]	0.170	0.857	0.573	0.160	0.851	0.567
EEGNet [ 11 ]	0.174	0.857	0.571	0.176	0.855	0.573
EEG-Conformer [ 10 ]	0.167	0.857	0.571	0.171	0.857	0.571

Table 5: Matched-split baseline comparison on study2_val (NULL condition).

Figure 3: Matched-split baseline accuracy heatmap.

Feature	Mean Acc (Real)	Mean Acc (Pseudo)	$t$ -stat	$p$ -value	Significant?
ERP	0.190	0.233	$-1.783$	0.095	No
DDA	0.079	0.075	$\phantom{-}0.187$	0.855	No

Table 6: Lexicality effect: per-subject paired

t

-test results.

Figure 4: Real vs. pseudoword lexicality effect (non-significant).

Feature	Place	$F$ -statistic	$p$ -value	Significant?
ERP	Bilabial	0.791	0.530	No
ERP	Alveolar	3.458	0.114	No
DDA	Bilabial	7.496	0.068	No (marginal)
DDA	Alveolar	0.516	0.626	No

Table 7: TMS condition ANOVA results by feature type and articulatory place.

Figure 5: TMS ANOVA $p$ -values by feature and place stratum.

Control	Task	Acc (mean, LOSO)	95% CI
EEG-only (ERP, pooled LR)	Phoneme identity	0.104	[0.088, 0.122]
EEG-only (DDA, pooled LR)	Phoneme identity	0.129	[0.114, 0.144]
EEG-only (ERP)	Manner	0.633	[0.567, 0.695]
EEG-only (DDA)	Manner	0.708	[0.636, 0.766]
Acoustic-only (metadata LR)	Phoneme identity	1.000	[1.000, 1.000]
Acoustic-only (metadata LR)	Manner	1.000	[1.000, 1.000]

Table 8: NULL-only LOSO controls vs. acoustic-only baseline.

Figure 6: NULL-only EEG controls versus acoustic-only controls.

Variant	Mean Val Acc	Std (3 seeds)	$\Delta$ vs Full
Full CIPHER	0.1675	0.0040	$\phantom{+}0.0000$
No SE	0.1674	0.0042	$-0.0001$
No stochastic depth	0.1661	0.0061	$-0.0013$
No attention pooling (mean pooling)	0.1680	0.0056	$+0.0006$
No multi-scale front-end	0.1698	0.0000	$+0.0023$

Table 9: Architecture ablation on 8-subject expanded split (DDA, phoneme identity).

Task	Base Acc	Masked 0–200 ms	$\Delta$
Phoneme identity	0.104	0.104	$-0.0003$
Manner	0.633	0.608	$-0.0251$
Place	0.518	0.513	$-0.0051$

Table 10: Early-window masking effect on NULL-only LOSO accuracy.

Feature	Task	True Acc	Perm Acc (mean)	$\Delta$	Empirical $p$
ERP	Phoneme identity	0.117	0.128	$-0.011$	0.706
ERP	Manner	0.649	0.665	$-0.016$	0.706
ERP	Place	0.517	0.520	$-0.002$	0.706
DDA	Phoneme identity	0.180	0.176	$+0.004$	0.529
DDA	Manner	0.822	0.806	$+0.016$	0.529
DDA	Place	0.525	0.531	$-0.005$	0.882

Table 11: Block-aware permutation test results (50 permutations/fold).

Figure 7: Time-window masking and block-aware permutation controls.

Feature	Task	Best Variant (acc)	Full CIPHER	Largest $-\Delta$ vs Full
ERP	Phoneme identity	No attn pool (0.150)	0.132	No multi-scale ( $-0.008$ )
ERP	Manner	No attn pool (0.822)	0.774	No SE ( $-0.069$ )
ERP	Place	No multi-scale (0.563)	0.551	No stoch. depth ( $-0.034$ )
DDA	Phoneme identity	No multi-scale (0.137)	0.125	No SE ( $-0.003$ )
DDA	Manner	No multi-scale (0.852)	0.704	None observed
DDA	Place	No attn pool (0.572)	0.571	No SE ( $-0.042$ )

Table 12: Multi-seed ablation matrix (8-8 split, 3 seeds).

Figure 8: Multi-seed ablation matrix (mean validation accuracy).

Figure 9: Ablation deltas relative to full CIPHER across feature-task settings.

Hyperparameter	Value
Model dimension $d_{\text{model}}$	256
Conformer blocks	4
Attention heads	8
Conv front-end channels	64 per branch (192 total)
Conv front-end kernels	$\{3,7,15\}$
Conformer conv kernel	15

Table 13: Full hyperparameter specification.

EEG-to-Text Decoding

See all 13 papers

Leakage-Audited Benchmarking Reveals Limited Evidence for Cross-Subject Auditory-Evoked EEG Vowel Perception Decoding
Apr 22, 2026Xiaoyang Li, Zeyan TaoCross-Subject EEG Decoding Electroencephalography
College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, 110016, China
Do EEG Foundation Models Transfer to Speech? A Benchmark on Overt and Imagined Speech Decoding
Jul 29, 2026Owais Mujtaba Khanday, Mohamed Baha Ben Ticha, Sanae Belfrouh +2EEG Decoding EEG Foundation Models
Dept. Signal Theory, Telematics and Communications, University of Granada, Spain · Research Centre for Information and Communication Technologies (CITIC-UGR), Spain · Laboratory of Information Technologies, University of Chouaib Doukkali, Morocco +1
Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark
Jul 21, 2026Zihan Zhang, Yu Bao, Xiao Ding +2Electroencephalography EEG Decoding
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology · Shanghai Innovation Institute · Shanghai Innovation Institute, Shanghai, China +3

Apr 22, 2026J/K move · Enter open · S save

Leakage-Audited Benchmarking Reveals Limited Evidence for Cross-Subject Auditory-Evoked EEG Vowel Perception Decoding

Xiaoyang Li, Zeyan Tao

College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, 110016, China

Cross-Subject EEG Decoding Electroencephalography Neural Decoding EEG Decoding

Open paper AskPDF arXiv

We tested whether auditory-evoked EEG supports subject-independent five-vowel perception decoding when trial identity, model identity, prediction provenance, and participant-level inference are controlled within a single benchmark. We reconstructed Study 2 event tables from OpenNeuro ds006104 version 1.0.1 and analysed the consonant-vowel pair task. One-to-one marker-stimulus pairing yielded 3,840 independent trials; control-condition selection and artifact rejection retained 1,094 epochs from 16 participants and 61 EEG channels. Thirteen unique implementations were evaluated using leave-one-subject-out testing, with participant metrics reconstructed from 36,102 trial predictions across 33 complete prediction replicas. Random Forest was numerically highest at 21.474% balanced accuracy (95% participant-bootstrap interval, 19.526-23.482%; chance, 20%), but neither its participant-level tests nor any implementation survived correction across the 13-model family. Deep-model performance was close to chance, and several architectures showed substantial seed-dependent variation and low trial-label agreement. An exploratory MDM analysis comprising 9,616 genuine refits across training cohorts of 3-15 participants showed no monotonic performance gain. Within this dataset and protocol, evidence for reliable cross-subject five-vowel decoding is limited. The benchmark provides a reproducible chain from source rows to retained epochs, predictions, participant-level metrics, multiplicity-adjusted inference, and bounded diagnostic analyses.

Abstract

EEG-to-Text Decoding

Explore similar work

Leakage-Audited Benchmarking Reveals Limited Evidence for Cross-Subject Auditory-Evoked EEG Vowel Perception Decoding

Do EEG Foundation Models Transfer to Speech? A Benchmark on Overt and Imagined Speech Decoding

Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark