Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levels

Anupam Sharma; Harish Katti; Prajwal Singh; Shanmuganathan Raman; Krishna Miyapuram

cs.LGFeb 24, 2026

Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levels

Authors: Anupam Sharma, Harish Katti, Prajwal Singh, Shanmuganathan Raman, Krishna Miyapuram

Organizations: IIT Gandhinagar Gandhinagar, Gujarat, India · NIMH, NIH USA

Abstract

An electroencephalogram (EEG) records the spatially averaged electrical activity of neurons in the brain, measured from the human scalp. Prior studies have explored EEG-based classification of objects or concepts, often for passive viewing of briefly presented image or video stimuli, with limited classes. Because EEG exhibits a low signal-to-noise ratio, recognizing fine-grained representations across a large number of classes remains challenging; however, abstract-level object representations may exist. In this work, we investigate whether EEG captures object representations across multiple hierarchical levels, and propose episodic analysis, in which a Machine Learning (ML) model is evaluated across various, yet related, classification tasks (episodes). Unlike prior episodic EEG studies that rely on fixed or randomly sampled classes of equal cardinality, we adopt hierarchy-aware episode sampling using WordNet to generate episodes with variable classes of diverse hierarchy. We also present the largest episodic framework in the EEG domain for detecting observed text from EEG signals in the PEERS dataset, comprising $931538$ EEG samples under $1610$ object labels, acquired from $264$ human participants (subjects) performing controlled cognitive tasks, enabling the study of neural dynamics underlying perception, decision-making, and performance monitoring. We examine how the semantic abstraction level affects classification performance across multiple learning techniques and architectures, providing a comprehensive analysis. The models tend to improve performance when the classification categories are drawn from higher levels of the hierarchy, suggesting sensitivity to abstraction. Our work highlights abstraction depth as an underexplored dimension of EEG decoding and motivates future research in this direction.

Figures & tables

Figure 1. Span Analysis for episodic evaluation of CBraMod. It depicts the effects of an episode’s hierarchical depth on performance. Higher span size indicates coarse-grained episodes, and lower span size indicates fine-grained episodes.

Embedder	Baseline			fo-MAML ( Finn et al., 2017 )			proto-fo-MAML ( Triantafillou et al., 2019 )
	2-way	6-way	10-way	2-way	6-way	10-way	2-way	6-way	10-way
EEGNet ( Lawhern et al., 2018 )	0.91±2.6	0.40±1.1	0.57±0.8	0.02±0.2	-0.02±0.1	0.00±0.1	0.14±0.5	0.04±0.4	0.00±0.3
NICE EEG ( Song et al., 2023 )	2.08±4.0	1.18±1.1	0.88±1.1	0.02±0.7	0.10±0.4	0.00±0.4	0.12± 0.8	0.03±0.4	-0.06±0.5
CBraMod ( Wang et al., 2024 ) (w/o pt)	0.00±0.0	0.00±0.0	0.00 ±0.0	0.28±5.3	0.22±2.1	0.22±1.1	1.22±7.1	0.99±2.8	0.36±1.30
CBraMod ( Wang et al., 2024 ) (w pt)	3.53±6.0	2.04±1.4	1.54 ±1.0	0.04±0.4	-0.04±0.2	-0.03±0.1	-0.01±7.6	1.09±2.3	-0.02±1.4

Table 1. Normalized Accuracy for episodic evaluation on unseen subjects

Embedder	Baseline			fo-MAML ( Finn et al., 2017 )			proto-fo-MAML ( Triantafillou et al., 2019 )
	2-way	6-way	10-way	2-way	6-way	10-way	2-way	6-way	10-way
EEGNet ( Lawhern et al., 2018 )	1.27±2.3	0.5±0.9	0.49±0.6	-0.03±0.3	-0.02±0.1	-0.01±0.1	0.09±0.7	0.01±0.4	0.06±0.3
NICE EEG ( Song et al., 2023 )	2.36±3.0	1.02±1.4	0.84±0.9	-0.11±0.7	0.01±0.4	-0.06±0.2	-0.34±1.1	-0.11±0.6	0.09±0.5
CBraMod ( Wang et al., 2024 ) (w/o pt)	0.00±0.0	0.00±0.0	0.00±0.0	-0.22±5.8	0.03±1.6	0.04±1.3	0.37±6.9	-0.18±2.4	0.08±1.2
CBraMod ( Wang et al., 2024 ) (w pt)	5.17±5.5	2.38±1.9	1.57±0.9	0.01±0.1	0.00±0.2	-0.02±0.2	2.66±6.6	-0.70±2.5	0.44±1.6

Table 2. Normalized Accuracy for episodic evaluation on seen subjects

Figure 2. Fine-grained Span Analysis for episodic evaluation of CBraMod on unseen subjects (w PT) trained via baseline learning algorithm. The word cloud represents the concepts represented by the internal nodes in the DAG of the meta-test split. Word size and numeric suffix represent the span size of the node, and brightness represents the normalized classification accuracy.

Embedder	Train	Test
EEGNet ( Lawhern et al., 2018 )	0.03	0.00
NICE EEG ( Song et al., 2023 )	53.24	0.01
CBraMod ( Wang et al., 2024 ) (w/o pt)	0.00	0.00
CBraMod ( Wang et al., 2024 ) (w pt)	1.55	0.02

Table 3. Normalized (balanced) accuracy on joint evaluation with 1126 classes, together with a stratified train-val-test split.

Figure 3. Abstraction Analysis for episodic evaluation of CBraMod. Value of

h

L-h

specifies the number of pruning steps performed on the DAG. As the value of

h

increases, more abstract classes are considered.

Figure 4. Subgraph of the DAG for meta-train split showing leaf nodes sharing the same internal node having high similarity in meaning ( e . g ., FARM, RANCH). In the presence of multiple such instances, EEG might not capture representations for each of them, making non-episodic EEG-to-Text classification with all labels together a difficult task.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparamters	Settings
Embedding dim	1024
Dropout	0.05
Learning rate	0.0003
Epoch	50
Weight Decay	0.001
Optimizer	AdamW

Appendix

Table 4. Hyperparameter settings for experiments with non-episodic evaluation

Learning Method	Hyperparameters	Settings
Common to both	Embedding Dim	512
Common to both	Dropout	0.05
Baseline	Learning Rate	0.0003
	Epochs	50
	Batch Size	128
	Optimizer	AdamW

Appendix

Table 5. Hyperparameter settings for experiments with episodic evaluation

Explore similar work

Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding
May 16, 2026Xiang Gao, Hui Tian, Yanming Zhu +2Disentangled Representation Learning Cross-Modal Representation Learning
School of Information and Communication Technology, Griffith University, Parklands Drive, Southport, 4222, QLD, Australia
Brain-CLIPLM: Semantic Compression for EEG-to-Text Decoding
Mar 23, 2026Xiaoli Yang, Huiyuan Tian, Yurui Li +3EEG Decoding
College of Computer Science and Technology, Zhejiang University, Hangzhou, China · State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding
Aug 4, 2026Yangxuan Zhou, Sha Zhao, Yuning Chen +4Electroencephalography LLM Evaluation
State Key Laboratory of Brain-machine Intelligence, Zhejiang University · College of Computer Science and Technology, Zhejiang University · MOE Frontier Science Center for Brain Science and Brain-machine Integration, Zhejiang University

May 16, 2026J/K move · Enter open · S save

Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding

Xiang Gao, Hui Tian, Yanming Zhu, Xuefei Yin+1

School of Information and Communication Technology, Griffith University, Parklands Drive, Southport, 4222, QLD, Australia

Disentangled Representation Learning Cross-Modal Representation Learning EEG-Based Visual Decoding Hierarchical Representation Learning Vision-Language Retrieval

Open paper AskPDF arXiv

Decoding visual information from electroencephalography (EEG) signals remains a fundamental challenge in brain-computer interfaces and medical rehabilitation. Existing EEG visual decoding methods mainly focus on learning a single global EEG embedding for cross-modal alignment, but they largely overlook the staged and hierarchical characteristics of human visual processing. To address this limitation, we propose a neuroscience-inspired staged representation learning framework that reformulates EEG visual decoding as a stage-specific representation decomposition problem. The proposed framework organizes EEG representation learning into three complementary phases: low-level visual representation learning, high-level semantic representation learning, and integrative information fusion. To strengthen semantic modeling, we further introduce a multimodal dual-level semantic learning mechanism that separates coarse label-level semantics from fine image-level visual-semantic information. In addition, semantic latent channels are introduced as computational representation channels generated from observed visual EEG signals, expanding the channel-level semantic representation space for structured semantic abstraction and cross-modal alignment. Extensive experiments on the THINGS-EEG benchmark demonstrate that the proposed method achieves superior performance under subject-dependent zero-shot evaluation and improved exact retrieval under subject-independent zero-shot evaluation. Additional analyses, including layer-wise retrieval, temporal accumulation, expanded multi-image retrieval, and ablation studies, further support the effectiveness of staged decomposition and structured semantic modeling. These results suggest that explicitly modeling staged perceptual, semantic, and integrative representations provides an effective neuroscience-inspired framework for EEG-based visual decoding.

Abstract

Explore similar work

Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding

Brain-CLIPLM: Semantic Compression for EEG-to-Text Decoding

BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding