cs.LGMar 17, 2026
SaveSENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrieval
Organizations: The University of Texas at Austin School of Information Austin, Texas, USA · Institute of Applied Computer Science Łódź University of Technology Poland
Abstract
Decoding brain activity into natural language is a major challenge in AI with important applications in assistive communication, neurotechnology, and human-computer interaction. Most existing Brain-Computer Interface (BCI) approaches rely on memory-intensive fine-tuning of Large Language Models (LLMs) or encoder-decoder models on raw EEG signals, resulting in expensive training pipelines, limited accessibility, and potential exposure of sensitive neural data. We introduce SENSE (SEmantic Neural Sparse Extraction), a lightweight and privacy-preserving framework that translates non-invasive electroencephalography (EEG) into text without LLM fine-tuning. SENSE decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation. EEG signals are locally mapped to a discrete textual space to extract a non-sensitive Bag-of-Words (BoW), which conditions an off-the-shelf LLM to synthesize fluent text in a zero-shot manner. The EEG-to-keyword module contains only ~6M parameters and runs fully on-device, ensuring raw neural signals remain local while only abstract semantic cues interact with language models. Evaluated on a 128-channel EEG dataset across six subjects, SENSE matches or surpasses the generative quality of fully fine-tuned baselines such as Thought2Text while substantially reducing computational overhead. By localizing neural decoding and sharing only derived textual cues, SENSE provides a scalable and privacy-aware retrieval-augmented architecture for next-generation BCIs.
Figures & tables
Figure 1. The SENSE privacy-preserving architecture. Raw, high-dimensional EEG signals are processed within a local "Privacy Preserving Boundary" to extract non-sensitive semantic anchors. Only discrete, interpretable keywords are transmitted to remote Large Language Models for final linguistic synthesis. A vertical flowchart illustrating the SENSE privacy-preserving data flow. At the top, a brain icon emits Raw EEG signals. These signals enter a box labeled YOUR LOCAL DEVICE (ON-DEVICE), which contains a dashed Privacy Preserving Boundary. Inside this boundary, an 'EEG ENCODER' generates a Pooled EEG (512-d) vector, which is then processed by the Similarity Refiner neural network. The refiner outputs a Non-sensitive Bag-of-words (BoW) —example keywords shown are backrest, chair, cushion, table. This discrete BoW is the only information that exits the local device to reach a Remote LLM Decoder (Gemini, ChatGPT, LLaMa). At the bottom, the remote LLM produces the final output: "A wooden bench with a cushion and backrest."
Figure 2. SENSE pipeline overview: ChannelNet EEG encoder ( Mishra et al., 2025 ) extracts vector ; a Similarity Refiner maps . is matrix-multiplied with CLIP-space vocabulary embeddings to produce logits. A top- selector ( ) yields salient tokens forming a Bag-of-Words (BoW). The BoW and predicted object label prompt an off-the-shelf LLM for caption reconstruction. The refiner is trained against ground-truth -hot vectors with specialized losses addressing class imbalance. A high-level architectural flowchart of the SENSE pipeline. The diagram is organized into three horizontal stages: Encoding, Refining/Matching, and Decoding. In the Encoding stage, Raw EEG Signals are processed by a frozen EEG ChannelNet Encoder to produce a 512-dimensional vector x. Simultaneously, a Word Corpus is processed by a frozen CLIP Text Encoder to generate a 1210-word vocabulary embedding matrix E. In the Refining stage, vector x enters a Similarity Refiner (indicated as the only trainable component) to produce a refined latent z. This latent z is matrix-multiplied with the transpose of the embedding matrix E to produce predicted logits. A loss calculation compares these logits against an N-hot vector derived from ground-truth caption words to update the refiner. In the Decoding stage, the predicted logits pass through a Top-k Selector funnel, which extracts a Bag-of-Words (e.g., backrest, chair, cushion). This BoW then prompts a frozen LLM Decoder (Gemini, ChatGPT, etc.) to perform the final Caption Reconstruction.
| Similarity Refiner | Decoder | ROUGE-N (%) | ROUGE-L (%) | BLEU-N (%) | METEOR (%) | BERT Score | GPT-5 | |||
| Loss Function Variant | N=1 95% CI | N=2 95% CI | 95% CI | N=1 95% CI | N=4 95% CI | 95% CI | Flu. | Ade. | ||
| Naive Baseline | 25.8 [ 0.5] | 4.8 [ 0.3] | 23.8 [ 0.5] | 21.6 [ 0.5] | 3.8 [ 0.2] | 20.2 [ 0.5] | 0.888 | 4.59 | 1.22 | |
| 22.7 [ 0.4] | 3.1 [ 0.2] | 20.7 [ 0.4] | 20.0 [ 0.4] | 3.1 [ 0.1] | 17.1 [ 0.4] | 0.877 | 4.58 | 1.28 | ||
| 25.8 [ 0.6] | 5.2 [ 0.4] | 24.5 [ 0.6] | 18.1 [ 0.6] | 3.6 [ 0.2] | 19.3 [ 0.5] | 0.888 | 4.70 | 1.28 | ||
| 23.9 [ 0.5] | 3.8 [ 0.3] | 22.3 [ 0.4] | 18.9 [ 0.5] | 3.1 [ 0.1] | 16.9 [ 0.4] | 0.879 | 4.45 | 1.07 | ||
| 10.5 [ 0.5] | 1.2 [ 0.2] | 9.9 [ 0.4] | 5.3 [ 0.4] | 1.0 [ 0.1] | 10.6 [ 0.3] | 0.877 | 4.24 | 1.22 | ||
Table 1. Average evaluation metrics and GPT-5 assessment ( Flu. : fluency, Ade. : adequacy) for text generated from Bag-of-Words (BoW) across four LLMs: GPT-4o-mini, Gemini 2.5 Flash Lite, LLaMa-3-8B, and Qwen2.5-7B. Decoders are appended with subscripts indicating configurations evaluated with visual grounding context ( w for WithObj ), solely from neural BoW ( wo for WithoutObj ), or under fully fine-tuned baselines ( all ). For ROUGE-1/2/L, BLEU-1/4, METEOR and BERT Score, metrics report the Mean the 95% Bootstrap Margin of Error calculated over 10,000 resamples; for BERT Score, the 95% Bootstrap Margin of Error is uniformly across all settings. For all metrics, higher values indicate superior performance. The best results within evaluation domains are bolded and second-best are italicized ; matching ties are underlined . A quantitative results table comparing four Similarity Refiner variants—Naive Baseline, Binary Cross Entropy, Contrastive Multi Label, and Focal Loss—across four LLM decoders: GPT-4o-mini, Gemini 2.5 Flash Lite, LLaMa-3-8B, and Qwen2.5-7B. The table reports ROUGE-1/2/L, BLEU-1/4, METEOR, BERT Score, and GPT-5-based Fluency and Adequacy scores. Each decoder is tested with and without the primary object label (WithObj vs. WithoutObj). A final section provides a performance comparison against the fine-tuned Thought2Text baseline. The data demonstrates that Focal Loss paired with Gemini 2.5 Flash Lite (WithObj) achieves the highest performance among the proposed SENSE configurations, matching or exceeding the fine-tuned Thought2Text baseline across most semantic and lexical overlap metrics.
Figure 3. Subject-wise BLEU-1 (%) performance across optimization strategies and LLM decoders. The radar charts demonstrate that premier closed-source models (Gemini and ChatGPT) consistently form the outer performance boundaries across all six subjects, with ChatGPT exhibiting a slight advantage in the Naive Baseline task. Furthermore, the topological consistency of the polygons indicates that our framework generalizes effectively across diverse neural patterns without requiring per-subject fine-tuning. A composite figure consisting of six radar (spider) charts arranged in two rows and three columns, corresponding to Subjects 1 through 6. Each chart plots BLEU-1 scores across four axes representing optimization strategies: Focal Loss (top), Contrastive Multi-Label (right), BCE Loss (bottom), and Naive Baseline (left). The radial axes are scaled from 0.1 to 0.4. Four colored polygons represent different LLM decoders: blue for Gemini, red for ChatGPT, orange for LLaMa-3-8B, and green for Qwen2.5-7B. In every subject chart, the blue and red polygons are the outermost, peaking near 0.4 for Focal Loss and Contrastive ML, indicating superior performance. The green and orange polygons are consistently nested inside, reflecting lower scores. The geometric shape of the performance polygons is highly consistent across all six subjects, demonstrating stable cross-subject generalization. A shared legend at the bottom identifies the four models by color, and the 4 training strategies.
| ID | Image | Ref. Obj | Pred. Obj | Reference Description | Thought2Text | Sample BoW | SENSE Description |
| 1 | mushroom | mushroom | A large yellow mushroom with a brown stem and a brown cap, surrounded by green foliage. | A group of mushrooms growing on a log. | yellow, mushroom, white, daisy, flower | A yellow mushroom growing in the green grass. | |
| 2 | piano | piano | A black grand piano in a living room. | A grand piano with a stool in front of it. | piano, black, room, grand, floor | A grand piano on a wooden floor. | |
| 3 | camping | camping | A tent in a mountainous area with trees and fog. | A tent set up in a forest with a campfire nearby. | tent, field, grassy, lawn | A tent is set up in a grassy field for camping. | |
| 4 | pumpkin | pumpkin | A carved pumpkin with a face and eyes, sitting on a table. | A carved pumpkin with a spooky face on it. | carved, pumpkin, face, halloween | A carved pumpkin with a spooky face sits on the porch. | |
| 5 | flower | mushroom | A black and white photograph of a single daisy with a white center and a dark brown center. | A group of mushrooms growing on a log. | mushroom, grow, white, forest, red | A white mushroom grows on the forest ground. | |
| 6 | coffee mug | coffee mug | A hand holding a mug with a blue background and a handprint design. | A person holding a coffee mug with a A person holding a coffee mug with the words “World’s Best Dad” written on it. | mug, coffee, white, black, cup | A white coffee mug with a black logo on a table. |
Table 2. Qualitative comparison of generated descriptions from SENSE and the Thought2Text baseline. The Sample BoW column shows a representative subset of the Bag-of-Words provided to the LLM during caption generation. Samples 7 and 8 illustrate ablation results where no object label was supplied to the model. ( SENSE captions generated by Gemini 2.5 Flash Lite .) A qualitative comparison table with 8 rows. Each row contains an image, its reference object, predicted object, reference description, a Thought2Text baseline caption, a Sample Bag-of-Words, and the SENSE Description. Row 1-4 show successful retrievals (mushroom, piano, camping, pumpkin) where SENSE captions are highlighted in green and match the semantic content of the images. Row 5-8 show error cases (flower misidentified as mushroom, coffee mug, guitar misidentified as train, and piano misidentified as ball) highlighted in red. In the error cases, the SENSE description remains fluent but reflects the incorrect object label provided by the refiner, illustrating the system's reliance on accurate semantic cues.