FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller

Anna Tegon; Nicholas Lehmann; Yawei Li; Andrea Cossettini; Luca Benini; Thorir Mar Ingolfsson

eess.SPMar 18, 2026

FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller

Authors: Anna Tegon, Nicholas Lehmann, Yawei Li, Andrea Cossettini, Luca Benini, Thorir Mar Ingolfsson

Organizations: Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · DEI, University of Bologna, Bologna, Italy

Abstract

Objective: To enable continuous, long-term neuro-monitoring on wearable devices by overcoming the computational bottlenecks of Transformer-based Electroencephalography (EEG) foundation models and the quantization challenges inherent to State-Space Models (SSMs). Methods: We present FEMBA, a bidirectional Mamba architecture pre-trained on over 21,000 hours of EEG. We introduce a novel Physiologically-Aware pre-training objective, consisting of a reconstruction with low-pass filtering, to prioritize neural oscillations over high-frequency artifacts. To address the activation outliers common in SSMs, we employ Quantization-Aware Training (QAT) to compress the model to 2-bit weights. The framework is deployed on a parallel ultra-low-power RISC-V microcontroller (GAP9) using a custom double-buffered memory streaming scheme. Results: The proposed low-pass pre-training improves downstream AUROC on TUAB from 0.863 to 0.893 and AUPR from 0.862 to 0.898 compared to the best contrastive baseline. QAT successfully compresses weights with negligible performance loss, whereas standard post-training quantization degrades accuracy by approximately 30%. The embedded implementation achieves deterministic real-time inference (1.70~s per 5~~s window) and reduces the memory footprint by 74% (to $\approx$ 2~~MB), achieving competitive accuracy with up to \textbf{27 $\times$ } fewer FLOPs than Transformer benchmarks. Conclusion: FEMBA demonstrates that Mamba-based foundation models can be effectively quantized and deployed on extreme-edge hardware without sacrificing the representation quality required for robust clinical analysis. Significance: This work establishes the first full-stack framework for deploying large-scale EEG foundation models on ultra-low-power wearables, facilitating continuous, SSM based monitoring for epilepsy and sleep disorders.

Figures & tables

Model	Architecture	Size	Datasets	Complexity	Deployment
EEGFormer	Transformer	2.3M	TUAR / TUSL	$\mathcal{O}(CN^{2})$	No
LaBraM	Transformer	5.9M	TUAB	$\mathcal{O}(C^{2}N^{2})$	No
LUNA	Transformer	7M	TUAB / TUAR / TUSL	$\mathcal{O}(CN^{2})+\mathcal{O}(CN)$	No
FEMBA	Mamba-based	7.8M	TUAB / TUAR / TUSL	$\mathcal{O}(CN)$	Yes

TABLE I : Comparison of EEG foundation models

Property	TUEG	TUAB	TUAR	TUSL
Number of Subjects	$14{,}987$	$2{,}329$	$213$	$38$
Number of Channels	22	22	22	22
Sampling Rate	256 Hz	256 Hz	256 Hz	256 Hz
Hours of Recordings	$21{,}787{.}32$	$1{,}139{.}31$	$83{.}74$	$27{.}54$
Training Samples	$13{,}236{,}000$	$591{,}357$	$49{,}241$	$16{,}088$
Validation Samples	$489{,}600$	$154{,}938$	$5{,}870$	$1{,}203$

TABLE II : Dataset Statistics

Panel of Figure 3 — Fig. 1 : t-SNE visualization of embedding spaces on downstream tasks. Left : Embeddings generated using the earlier training pipeline show lower class separability. Right : Embeddings obtained with the updated pipeline yield tighter, more distinct clusters for artifact classes (e.g., muscle movement), illustrating an improvement in the learned representation quality.

SSL Task	TUAR		TUSL		TUAB
SSL Task	AUROC	AUPRC	AUROC	AUPRC	Accuracy (%)	AUROC	AUPRC
Rec. – Random Masking	$0.912\pm 0.002$	$0.532\pm 0.011$	$0.699\pm 0.028$	$0.281\pm 0.019$	$80.38\pm 0.08$	$0.8762\pm 0.0012$	$0.8773\pm 0.0011$
Rec. – Clustered Random	$0.917\pm 0.004$	$0.547\pm 0.027$	$0.712\pm 0.023$	$0.281\pm 0.019$	$81.38\pm 0.09$	$0.8663\pm 0.001$	$0.8654\pm 0.0011$
Rec. – Random w/ Lowpass	$0.916\pm 0.008$	$0.533\pm 0.024$	$0.750\pm 0.035$	$0.294\pm 0.009$	$\mathbf{81.95\pm 0.09}$	$\mathbf{0.8930\pm 0.0007}$	$\mathbf{0.8976\pm 0.0018}$
Contrastive – Frequency	$0.921\pm 0.007$	$0.534\pm 0.023$	$0.751\pm 0.020$	$0.291\pm 0.014$	$78.63\pm 0.09$	$0.8633\pm 0.0007$	$0.8615\pm 0.0015$
Contrastive – Masking	$0.915\pm 0.012$	$0.533\pm 0.014$	$0.695\pm 0.020$	$0.272\pm 0.004$	$80.69\pm 0.07$	$0.8732\pm 0.0006$	$0.8691\pm 0.0007$

TABLE III : Comparison of Fine-tuning Results Across TUAR, TUSL, and TUAB Datasets for Different Pre-training Strategies

Model	Size	TUAR		TUSL
		AUROC $\uparrow$	AUC-PR $\uparrow$	AUROC $\uparrow$	AUC-PR $\uparrow$
Supervised Models
EEGNet [ 21 ]	-	$0.752\pm 0.006$	$0.433\pm 0.025$	$0.635\pm 0.015$	$0.351\pm 0.006$
EEG-GNN [ 50 ]	-	$0.837\pm 0.022$	$0.488\pm 0.015$	$0.721\pm 0.009$	$0.381\pm 0.004$
GraphS4mer [ 51 ]	-	$0.833\pm 0.006$	$0.461\pm 0.024$	$0.632\pm 0.017$	$0.359\pm 0.001$
Self-supervised Models

TABLE IV : Comparison of model performance on TUAR and TUSL datasets.

Model	Size	Bal. Acc. (%) $\uparrow$	AUC-PR $\uparrow$	AUROC $\uparrow$
Supervised Models
SPaRCNet [ 52 ]	0.8M	78.96 $\pm$ 0.18	0.8414 $\pm$ 0.0018	0.8676 $\pm$ 0.0012
ContraWR [ 53 ]	1.6M	77.46 $\pm$ 0.41	0.8421 $\pm$ 0.0140	0.8456 $\pm$ 0.0074
CNN-Transformer [ 54 ]	3.2M	77.77 $\pm$ 0.22	0.8433 $\pm$ 0.0039	0.8461 $\pm$ 0.0013
FFCL [ 55 ]	2.4M	78.48 $\pm$ 0.38	0.8448 $\pm$ 0.0065	0.8569 $\pm$ 0.0051
ST-Transformer [ 56 ]	3.2M	79.66 $\pm$ 0.23	0.8521 $\pm$ 0.0026	0.8707 $\pm$ 0.0019

TABLE V : Performance comparison on TUAB abnormal EEG detection.

Configuration	Method	AUROC	AUPR	Accuracy
FP32 (Baseline)	–	0.89	0.89	81.84
W8A8	PTQ	0.77	0.71	55.91
	QAT	0.88	0.88	81.02
W4A8	PTQ	0.71	0.67	55.79
	QAT	0.88	0.88	80.88
W2A8	PTQ	0.56	0.49	54.12

TABLE VI : Quantization Results: Comparison of PTQ and QAT Performance

Metric	INT8 (W8A8)	2-bit (W2A8)
Total Cycles	629.4 M	625.9 M
Inference Time	1.70 s	1.69 s
Compute Utilization	98.6%	98.7%
Idle/Overhead	0.5%	0.5%
DMA/Compute Overlap	99.4%	99.5%

TABLE VII : FEMBA-Tiny Performance on GAP9 @ 370 MHz

Layer	Cycles (M)	% Total	Overlap
patch_embed	8.0	1.3%	81.1%
pos_embed	2.3	0.4%	100.0%
mamba_blocks.0	310.3	49.3%	100.0%
mamba_blocks.1	308.2	49.0%	99.4%
global_pool	0.5	0.1%	100.0%
classifier	$<$ 0.1	$<$ 0.01%	41.9%

TABLE VIII : Layer-by-Layer Cycle Breakdown

Operation	Cycles (M)	%	MACs (M)	MACs/Cyc
Input Projection	71.7	23.2%	189.7	2.65
Sequence Reversal	1.6	0.5%	—	—
Local Temporal Conv.	8.8	2.8%	1.0	0.11
Selective SSM Scan	199.5	64.6%	63.1	0.32
Output Projection	24.3	7.9%	94.9	3.91
Sequence Reversal	1.0	0.3%	—	—

TABLE IX : Sub-operation Breakdown for mamba_blocks.0

EEG Foundation Models

Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.

See all 100 papers

Mar 19, 2026

LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling

Electroencephalography (EEG) enables non-invasive monitoring of brain activity across clinical and neurotechnology applications, yet building foundation models for EEG remains challenging due to differing electrode topologies and computational scalability, as Transformer architectures incur quadratic sequence complexity. As a joint solution, we propose LuMamba (Latent Unified Mamba), a self-supervised framework combining topology-invariant encodings with linear-complexity state-space modeling, using LUNA's learned-query cross-attention mechanism for channel unification, and FEMBA's bidirectional Mamba blocks for efficient temporal modeling. Within this architecture, we provide the first systematic investigation of the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) for biosignal learning. Pre-trained on over 21,000 hours of unlabeled EEG from the TUEG corpus, LuMamba is evaluated on five downstream tasks spanning abnormality detection, artifact recognition, and mental condition classification across electrode configurations ranging from 16 to 26 channels. In the pre-training objective, masked reconstruction alone yields structured but less generalizable representations, while LeJEPA alone produces diffuse embeddings; combining both objectives achieves the most robust performance. With only 4.6M parameters, LuMamba attains 80.99% balanced accuracy on TUAB and achieves state-of-art performance on Alzheimer's detection (0.97 AUPR), while requiring 377x fewer FLOPS than state-of-art models at equivalent sequence lengths and scaling to 12x longer sequences before reaching typical GPU memory limits. Code is available at https://github.com/pulp-bio/biofoundation.

Danaé Broustail, Anna Tegon, Thorir Mar Ingolfsson +2

Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · DEI, University of Bologna, Bologna, Italy

EEG Foundation Models Electroencephalography Self-Supervised Time Series Representation Learning Mamba Joint-Embedding Predictive Architecture

Ask PDF

Oct 21, 2026

BrainCodes: A Lightweight EEG Foundation Model over Discrete Neural Codes

Recent electroencephalography (EEG) foundation models increasingly emphasize model scale, potentially limiting their deployment on resource-constrained devices and in everyday environments. We argue that transferable EEG features can instead be learned with a compact architecture and introduce BrainCodes, a lightweight foundation model pretrained through self-supervised learning on more than 800 hours of recordings from over 1,000 participants. Our architecture combines a quantization-based neural tokenizer with a spatiotemporal encoder that supports arbitrary electrode configurations. The pretraining objective explicitly encourages representations that capture spatial connectivity through channel positional encoding, and sensitivity to narrowband neural oscillations through an auxiliary training objective. Our model matches or surpasses the state of the art across diverse neurophysiological monitoring tasks, including sleep staging, attention assessment, and cognitive load estimation. BrainCodes transfers effectively to low-resource wearable devices while using up to 97% fewer trainable parameters than comparable architectures. Beyond predictive accuracy, our evaluation framework characterizes the geometry and invariances of learned EEG representations across measurable spatial, temporal, and spectral signal properties.

Kleanthis Avramidis, Ivan Tashev, Dimitra Emmanouilidou

Signal Analysis and Interpretation Lab, University of Southern California, Los Angeles, CA 90007, USA · Microsoft Research, Redmond, WA, USA

EEG Foundation Models Representation Learning Lightweight Neural Networks Self-Supervised Pre-Training

Ask PDF

Aug 7, 2026

ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?

This work investigates whether Electroencephalograph (EEG) foundation models (EFMs) can be made faster and locally deployable without sacrificing accuracy. EEG foundation models are a major trend, offering strong general-purpose representations. However, their computational burden grows quadratically with input length, hindering deployment on resource-constrained scenario, particularly for real-time clinical monitoring. EEG's low SNR further suggests many of these tokens are redundant and compressible with little accuracy cost. We propose ZIPBrain, a novel redundancy-aware EEG token pooling module that leverages this low-SNR characteristic to reduce token count. Given a token sequence, ZIPBrain partitions tokens into redundant and unique groups, then merges each redundant token with its most similar counterpart in the unique group. Furthermore, ZIPBrain serves as a training-free, plug-and-play module that seamlessly integrates into standard Transformer encoders with negligible computational overhead. Extensive experiments across multiple EEG foundation models show ZIPBrain's strong versatility, achieving 1.3%-10.5% average improvement over baselines, while reducing wall-clock inference time by 32.7% (up to 41.8% with CUDA Graph) compared to the original EEG foundation models.

Lingwei Li, Yirong Kan, Peng Chen +3

Nara Institute of Science and Technology, Nara, Japan · RIKEN Center for Computational Science, Hyogo, Japan · University of Illinois Urbana-Champaign, USA +1

Efficient Transformer Inference Token Merging EEG Foundation Models

Ask PDF

Abstract

EEG Foundation Models

Explore similar work

LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling

BrainCodes: A Lightweight EEG Foundation Model over Discrete Neural Codes

ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?