FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller

Anna Tegon; Nicholas Lehmann; Yawei Li; Andrea Cossettini; Luca Benini; Thorir Mar Ingolfsson

eess.SPMar 18, 2026

FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller

Authors: Anna Tegon, Nicholas Lehmann, Yawei Li, Andrea Cossettini, Luca Benini, Thorir Mar Ingolfsson

Organizations: Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · DEI, University of Bologna, Bologna, Italy

Abstract

Objective: To enable continuous, long-term neuro-monitoring on wearable devices by overcoming the computational bottlenecks of Transformer-based Electroencephalography (EEG) foundation models and the quantization challenges inherent to State-Space Models (SSMs). Methods: We present FEMBA, a bidirectional Mamba architecture pre-trained on over 21,000 hours of EEG. We introduce a novel Physiologically-Aware pre-training objective, consisting of a reconstruction with low-pass filtering, to prioritize neural oscillations over high-frequency artifacts. To address the activation outliers common in SSMs, we employ Quantization-Aware Training (QAT) to compress the model to 2-bit weights. The framework is deployed on a parallel ultra-low-power RISC-V microcontroller (GAP9) using a custom double-buffered memory streaming scheme. Results: The proposed low-pass pre-training improves downstream AUROC on TUAB from 0.863 to 0.893 and AUPR from 0.862 to 0.898 compared to the best contrastive baseline. QAT successfully compresses weights with negligible performance loss, whereas standard post-training quantization degrades accuracy by approximately 30%. The embedded implementation achieves deterministic real-time inference (1.70~s per 5~~s window) and reduces the memory footprint by 74% (to $\approx$ 2~~MB), achieving competitive accuracy with up to \textbf{27 $\times$ } fewer FLOPs than Transformer benchmarks. Conclusion: FEMBA demonstrates that Mamba-based foundation models can be effectively quantized and deployed on extreme-edge hardware without sacrificing the representation quality required for robust clinical analysis. Significance: This work establishes the first full-stack framework for deploying large-scale EEG foundation models on ultra-low-power wearables, facilitating continuous, SSM based monitoring for epilepsy and sleep disorders.

Figures & tables

Model	Architecture	Size	Datasets	Complexity	Deployment
EEGFormer	Transformer	2.3M	TUAR / TUSL	$\mathcal{O}(CN^{2})$	No
LaBraM	Transformer	5.9M	TUAB	$\mathcal{O}(C^{2}N^{2})$	No
LUNA	Transformer	7M	TUAB / TUAR / TUSL	$\mathcal{O}(CN^{2})+\mathcal{O}(CN)$	No
FEMBA	Mamba-based	7.8M	TUAB / TUAR / TUSL	$\mathcal{O}(CN)$	Yes

TABLE I : Comparison of EEG foundation models

Property	TUEG	TUAB	TUAR	TUSL
Number of Subjects	$14{,}987$	$2{,}329$	$213$	$38$
Number of Channels	22	22	22	22
Sampling Rate	256 Hz	256 Hz	256 Hz	256 Hz
Hours of Recordings	$21{,}787{.}32$	$1{,}139{.}31$	$83{.}74$	$27{.}54$
Training Samples	$13{,}236{,}000$	$591{,}357$	$49{,}241$	$16{,}088$
Validation Samples	$489{,}600$	$154{,}938$	$5{,}870$	$1{,}203$

TABLE II : Dataset Statistics

Panel of Figure 3 — Fig. 1 : t-SNE visualization of embedding spaces on downstream tasks. Left : Embeddings generated using the earlier training pipeline show lower class separability. Right : Embeddings obtained with the updated pipeline yield tighter, more distinct clusters for artifact classes (e.g., muscle movement), illustrating an improvement in the learned representation quality.

SSL Task	TUAR		TUSL		TUAB
SSL Task	AUROC	AUPRC	AUROC	AUPRC	Accuracy (%)	AUROC	AUPRC
Rec. – Random Masking	$0.912\pm 0.002$	$0.532\pm 0.011$	$0.699\pm 0.028$	$0.281\pm 0.019$	$80.38\pm 0.08$	$0.8762\pm 0.0012$	$0.8773\pm 0.0011$
Rec. – Clustered Random	$0.917\pm 0.004$	$0.547\pm 0.027$	$0.712\pm 0.023$	$0.281\pm 0.019$	$81.38\pm 0.09$	$0.8663\pm 0.001$	$0.8654\pm 0.0011$
Rec. – Random w/ Lowpass	$0.916\pm 0.008$	$0.533\pm 0.024$	$0.750\pm 0.035$	$0.294\pm 0.009$	$\mathbf{81.95\pm 0.09}$	$\mathbf{0.8930\pm 0.0007}$	$\mathbf{0.8976\pm 0.0018}$
Contrastive – Frequency	$0.921\pm 0.007$	$0.534\pm 0.023$	$0.751\pm 0.020$	$0.291\pm 0.014$	$78.63\pm 0.09$	$0.8633\pm 0.0007$	$0.8615\pm 0.0015$
Contrastive – Masking	$0.915\pm 0.012$	$0.533\pm 0.014$	$0.695\pm 0.020$	$0.272\pm 0.004$	$80.69\pm 0.07$	$0.8732\pm 0.0006$	$0.8691\pm 0.0007$

TABLE III : Comparison of Fine-tuning Results Across TUAR, TUSL, and TUAB Datasets for Different Pre-training Strategies

Model	Size	TUAR		TUSL
		AUROC $\uparrow$	AUC-PR $\uparrow$	AUROC $\uparrow$	AUC-PR $\uparrow$
Supervised Models
EEGNet [ 21 ]	-	$0.752\pm 0.006$	$0.433\pm 0.025$	$0.635\pm 0.015$	$0.351\pm 0.006$
EEG-GNN [ 50 ]	-	$0.837\pm 0.022$	$0.488\pm 0.015$	$0.721\pm 0.009$	$0.381\pm 0.004$
GraphS4mer [ 51 ]	-	$0.833\pm 0.006$	$0.461\pm 0.024$	$0.632\pm 0.017$	$0.359\pm 0.001$
Self-supervised Models

TABLE IV : Comparison of model performance on TUAR and TUSL datasets.

Model	Size	Bal. Acc. (%) $\uparrow$	AUC-PR $\uparrow$	AUROC $\uparrow$
Supervised Models
SPaRCNet [ 52 ]	0.8M	78.96 $\pm$ 0.18	0.8414 $\pm$ 0.0018	0.8676 $\pm$ 0.0012
ContraWR [ 53 ]	1.6M	77.46 $\pm$ 0.41	0.8421 $\pm$ 0.0140	0.8456 $\pm$ 0.0074
CNN-Transformer [ 54 ]	3.2M	77.77 $\pm$ 0.22	0.8433 $\pm$ 0.0039	0.8461 $\pm$ 0.0013
FFCL [ 55 ]	2.4M	78.48 $\pm$ 0.38	0.8448 $\pm$ 0.0065	0.8569 $\pm$ 0.0051
ST-Transformer [ 56 ]	3.2M	79.66 $\pm$ 0.23	0.8521 $\pm$ 0.0026	0.8707 $\pm$ 0.0019

TABLE V : Performance comparison on TUAB abnormal EEG detection.

Configuration	Method	AUROC	AUPR	Accuracy
FP32 (Baseline)	–	0.89	0.89	81.84
W8A8	PTQ	0.77	0.71	55.91
	QAT	0.88	0.88	81.02
W4A8	PTQ	0.71	0.67	55.79
	QAT	0.88	0.88	80.88
W2A8	PTQ	0.56	0.49	54.12

TABLE VI : Quantization Results: Comparison of PTQ and QAT Performance

Metric	INT8 (W8A8)	2-bit (W2A8)
Total Cycles	629.4 M	625.9 M
Inference Time	1.70 s	1.69 s
Compute Utilization	98.6%	98.7%
Idle/Overhead	0.5%	0.5%
DMA/Compute Overlap	99.4%	99.5%

TABLE VII : FEMBA-Tiny Performance on GAP9 @ 370 MHz

Layer	Cycles (M)	% Total	Overlap
patch_embed	8.0	1.3%	81.1%
pos_embed	2.3	0.4%	100.0%
mamba_blocks.0	310.3	49.3%	100.0%
mamba_blocks.1	308.2	49.0%	99.4%
global_pool	0.5	0.1%	100.0%
classifier	$<$ 0.1	$<$ 0.01%	41.9%

TABLE VIII : Layer-by-Layer Cycle Breakdown

Operation	Cycles (M)	%	MACs (M)	MACs/Cyc
Input Projection	71.7	23.2%	189.7	2.65
Sequence Reversal	1.6	0.5%	—	—
Local Temporal Conv.	8.8	2.8%	1.0	0.11
Selective SSM Scan	199.5	64.6%	63.1	0.32
Output Projection	24.3	7.9%	94.9	3.91
Sequence Reversal	1.0	0.3%	—	—

TABLE IX : Sub-operation Breakdown for mamba_blocks.0

EEG Foundation Models

Electroencephalography (EEG) foundation models learn reusable representations of brain signals for transfer across people, tasks, and recording setups.

See all 101 papers

LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling
Mar 19, 2026Danaé Broustail, Anna Tegon, Thorir Mar Ingolfsson +2EEG Foundation Models Electroencephalography
Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · DEI, University of Bologna, Bologna, Italy
BrainCodes: A Lightweight EEG Foundation Model over Discrete Neural Codes
Oct 21, 2026Kleanthis Avramidis, Ivan Tashev, Dimitra EmmanouilidouEEG Foundation Models Representation Learning
Signal Analysis and Interpretation Lab, University of Southern California, Los Angeles, CA 90007, USA · Microsoft Research, Redmond, WA, USA
ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?
Aug 7, 2026Lingwei Li, Yirong Kan, Peng Chen +3Efficient Transformer Inference Token Merging
Nara Institute of Science and Technology, Nara, Japan · RIKEN Center for Computational Science, Hyogo, Japan · University of Illinois Urbana-Champaign, USA +1

Mar 19, 2026J/K move · Enter open · S save

LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling

Danaé Broustail, Anna Tegon, Thorir Mar Ingolfsson, Yawei Li+1

Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · DEI, University of Bologna, Bologna, Italy

EEG Foundation Models Electroencephalography Self-Supervised Time Series Representation Learning Mamba Joint-Embedding Predictive Architecture

Open paper AskPDF arXiv

Electroencephalography (EEG) enables non-invasive monitoring of brain activity across clinical and neurotechnology applications, yet building foundation models for EEG remains challenging due to differing electrode topologies and computational scalability, as Transformer architectures incur quadratic sequence complexity. As a joint solution, we propose LuMamba (Latent Unified Mamba), a self-supervised framework combining topology-invariant encodings with linear-complexity state-space modeling, using LUNA's learned-query cross-attention mechanism for channel unification, and FEMBA's bidirectional Mamba blocks for efficient temporal modeling. Within this architecture, we provide the first systematic investigation of the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) for biosignal learning. Pre-trained on over 21,000 hours of unlabeled EEG from the TUEG corpus, LuMamba is evaluated on five downstream tasks spanning abnormality detection, artifact recognition, and mental condition classification across electrode configurations ranging from 16 to 26 channels. In the pre-training objective, masked reconstruction alone yields structured but less generalizable representations, while LeJEPA alone produces diffuse embeddings; combining both objectives achieves the most robust performance. With only 4.6M parameters, LuMamba attains 80.99% balanced accuracy on TUAB and achieves state-of-art performance on Alzheimer's detection (0.97 AUPR), while requiring 377x fewer FLOPS than state-of-art models at equivalent sequence lengths and scaling to 12x longer sequences before reaching typical GPU memory limits. Code is available at https://github.com/pulp-bio/biofoundation.

Abstract

EEG Foundation Models

Explore similar work

LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling

BrainCodes: A Lightweight EEG Foundation Model over Discrete Neural Codes

ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?