eess.SPOct 23, 2026

EMG Tokenization Preserves Movement Information at a Fraction of the Bitrate

Authors: Raul C. Sîmpetru, Amin Olamazadeh, Devon Rohlf, Vlad Cnejevici, Alessandro Del Vecchio

Organizations: Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91052 Erlangen, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Wireless electromyography (EMG) interfaces must balance electrode coverage against transmission bandwidth. Here, we show that compact EMG tokens substantially reduce payload while retaining information for three-dimensional hand-pose decoding. We evaluate two causal tokenizers fitted without pose labels: an encoder pretrained to reconstruct EMG spectra and one based on SPAR, an engineered spectral-spatial representation of neighbouring electrodes. In personalized, within-session experiments with 320-channel EMG from nine participants, we compare tokenized and continuous features using matched decoder architectures and training procedures. Averaged across decoders, tokenization changed mean error by +0.01 mm for SPAR (95% confidence interval: -0.12 to 0.14) and +0.45 mm for the learned representation (0.10 to 0.82). Both increases met a 1 mm non-inferiority criterion. Stored token arrays reduced payloads 41-fold relative to raw EMG and 36-fold relative to continuous SPAR features. On sampled windows of the public emg2pose benchmark's test partition, absolute differences in mean error between learned tokens and continuous features were below 0.05 mm across 176 participants and in the subset of 20 participants unseen during training. Electrode coverage, calibration, and unseen recordings affected SPAR decoding more than quantization did. In a computational benchmark, optimized SPAR generated tokens for 63 channels within 20 ms on a Cortex-M55 microcontroller. These findings support tokenization as a compact interface between wearable sensing and movement decoding, allowing decoders to be updated while the wearable encoder and transmitted representation remain fixed.

Figures & tables

Fig. 1: From high-density electromyography (EMG) recordings to a transmittable representation. A: Nine participants performed individual finger movements, grasping, and two- and three-finger pinches. Markerless hand tracking was recorded alongside surface EMG (sEMG) from 320 electrodes distributed across five grids. The lower band illustrates an eight-channel subset obtained by selecting every 40th channel from the 320-channel recording. B: Continuous EMG features and discrete tokens are used to train three-dimensional (3D) hand-pose decoders on a high-performance computing (HPC) system. Their decoding errors are compared using the same decoder architecture and training procedure within each pair. C: An assumed Bluetooth LE (BLE) 2M application throughput of 1.4 Mbit/s [ 1 ] supports up to 42 channels of raw 16-bit EMG at 2,048 Hz, and the 43rd exceeds the budget. Packed 10-bit tokens at 50 Hz would support 2,800 channels. These are payload-only calculations before application packet headers, not measured transmission or computational capacities (Section ; Fig. ; Supplementary Data 1).
Fig. 2: Learned electromyography (EMG) tokenizer and intermediate representations. A: A shared encoder processes 160 ms EMG windows every 20 ms. Four residual blocks extend the receptive field to 480 ms, while a fixed codebook maps each 256-dimensional output to one of 1,024 tokens (10 bits). Layer dimensions such as 32→25632\rightarrow 256 denote the input and output feature dimensions. Conv1D denotes a one-dimensional convolution, GELU the Gaussian error linear unit, LayerNorm layer normalization, and L2 the Euclidean distance. B: Example normalized EMG waveform spanning 480 ms (participant 1, electrode 1). C: Three of the 256 intermediate features (indices 1, 128, and 256) after the second residual block, shown at nine consecutive time points spaced 20 ms apart. D: The resulting 256-dimensional feature vector, mapped to token identifier (ID) 93. Time zero in C corresponds to 480 ms in B.
Fig. 3: Neighbourhoods and boundary handling of the spectral–spatial representation. Grey marks the centre electrode, red the horizontal and vertical neighbours, and blue the auxiliary electrode. At a boundary, an electrode on the opposite side of the centre replaces the missing neighbour (dashed arrow), and its coupling is set to zero after the matrix logarithm. Each band contributes the centre’s diagonal entry and the real horizontal and vertical couplings.
Fig. 4: Hand-pose decoding accuracy, computational cost, and quantization effects. A: Unfiltered three-dimensional (3D) error over 20 wrist-relative landmarks, decoder parameters including frozen backbones, and 32-bit floating-point (FP32) decoder latency. Qwen3 [ 29 ] and TabPFN [ 30 , 31 ] are frozen pretrained decoders, and TCN is a temporal convolutional network. Grey points are participant means over three seeds, and coloured symbols group means. Latency is the mean and standard deviation per complete-pose prediction (batch size one, RTX 4090), excluding feature extraction and tokenization (TabPFN also includes host-processor work). Dashed lines mark the 20 ms update interval and the retrained convolutional neural network (CNN) from IEEE Transactions on Biomedical Engineering (TBME CNN) [ 2 ] . Brackets compare the spectral–spatial representation (SPAR) with learned tokens in the proposed decoder (*, p=0.016p=0.016 ) and SPAR tokens with the adapted CNN (< 1 mm, p=0.016p=0.016 ). B: Stored-array payload for 320 channels before packet headers. The dashed line marks the assumed Bluetooth LE (BLE) 2M throughput of 175 kB/s [ 1 ] . Labels give stored types: unsigned (uint16) and signed (int16) 16-bit integers for tokens and raw electromyography (EMG), and FP32 for features. C: Token error minus continuous-feature error. Circles are participants, diamonds are means, and bars are 95% confidence intervals. Bold Pooled rows average the differences across decoders within each tokenizer, and both upper 95% confidence limits lie below the dashed 1 mm bound ( p=0.016p=0.016 each). The four pp -values in A and C are from exact paired sign-flip permutation tests, Holm-adjusted together (Section II-I ).
Fig. 5: Decoded hand movements and movement-specific errors. A: Example predictions from participant 6. Traces span 4 s, and scoring uses the final second of each predefined test window. Black indicates measured movement, red spectral–spatial representation (SPAR) tokens, and dashed blue continuous SPAR. Predictions are smoothed with a causal One Euro filter for display only. Traces show the thumb-tip-to-pinky-base distance for the thumb, the fingertip-to-wrist distance for the other digits (its mean over fingertips for the fist), and the thumb-to-fingertip distance for the pinches (averaged over index and middle fingers for the three-finger pinch). Section describes the predictions shown between scored seconds and the filter. B: Unfiltered three-dimensional (3D) error across 20 landmarks for each movement, averaged over nine participants and three seeds. Bars show group means with 95% confidence intervals, and diamonds identify participant 6. Supplementary Data 2 reports the paired comparisons.
Fig. 6: Hand-pose structure in spectral–spatial representation (SPAR) token space. A–C: One three-dimensional Uniform Manifold Approximation and Projection (UMAP) of participant 1’s token histories across eight movements. Each movement is projected separately onto its mean-trajectory plane and repositioned for display (Section II-G ). Figure shows how the projected trajectories change as token history increases from a single update to 960 ms. A: Movement identity, with icons indicating the active fingers. B: Hand closure normalized separately for each movement. C: Phase-aligned, smoothed repetitions (thin) and their mean trajectory (thick), with cycle starts near the open-hand pose (dots) and traversal direction (arrows). D: Pose retrieval for single updates in the test windows, matched against repetitions outside the training windows, using continuous SPAR, dequantized tokens, or token identifiers (IDs) in the original representation space. Thin lines show the nine participants and the thick line their mean. Dashed lines are descriptive references: disrupted token–pose alignment (11.3 mm, a mean of participant medians, for which 1,024-code tokens give 7.3 mm) and matching using measured poses (4.4 mm). The curves and the pose-space reference are phase-balanced means. Asterisks indicate increases over continuous SPAR (exact paired permutation (sign-flip) tests, Holm-adjusted across three comparisons; p=0.043p=0.043 , 0.0160.016 , and 0.0120.012 ). Retrieval, display-selection rules, and reference construction are detailed in Section .
Fig. 7: Computational performance of electromyography (EMG) tokenization on a Cortex-M55 microcontroller. The spectral–spatial representation (SPAR) implementations were tested at 600 MHz with synthetic 2 kHz EMG. Timing includes windowing, normalization, feature extraction, codebook search, and token packing, but excludes acquisition, radio transmission, and pose decoding. A: Median compute time per update over 16 timed updates, after 16 untimed warm-up updates. Open symbols and dashed lines show the original C implementations, and filled symbols and solid lines the optimized ones. Every timed update at the marked SPAR channel limits met the 20 ms interval. B: Optimized SPAR time per channel by processing stage, with labels giving values at 320 channels. The dashed line shows time per channel from a linear fit of total time against channel count through the origin (Section ).
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. S1: Electrode coverage, error, and payload. Points show mean three-dimensional (3D) pose error across nine participants and three decoder initializations, with labels indicating electrode counts. Payload excludes headers and uses ideal 10-bit packing for tokens, 32-bit floating-point values for the continuous spectral–spatial representation (SPAR), and 16-bit integers for raw electromyography (EMG). The dashed line marks the assumed 175 kB/s Bluetooth LE (BLE) application throughput (BLE Limit). The 63-electrode SPAR and 42-electrode raw-EMG convolutional neural network (CNN) pipelines were refitted offline with different decoders. Supplementary Data 1 provides the full results.
DecoderArchitectureTraining and checkpointParameters
Matched TCNLearned encoder features or codebook vectors; width 404; geometry-aware pooling (four slots, four heads); 7 causal blocks, kernel 2, dilations 1–64; temporal dropout 0.1; up to 100 updates (2 s) of contextSmooth- L1L_{1} ; 15,000 steps of two windows sampled with replacement, scoring the final second of each; 500-step warmup to LR 3×10−43\times 10^{-4} , cosine decay to 3×10−63\times 10^{-6} ; seeds 260911–260913; earliest checkpoint with the lowest validation error; 16-bit mixed precision7.55 M
Matched TransformerAs the matched TCN but width 468; 2 pre-normalization layers, 4 heads, feed-forward 936, causal maskAs the matched TCN7.45 M
Matched waveform CNNAdapted CNN architecture with dropout 0.1As the matched TCN, on 156.25 ms waveform windows ending at the scored updates7.49 M
Spatial CNN + TCN (proposed)1×11\times 1 convolution to 64; two spatial convolutions over five groups of 64 electrodes in acquisition order (32 filters, GELU, instance normalization; 5×325\times 32 with dilation (1,2)(1,2) and circular padding, then 5×105\times 10 ); 7 causal temporal blocks (kernel 2, dilations 1–64, MLP width 128, dropout 0.1); head 64→128→128→6064{\to}128{\to}128{\to}60 ; structural zeros reapplied for SPARSmooth- L1L_{1} ; batches of two windows; LR 3×10−43\times 10^{-4} ; up to 40 epochs, stopping after 6 without improvement; lowest validation error; FP32704,700 (689,468 with SPAR)
Adapted waveform CNN [ 6 , 2 ]156.25 ms raw EMG and a 20 Hz low-passed copy, standardized per grid and window; dropout 0.20Coordinate L1L_{1} in mm; AMSGrad, weight decay 0.32, no gradient clipping; 150 epochs of 10,000 endpoints, batch 64; input noise σ=0.1\sigma=0.1 ; OneCycle from 10−410^{-4} to 3.16×10−33.16\times 10^{-3} until 75 %, then SWA with LR annealed to 10−410^{-4} in five epochs; lowest validation error among epoch checkpoints and averaged weights; 16-bit mixed precision7.49 M
Retrained TBME CNN [ 2 ]Published architecture and preprocessing: 93.75 ms windows, notch filter, raw and low-pass inputs, per-grid and per-window min–max normalizationAdapted recipe: L1L_{1} on training-only min–max-scaled targets; AMSGrad; 50 epochs; batches of 64 accumulated to 512; OneCycle peaking at 10−2.510^{-2.5} ; original, 5 dB-noise and magnitude-warped copies; earliest checkpoint with the lowest validation error; 16-bit mixed precision27.4 M
Appendix
TABLE S1: Pose decoders. Architecture, training, and trainable parameters. Neural decoders use AdamW with weight decay 0.01 and gradient-norm clipping at 1 unless their row states otherwise. TCN, temporal convolutional network; CNN, convolutional neural network; GELU, Gaussian error linear unit; MLP, multilayer perceptron; SPAR, spectral–spatial representation; PCA, principal component analysis; TBME, IEEE Transactions on Biomedical Engineering; EMG, electromyography; LR, learning rate; SWA, stochastic weight averaging; MSE, mean squared error; FP32, 32-bit floating point; M, million.
RepresentationConfigurationFittingBits
Learned tokensVQ-pretrained encoder; 1,024-entry codebook of 256-dimensional vectorsSection S-II-B10
Continuous featuresSame encoder, quantization skipped–8,192
Quantized PCAReference encoder, four components; 4/4/4/4, 8/8/4/4, or 16/16/16/16 levelsAxes from sampled training vectors; uniform levels between the 0.1st and 99.9th training percentiles8, 10, 16
FSQ [ 7 ]Reference encoder; 256→128→4256{\to}128{\to}4 projection with GELU, bounded by tanh; 8/5/5/5 levels (1,000 combinations)Straight-through rounding with a temporary 4→128→2564{\to}128{\to}256 reconstruction network, discarded afterwards; MSE, AdamW, LR 3×10−43\times 10^{-4} , weight decay 0.01, 5,000 steps of 1,024 vectors; validation reconstruction error selects the checkpoint10
Engineered kk -means67 features of the mean-removed preceding 480 ms: log⁡(1+x)\log(1+x) of the RMS, mean absolute value, mean absolute successive difference, and 64 Hann-windowed band powersTraining-only standardization; 256 or 1,024 centroids, 40 Lloyd iterations; no pretrained encoder8, 10
Engineered continuousThe same 67 featuresTraining-only standardization2,144
Appendix
TABLE S2: Representations compared on emg2pose. Bits per channel and update, assuming packed indices for tokens and 32-bit floating-point storage for continuous features, before headers. VQ, vector quantization; PCA, principal component analysis; FSQ, finite scalar quantization; GELU, Gaussian error linear unit; MSE, mean squared error; LR, learning rate; RMS, root mean square; EMG, electromyography; FP32, 32-bit floating point; M, million parameters.
Fig. S2: Quantization effects across participants. Points show mean token-minus-continuous error, with negative values favouring tokens. A: Tokenizers were fitted on four separate participants, and personalized decoders were evaluated on nine participants. Bars show 95% confidence intervals. Two-sided tests of a zero token-minus-continuous difference gave Holm-adjusted p=0.32p=0.32 for the spectral–spatial representation (SPAR) and p=0.12p=0.12 for the learned representation. B: Errors are averaged across three pretrained encoders and three decoder initializations. Bars show pointwise 95% participant-bootstrap intervals, with participant counts in parentheses. Unseen participants are part of the official test cohort, while held-out sessions were recorded on the same day. Supplementary Data 4 provides the values.
Fig. S3: Accuracy, transmission rate, and processing time on emg2pose. Rows identify the same method across panels. A: Error differences of finite scalar quantization (FSQ), quantized principal component analysis (PCA), and the other alternatives relative to learned tokens from the reference encoder. Positive values indicate higher error. Bars show pointwise 95% participant-bootstrap intervals over 176 participants and three decoder initializations. Marks use max- TT correction across 12 comparisons: * p<0.05p<0.05 , ** p<0.01p<0.01 ; n.s., not significant. This comparison uses one encoder, whereas Fig. S2 B averages three. B: Data rates for 16 channels at 50 Hz include 9-byte headers, with the dashed line marking raw electromyography (EMG) stored as 32-bit floating-point values at 128.45 kB/s. C: Median processing time from recorded EMG to pose output over 100 updates on a V100 graphics card (NVIDIA) and on four threads of a central processing unit (CPU), relative to the dashed 20 ms interval. BioCodec timing is omitted (Section S-I-B ). Supplementary Data 5 provides all values.
Fig. S4: Effect of temporal context on the Uniform Manifold Approximation and Projection (UMAP) display. The same updates from participant 1 are embedded with increasing amounts of token history, using the UMAP settings, movement-specific projection planes, and group-placement procedure described in Section S-IV-C . A: Current tokens alone. B–F: Current tokens together with lagged updates every 120 ms, extending back to 120, 240, 480, 720, or 960 ms. Colours identify the eight movements. Black lines connect the mean coordinates of 20 equal phase bins, averaged across complete repetitions, with the last bin connected to the first. Panel titles give circularity 4πA/P24\pi A/P^{2} , averaged over movements, where AA and PP are the area and perimeter of these projected trajectories and a circle has circularity one. Circularity changes little beyond the 480 ms used in Fig. . Each panel is fitted and arranged separately, so coordinates are not comparable across panels.
Fig. S5: Decoding error under noise and electrode loss. Decoders for the learned representation, the spectral–spatial representation (SPAR), and the adapted convolutional neural network (CNN) are held fixed during evaluation, following the training procedures in Table S1 . Points show mean three-dimensional (3D) error across nine participants after averaging three decoder initializations and three degradation instances, with 95% confidence intervals. A: The standard deviation of the added Gaussian noise is expressed relative to the root mean square (RMS) of each channel’s electromyography (EMG). B: Scattered electrodes or complete arrays are set to zero after calibration. Curves show absolute, unfiltered errors, whereas tests compare increases from each model’s clean baseline. SPAR comparisons are reported in Supplementary Data 8.
More in this topic

Time Series Tokenization

See all 13 papers

Explore similar work

CardsList
May 5, 2026

From Muscle Bursts to Motor Intent: Self-Supervised Token Modeling for Heterogeneous EMG

Surface electromyography provides a practical way to infer human movement intention from wearable muscle recordings, but models trained under a single acquisition setting often lose reliability when the user, session, electrode layout, or gesture protocol changes. This paper proposes AEMG, a self-supervised learning approach designed to extract reusable neuromuscular representations from diverse EMG sources. Eight public gesture datasets are first transformed into a shared signal format to reduce discrepancies in channel configuration, sensor topology, and recording protocol. Instead of relying on fixed-length sliding windows, AEMG identifies contraction events from energy variations and represents them as compact neuromuscular tokens, while ordered token groups describe the coordinated activity of multiple muscles during motion. A spatially and temporally conditioned Transformer is then used to encode these token sequences, preserving information about electrode position, activation timing, and sequential structure. For pre-training, the model constructs a discrete library of contraction prototypes through vector-quantized reconstruction and further learns contextual dependencies by recovering masked neuromuscular tokens from surrounding observations. Experiments under leave-one-subject-out and low-label adaptation settings show that the learned representation improves robustness to unseen users and reduces the amount of calibration data required for gesture recognition. These findings suggest that event-level token modeling offers a scalable route toward adaptable and data-efficient EMG-based motor-intent understanding.

Zhenghao Huang, Huilin Yao, Kaikai Wang

South China University of Technology

EMG-Based Gesture RecognitionHuman Activity RecognitionSelf-Supervised Time Series Representation LearningSurface Electromyography
AskPDF
Sep 29, 2026

MyoCodec: A Streaming Neural Codec for Electromyography

Neural codecs encode continuous signals into compact sequences of discrete tokens, providing an interface for efficient transmission, storage, and token-based sequence modeling. This paradigm has been widely adopted in modern speech and audio frameworks; however, the biosignal domain still lacks a neural codec designed specifically for low-bitrate streaming and generalization across diverse downstream tasks. We present MyoCodec, a streaming neural codec designed for electromyography (EMG). Inspired by recent neural audio codecs, MyoCodec combines causal Transformers with residual vector quantization to encode continuous EMG signals into different levels of EMG representations spanning from continuous latent features to discrete tokens operating at 50 Hz. Trained on twelve public EMG datasets, MyoCodec achieves favorable performance in both intrinsic codec quality and representative downstream tasks, including typing (emg2qwerty), hand-pose (emg2pose), speech decoding (emg2speech), and speech-to-EMG synthesis (speech2emg). Across these tasks, MyoCodec exhibits strong performance against prior models while providing a compact and causal EMG representation. During streaming inference, it requires compute time of only 0.482 ms for each 20 ms frame, enabling real-time streaming. Also, the discrete token representation provided by MyoCodec has the potential to support integration into language-model based approaches, creating a path toward LLM-based interactive systems, where tokenized EMG representations are directly processed into such language or speech models. Code and model weights are released.

Jihwan Lee, Kleanthis Avramidis, Junhyeok Lee +3

Signal Analysis and Interpretation Lab (SAIL), University of Southern California, USA · Center for Language and Speech Processing, Johns Hopkins University, USA

Topic pending
AskPDF
Oct 4, 2026

EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation

Surface electromyography (sEMG) is a low-power, cost-effective biosignal for hand-pose estimation and gesture classification. In this work, we examine whether self-supervised pretraining on sEMG can yield transferable representations for continuous hand-pose estimation. We introduce EMG-GPT, a causal transformer-based model that operates on discrete sEMG representations from a frozen residual vector quantization (RVQ) tokenizer and learns temporal dynamics through depth-autoregressive future-code prediction. The model combines within-frame integration with causal temporal modeling while preserving the geometry of the pretrained codebook. EMG-GPT shows competitive results in both Regression and Tracking tasks, supporting EMG-only pretraining as a viable approach for learning transferable sEMG representations.

Ettore Magni, Rolandos Alexandros Potamias, Stefanos Zafeiriou +1

Imperial College London

Representation LearningSelf-Supervised Time Series Representation LearningPredictive Representation LearningSurface Electromyography
AskPDF