EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation

Ettore Magni; Rolandos Alexandros Potamias; Stefanos Zafeiriou; Konstantinos Barmpas

cs.LGOct 4, 2026

EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation

Authors: Ettore Magni, Rolandos Alexandros Potamias, Stefanos Zafeiriou, Konstantinos Barmpas

Organizations: Imperial College London

Abstract

Surface electromyography (sEMG) is a low-power, cost-effective biosignal for hand-pose estimation and gesture classification. In this work, we examine whether self-supervised pretraining on sEMG can yield transferable representations for continuous hand-pose estimation. We introduce EMG-GPT, a causal transformer-based model that operates on discrete sEMG representations from a frozen residual vector quantization (RVQ) tokenizer and learns temporal dynamics through depth-autoregressive future-code prediction. The model combines within-frame integration with causal temporal modeling while preserving the geometry of the pretrained codebook. EMG-GPT shows competitive results in both Regression and Tracking tasks, supporting EMG-only pretraining as a viable approach for learning transferable sEMG representations.

Figures & tables

Figure 1: Overview of EMG-GPT. sEMG patches are mapped by frozen NeuroRVQ to structured time–electrode–branch–depth codes. Frozen vector sums are integrated within each frame and modeled causally across frames. The pre-head state either predicts a future frame coarse-to-fine with code-index cross-entropy, or bypasses the categorical head and transfers to parallel Regression and Tracking decoders under frozen or full-backbone adaptation. Regression interpolates features to the $50$ -Hz pose grid without an initial pose, whereas Tracking repeats features and receives the boundary pose. Only token-frame modeling is causal.

Figure 2: Tracking poses for EMG-GPT and vEMG2Pose. Top: motion capture; middle: EMG-GPT Tracking; bottom: vEMG2Pose ( Salter et al., 2024 ) .

Test set	Model	Angular MAE ( ∘ ) $\downarrow$	Landmark (mm) $\downarrow$
User	SensingDynamics	15.5 $\pm$ 1.4	21.8 $\pm$ 2.1
	NeuroPose	13.2 $\pm$ 1.1	17.5 $\pm$ 1.3
	vEMG2Pose	12.2 $\pm$ 1.3	15.8 $\pm$ 1.9
	EMG-GPT	12.8 $\pm$ 1.1	16.3 $\pm$ 1.3
Stage	SensingDynamics	18.8 $\pm$ 1.6	26.6 $\pm$ 2.0
	NeuroPose	17.2 $\pm$ 1.7	24.0 $\pm$ 2.1

Table 1: Regression test results. Published baselines are from Salter et al. (2024) ; EMG-GPT is a fully adapted model. Angular and landmark values are mean

\pm

sample SD across users within each generalization condition.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Verified value
Token grid	25 Hz; 40-ms hop; independently encoded 200-ms patches
RVQ depth	$K=4$ retained depths
Forecast	$L=150$ frames (6.0 s context); $\Delta=5$ frames (200 ms)
Offline preprocessing	Nominal 20–400 Hz third-order Butterworth with forward–reverse filtering; polyphase resampling from 2 kHz to 1 kHz
Input representation	Frozen $K=4$ RVQ vectors summed across depth, LayerNorm, bias-free projection, learned channel/branch embeddings, CLS pooling
Temporal backbone	$d_{\mathrm{model}}=512$ ; 8 Transformer layers; 8 heads; dropout 0.1; 2-layer within-frame encoder

Appendix

Table 2: Configuration of the selected 25-Hz,

K=4

EMG-GPT model.

Test set	Model	Angular MAE ( ∘ ) $\downarrow$	Landmark (mm) $\downarrow$
User	vEMG2Pose	7.7 $\pm$ 1.0	10.3 $\pm$ 1.5
	EMG-GPT	8.9 $\pm$ 0.9	11.6 $\pm$ 1.2
Stage	vEMG2Pose	11.2 $\pm$ 1.4	15.2 $\pm$ 1.9
	EMG-GPT	12.4 $\pm$ 1.4	16.9 $\pm$ 1.8
User, Stage	vEMG2Pose	11.0 $\pm$ 1.0	15.4 $\pm$ 1.4
	EMG-GPT	12.2 $\pm$ 1.3	16.9 $\pm$ 1.4

Appendix

Table 3: Tracking test results, where the boundary pose is given. vEMG2Pose values are from ( Salter et al., 2024 ) .

Test set	Model	Angular MAE ( ∘ ) $\downarrow$	Landmark (mm) $\downarrow$
Regression
User	vEMG2Pose	12.80 $\pm$ 1.31	16.29 $\pm$ 1.85
	EMG-GPT	12.80 $\pm$ 1.08	16.30 $\pm$ 1.29
Stage	vEMG2Pose	15.40 $\pm$ 1.59	20.43 $\pm$ 2.09
	EMG-GPT	16.30 $\pm$ 1.57	22.02 $\pm$ 1.92
User, Stage	vEMG2Pose	15.73 $\pm$ 1.38	21.25 $\pm$ 1.79

Appendix

Table 4: Locally matched test evaluation of the released vEMG2Pose checkpoints and EMG-GPT. Values are mean

\pm

sample SD across users within each generalization condition. Bold indicates the lower mean; rounded ties are bolded together. Each method retains its native input frontend.

Figure 3: Median held-out user and median held-out stage ( FingerFreeform ). Top: motion capture; middle: EMG-GPT Tracking; bottom: vEMG2Pose. Six poses unroll from left to right over a five-second validation clip.

Figure 4: Median held-out user and lower-error held-out stage ( ShakaVulcanPeace ). Rows and temporal layout follow Figure 3 .

Figure 5: Median held-out user and higher-error held-out stage ( CountingUpDownFingerWigglingSpreading ). Rows and temporal layout follow Figure 3 .

Figure 6: Lower-error held-out user and median held-out stage ( HandOverHandCountingUpDownFingerWigglingSpreading ). Rows and temporal layout follow Figure 3 .

Figure 7: Higher-error held-out user and median held-out stage ( FingerFreeform ). Rows and temporal layout follow Figure 3 .

Figure 8: Regression example for a median held-out user and median held-out stage. Top: motion-capture target; middle: EMG-GPT Regression; bottom: vEMG2Pose Regression. Six poses unroll from left to right over the same target-aligned five-second validation clip.

Figure 9: Regression example for a median held-out user and lower-error held-out stage. Rows and target-aligned temporal layout follow Figure 8 .

Figure 10: Regression example for a median held-out user and higher-error held-out stage. Rows and target-aligned temporal layout follow Figure 8 .

Explore similar work

EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding
Sep 22, 2026Yuwei Jia, Cheng Zhong, Jinyang Yu +1Surface Electromyography Electrocardiogram Foundation Models
Beijing University of Posts and Telecommunications, China · Dexwise, China · Shenzhen University, China
From Muscle Bursts to Motor Intent: Self-Supervised Token Modeling for Heterogeneous EMG
May 5, 2026Zhenghao Huang, Huilin Yao, Kaikai WangSurface Electromyography Musculoskeletal
South China University of Technology
EFormer: Temporally Aligned Local Correction for Continuous sEMG-Based Hand Pose Tracking
Sep 30, 2026JiaCheng Ge, SiYu ZhangSurface Electromyography 3D Hand

Sep 22, 2026, cs.LGJ/K move · Enter open · S save

EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding

Yuwei Jia, Cheng Zhong, Jinyang Yu, Zhe Cui

Beijing University of Posts and Telecommunications, China · Dexwise, China · Shenzhen University, China

Surface Electromyography Electrocardiogram Foundation Models Ground Reaction Forces Force Decoding

Open paper AskPDF arXiv

Public surface electromyography (EMG) datasets vary widely in electrode layout, channel count, frequency support, and size. Simply mixing them for pretraining can misalign channel semantics, introduce spectral targets that some devices cannot observe, and let large or high-channel-count datasets dominate learning. We introduce EMGBlend, a self-supervised framework designed around these differences. It combines shared channel patches with geometry-aware attention, restricts spectral targets to each recording's supported frequency band, and balances exposure across data sources. We pretrain a 109M-parameter model on 11 public EMG sources and evaluate it on gesture recognition, continuous-force regression, and contact classification. EMGBlend consistently outperforms matched random initialization and waveform reconstruction controls. Fixed-budget source controls show that multi-source pretraining improves gesture recognition and remains competitive for force decoding. Ablations confirm that geometry, band-aware targets, and source balancing each contribute to transfer, although cross-person NinaPro force estimation remains difficult. Overall, EMGBlend shows how heterogeneous EMG datasets can be combined through explicit mechanism design rather than simple concatenation. Code is available at https://github.com/tamanano/EMGBlend

Abstract

Explore similar work

EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding

From Muscle Bursts to Motor Intent: Self-Supervised Token Modeling for Heterogeneous EMG

EFormer: Temporally Aligned Local Correction for Continuous sEMG-Based Hand Pose Tracking