Typing Matters: Muscle, Language and Typing Behaviour in Live Keystroke Decoding from sEMG
Organizations: CRCV, University of Central Florida · Department of MAE, University of Central Florida
Abstract
Surface electromyography (sEMG) from the wrists can recover typed text without a keyboard. It is usually posed as a signal-inference problem: characters are read out of the muscle signal and character error rate is reported offline. We argue that this framing removes most of the problem. The decoder is constrained simultaneously by the muscles, by the language being typed, and by typing behaviour such as typos, corrections, backspaces and line breaks, and a usable system must predict keys live while the hand keeps moving. Existing systems fall short on two counts: (1) they process sEMG in fixed windows and carry nothing from one window to the next, discarding context accumulated over a session, and (2) they lack any understanding of the linguistic structure that drives typing, so every keystroke is inferred from the muscle signal alone. We propose a fully causal, zero-lookahead sEMG typing decoder built on a state space model (SSM), in which a single model decodes a whole session without windowed partitions, conditioned on what has already been typed. We achieve this by fusing a traditional typing decoder with an efficient autoregressive language model, instilling both sEMG and linguistic understanding in one model. To train such a model for per-character typing rather than whole-word text, we introduce a retokenization method that rewrites ordinary prose as human-like typing, with keystrokes, typos, backspaces and line breaks included, converting WikiText into a large typing corpus where none existed. Our model, having never seen a single sEMG-to-character transcript in typing, achieves a state-of-the-art character error rate of 25.13% on emg2qwerty's cross-user benchmark, a 5.4% relative improvement over the previous best with 6.7 times fewer parameters, while running strictly causally with no future context.
Figures & tables
| System | Encoder | Params | CER ( ) |
| emg2qwerty, greedy ( Sivakumar et al., 2024 ) | TDS conv | 5.29M | |
| emg2qwerty, beam + char. 6-gram | TDS conv | 5.29M | |
| splashnet-mini ( Hadidi et al., 2026 ) | conv | 1.38M | |
| splashnet | conv | 2.58M | |
| Mehlman et al. Small ( Mehlman et al., 2025 ) | Transformer | 5.4M | |
| Mehlman et al. Tiny, distilled from Large | Transformer | 2.2M |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Session length | Ours | Transformer |
|---|---|---|
| 30 s | 0.79 | 46 |
| 1 min | 0.79 | 92 |
| 2 min | 0.79 | 184 |
| 5 min | 0.79 | 461 |
| 10 min | 0.79 | 922 |
| 20 min | 0.79 | 1843 |
| Decoder started from | CER | |
|---|---|---|
| cold, as shipped | 25.24 | |
| a replay of this session’s first 4 min | 25.30 | |
| another session of the same user | 26.07 | |
| another user’s session | 28.90 |
| Replacement after the cut | Cells | Argmax agreement | |
|---|---|---|---|
| session truncated there | 28 | 1.000000 | 0.0 |
| noise | 28 | 1.000000 | 0.0 |
| the same frames, shuffled | 28 | 1.000000 | 0.0 |
| reset at the boundary | every 4 s | every 32 s |
|---|---|---|
| nothing reset (CER) | ||
| recurrent state only | ||
| causal normaliser only | ||
| both |
| Internal | What it does | Cost |
|---|---|---|
| step size | how fast the state moves | it stops emitting |
| decay | how long the state remembers | |
| rotary clock | the state’s phase | |
| trapezoid split | how the step is discretised |
| Training text | 8-gram | 16-gram | Lines verbatim |
|---|---|---|---|
| WikiText-103, prose prompts | 1.13 | 0.38 | 0.00 |
| WikiText-103, word-list prompts | 0.00 | 0.00 | 0.00 |
| the benchmark’s own transcripts | 6.80 | 3.57 | 2.00 |
| Component | Statistic it accounts for | Real | Ours | Without it |
|---|---|---|---|---|
| per-typist layer | spread of error rate across typists | 6.38 | 3.77 | 1.65 |
| count frailty | spread of error rate across typists | 6.38 | 3.77 | 3.57 |
| real-word slips | typos landing on another real word | 0.01329 | 0.01315 | 0.00799 |
| burst process | errors never corrected, per 1,000 | 4.849 | 4.871 | 5.017 |
| correction process | errors never corrected, per 1,000 | 4.849 | 4.871 | 0.000 |