cs.SDOct 20, 2026

PhonoBridge: Error-Aware Phoneme-Mediated EMG-to-Text Reconstruction

Authors: Wen Qiu, Jiacheng Zhang, Xuanda Chen, Siqi Cai, Haizhou Li

Organizations: School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, Shenzhen, China · Harbin Institute of Technology, Shenzhen, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Electromyography (EMG) provides a non-invasive means of decoding silently articulated speech into text, but accurate decoding remains challenging due to limited availability of paired EMG-text data. We introduce PhonoBridge, a two-stage framework comprising a channel-aware EMG-to-phoneme frontend and a large language model (LLM) backend for text reconstruction. The frontend is trained on paired EMG-text data that consists of channel-wise feature extraction, cross-channel fusion, and electrode embeddings. With phonemes as the intermediate representation, we expand the backend's training data by converting LibriSpeech transcripts into phoneme-text pairs without additional EMG recordings. To adapt the LLM to frontend noise, we introduce Error-Profile-Guided Augmentation (EPGA), which perturbs transcript-derived phoneme sequences according to error patterns estimated from frontend predictions, generating additional training pairs for the LLM. On the Open Vocabulary silent-speech dataset, PhonoBridge achieves 24.77% WER, outperforming the previously reported best of 28.27% on the same test split.

Figures & tables

Figure 1: Electrode placement and qualitative comparison of decoding errors. Left: Electrode placement for EMG acquisition. Right: A comparison of EMG-decoded text and human transcription of the corresponding voiced utterance against the reference text.
Figure 2: Overall architecture of PhonoBridge. The framework comprises two stages: (a) a channel-aware frontend that maps multichannel EMG to phoneme sequences and is trained with CTC loss; and (b) an LLM backend that reconstructs text from the predicted phoneme sequence and is fine-tuned on clean and EPGA-augmented phoneme–text pairs.
MethodWER (%)
Gaddy and Klein [ 6 ]28.27
EMG2Vec [ 16 ]29.45
EMG-LLM [ 17 ]108.29
CNN-Transformer51.29
CNN-BiLSTM45.31
PhonoBridge (Ours)24.77
Table 1: End-to-end WER (%) on the Open Vocabulary test split. All systems are evaluated on the same split.
MethodChannel-wiseElec. emb.PER (%)cWER (%)
Gaddy and Klein [ 6 ]✗✗14.1236.35
PhonoBridge✗✗33.4367.35
PhonoBridge✗✓32.3963.76
PhonoBridge✓✗14.1737.62
PhonoBridge✓✓13.6535.53
Table 2: Channel-aware encoder ablation on the test split. Results are reported in PER and collapse WER (cWER), with Gaddy and Klein included for reference.
Adaptation settingWER (%)CER (%)
Qwen3-8B (zero-shot)188.22116.77
Clean only66.5334.53
EPGA-augmented only29.4117.23
Mixed, single phase25.8922.39
Clean →\rightarrow mixed (ours)24.7714.58
Table 3: Ablation of LLM backend adaptation on the test set. Mixed training combines EPGA-augmented pairs with replayed clean pairs, and →\rightarrow denotes sequential adaptation.
More in this topic

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Explore similar work

CardsList

  1. Bypassing Direct Reconstruction: Speech Detection from MEG via Large-Scale Audio Retrieval

    May 13, 2026Boda Xiao, Bo Wang, Heping ChengNeural DecodingNeuroimaging

    Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China · Speech and Hearing Research Center, School of Intelligence Science and Technology, Peking University, Beijing, China · National Biomedical Imaging Center, State Key Laboratory of Membrane Biology, Institute of Molecular Medicine, Peking-Tsinghua Center for Life Sciences, College of Future Technology, Peking University, Beijing, China

  2. Brain2Speech-Net: Fast and Intelligible Brain-to-Speech Synthesis Without Text Decoding

    Sep 3, 2026Shreeram Suresh Chandra, Zexin Cai, Yu Tsao +2Brain-Computer InterfacesNeural Decoding

    Johns Hopkins University, USA · Academia Sinica, Taiwan · University of Edinburgh, UK

  3. One Mean Log-Mel Vector per Phoneme: Simple and Fast Text Injection for Speech Recognition

    Oct 25, 2026Albert Zeyer, Davyd Naveriani, Dorian Koch +2Automatic Speech RecognitionSynthetic Data Augmentation for Speech Recognition

    Machine Learning and Human Language Technology Group, RWTH Aachen University, Germany · AppTek GmbH, Aachen, Germany