Dual-Stage Channel-Posterior Integration for MLM-Aided Token Communications
Organizations: School of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea · Department of Smart Mobility Engineering, Inha University, Incheon, South Korea · School of Electrical and Mechanical Engineering, Adelaide University, Adelaide, Australia
Abstract
Token transmission over conventional digital modulation is vulnerable to channel impairments, leading to bit errors that cause received tokens to become contextually inconsistent. This paper proposes a receiver-side dual-stage channel-posterior integration framework for masked language model (MLM)-aided token reconstruction. In the proposed framework, the receiver detects unreliable token positions from channel reliability and constructs a vocabulary-level channel-posterior. In particular, within the receiver, Posterior-Weighted Input Embedding (PWE) injects this posterior into the MLM input, while Adaptive Output Logit Fusion (AOF) reuses it at the output with a channel-adaptive weight. Consequently, multiple unreliable tokens can be reconstructed in a single bidirectional MLM inference without retransmission, transmitter-side encoding, or additional MLM training. Experiments over noisy channels show up to a % improvement in cosine similarity over context-only MLM reconstruction, with consistent gains across different MLM architectures and existing language-model-aided methods.
Masked Language Modeling
Masked language modeling trains text encoders to infer hidden tokens, producing contextual representations for language tasks and specialized domains.
See all 49 papers