ExPosST: Explicit Positioning with Adaptive Masking for LLM-Based Simultaneous Machine Translation

Yuzhe Shang; Pengzhi Gao; Yazheng Yang; Jiayao Ma; Wei Liu; Jian Luan; Jinsong Su

cs.CLMar 16, 2026

ExPosST: Explicit Positioning with Adaptive Masking for LLM-Based Simultaneous Machine Translation

Authors: Yuzhe Shang, Pengzhi Gao, Yazheng Yang, Jiayao Ma, Wei Liu, Jian Luan, Jinsong Su

Organizations: School of Informatics, Xiamen University, China · MiLM Plus, Xiaomi Inc., Beijing, China · The University of Hong Kong, Hong Kong, China

Abstract

Large language models (LLMs) have recently demonstrated promising performance in simultaneous machine translation (SimulMT). However, applying decoder-only LLMs to SimulMT introduces a positional mismatch, which leads to a dilemma between decoding efficiency and positional consistency. Existing approaches often rely on specific positional encodings or carefully designed prompting schemes, and thus fail to simultaneously achieve inference efficiency, positional consistency, and broad model compatibility. In this work, we propose ExPosST, a general framework that resolves this dilemma through explicit position allocation. ExPosST reserves fixed positional slots for incoming source tokens, enabling efficient decoding with KV cache across different positional encoding methods. To further bridge the gap between fine-tuning and inference, we introduce a policy-consistent fine-tuning strategy that aligns training with inference-time decoding behavior. Experiments across multiple language pairs demonstrate that ExPosST effectively supports simultaneous translation under diverse policies.

Figures & tables

Figure 1: Comparison of different strategies for LLM-based SimulMT. $\langle u\rangle$ and $\langle a\rangle$ denote the user and assistant role prompts in the LLM conversational template. (a) Cache Reuse : Inserting a new source token ( $s_{3}$ ) shifts the positional indices of subsequent tokens (e.g., $t_{1}$ shifts from 3 to 4), causing a mismatch with the existing KV cache. (b) Cache Recomputation : Positional consistency is restored by re-encoding the shifted tokens, but this incurs prohibitive computational latency. (c) ExPosST (Ours) : By explicitly pre-allocating positional slots for potential source tokens, the positional indices of the target sequence remain invariant during READ/WRITE cycles, enabling efficient KV cache reuse without positional misalignment.

Figure 2: Overview of the ExPosST framework. Left: The Inference with Pre-allocated Positions strategy. We adopt a wait-2 policy and set the pre-allocated slot length $L_{slot}=3$ . When $s_{3}$ is read, it fills the reserved slot without shifting the positions of generated target tokens (e.g., $t_{1}$ ). Upon reading $s_{4}$ , a new source slot is allocated immediately after the current output to maintain positional consistency. Right: The Policy-Consistent Fine-tuning strategy. The source sentence is segmented into parts to match the inference slot. A policy-consistent attention mask (bottom right) is applied to ensure the visibility of source tokens aligns with the specific simultaneous policy.

Figure 3: Sensitivity analysis of the pre-allocated slot length

L_{slot}

on the IWSLT 2017 En-De dev set, illustrating its impact on translation performance (BLEU score) and average training sequence length for the Llama-3.1-8B-Instruct model.

Panel of Figure 4 — Figure 4: Main results on IWSLT 2017 tasks on Llama-3.1-8B-Instruct . The figures illustrate the BLEU-LAAL trade-off curves, comparing ExPosST with various baselines across two mainstream LLM architectures. The dashed horizontal lines indicate the performance of the corresponding offline models. Higher curves and those shifted toward the top-left represent a superior quality-latency balance.

Panel of Figure 5 — Figure 5: BLEU-LAAL trade-off on IWSLT 2017 datasets on Qwen2.5-7B-Instruct .

Figure 6: The BLEU vs LAAL result in falcon-rw-1b between SimulMask and ExPosST on En-De.

Figure 7: Ablation study of ExPosST components on the En-De task using Llama-3.1-8B-Instruct . The curves represent the quality-latency trade-off for the full model versus variants where masking or slot allocation is removed.

$L_{slot}$	wait-1	wait-3	wait-5	wait-7
4	12.94	19.81	13.31	13.23
8	16.84	25.92	29.18	26.81
16	18.16	26.97	30.94	32.15
32	17.82	27.11	31.08	32.39
64	15.94	26.79	31.07	32.46

Table 1: Sensitivity analysis of the mismatch between training and inference slot lengths (

L_{slot}

) on the IWSLT 2017 En-De dataset using Llama-3.1-8B-Instruct . The model is fine-tuned with a fixed training slot length of

L_{slot}=16

and evaluated under various wait-k policies with different inference slot lengths.

Figure 8: Box plots of cumulative computational cost (GFLOPs) during inference on the IWSLT 2017 En-De test set using Llama-3.1-8B-Instruct . We report results under the wait-3 and read-5 policies.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Group 1
Weight Precision	bfloat16
Optimizer	AdamW
Learning Rate	$2\cdot 10^{-4}$
LR Scheduler	Inverse Sqrt
Weight Decay	0.1
Warmup Steps	500

Appendix

Table 2: Fine-tuning hyperparameters for all models.

Panel of Figure 9 — Figure 9: COMET results on IWSLT 2017 tasks on Llama-3.1-8B-Instruct . The curves demonstrate the trade-off between translation quality (COMET) and latency (LAAL) for ExPosST against various baselines.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture
Apr 16, 2025Biao Fu, Donglei Yu, Minpeng Liao +5Speech Translation Simultaneous Speech Translation
School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · Alibaba Group Tongyi Lab
DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
May 29, 2026Sara Papi, Luisa BentivogliSpeech Translation Decoder-Only Language Models
Fondazione Bruno Kessler, Italy
Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach
Jul 14, 2026Junkun Chen, Jian Xue, Ming Tang +4Speech Translation Decoder-Only Language Models
Microsoft, USA

Apr 16, 2025J/K move · Enter open · S save

Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture

Biao Fu, Donglei Yu, Minpeng Liao, Chengxi Li+4

School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · Alibaba Group Tongyi Lab

Speech Translation Simultaneous Speech Translation Speech Language Models

Open paper AskPDF arXiv

Simultaneous speech translation (SimulST) produces translations incrementally while processing partial speech input. Although large language models (LLMs) have shown strong capabilities in offline translation tasks, applying them to SimulST poses notable challenges. Existing LLM-based SimulST approaches either incur significant computational overhead due to repeated encoding of bidirectional speech encoder, or they depend on a fixed read/write policy, limiting the efficiency and performance. In this work, we introduce Efficient and Adaptive Simultaneous Speech Translation (EASiST) with fully unidirectional architecture, including both speech encoder and LLM. EASiST includes a multi-latency data curation strategy to generate semantically aligned SimulST training samples and redefines SimulST as an interleaved generation task with explicit read/write tokens. To facilitate adaptive inference, we incorporate a lightweight policy head that dynamically predicts read/write actions. Additionally, we employ a multi-stage training strategy to align speech-text modalities and optimize both translation and policy behavior. Experiments on both in-domain (MuST-C) and out-of-domain (Europarl-ST) En-De and En-Es datasets demonstrate that EASiST offers superior latency-quality trade-offs compared to several strong baselines.

Abstract

Machine Translation

Explore similar work

Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture

DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs

Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach