Organizations: School of Informatics, Xiamen University, China · MiLM Plus, Xiaomi Inc., Beijing, China · The University of Hong Kong, Hong Kong, China
Large language models (LLMs) have recently demonstrated promising performance in simultaneous machine translation (SimulMT). However, applying decoder-only LLMs to SimulMT introduces a positional mismatch, which leads to a dilemma between decoding efficiency and positional consistency. Existing approaches often rely on specific positional encodings or carefully designed prompting schemes, and thus fail to simultaneously achieve inference efficiency, positional consistency, and broad model compatibility. In this work, we propose ExPosST, a general framework that resolves this dilemma through explicit position allocation. ExPosST reserves fixed positional slots for incoming source tokens, enabling efficient decoding with KV cache across different positional encoding methods. To further bridge the gap between fine-tuning and inference, we introduce a policy-consistent fine-tuning strategy that aligns training with inference-time decoding behavior. Experiments across multiple language pairs demonstrate that ExPosST effectively supports simultaneous translation under diverse policies.
Figures & tables
Figure 1: Comparison of different strategies for LLM-based SimulMT. ⟨u⟩ and ⟨a⟩ denote the user and assistant role prompts in the LLM conversational template. (a) Cache Reuse : Inserting a new source token ( s3 ) shifts the positional indices of subsequent tokens (e.g., t1 shifts from 3 to 4), causing a mismatch with the existing KV cache. (b) Cache Recomputation : Positional consistency is restored by re-encoding the shifted tokens, but this incurs prohibitive computational latency. (c) ExPosST (Ours) : By explicitly pre-allocating positional slots for potential source tokens, the positional indices of the target sequence remain invariant during READ/WRITE cycles, enabling efficient KV cache reuse without positional misalignment.
Figure 2: Overview of the ExPosST framework. Left: The Inference with Pre-allocated Positions strategy. We adopt a wait-2 policy and set the pre-allocated slot length Lslot=3 . When s3 is read, it fills the reserved slot without shifting the positions of generated target tokens (e.g., t1 ). Upon reading s4 , a new source slot is allocated immediately after the current output to maintain positional consistency. Right: The Policy-Consistent Fine-tuning strategy. The source sentence is segmented into parts to match the inference slot. A policy-consistent attention mask (bottom right) is applied to ensure the visibility of source tokens aligns with the specific simultaneous policy.
Figure 3: Sensitivity analysis of the pre-allocated slot length Lslot on the IWSLT 2017 En-De dev set, illustrating its impact on translation performance (BLEU score) and average training sequence length for the Llama-3.1-8B-Instruct model.
Figure 4: Main results on IWSLT 2017 tasks on Llama-3.1-8B-Instruct . The figures illustrate the BLEU-LAAL trade-off curves, comparing ExPosST with various baselines across two mainstream LLM architectures. The dashed horizontal lines indicate the performance of the corresponding offline models. Higher curves and those shifted toward the top-left represent a superior quality-latency balance.
Figure 5: BLEU-LAAL trade-off on IWSLT 2017 datasets on Qwen2.5-7B-Instruct .
Figure 6: The BLEU vs LAAL result in falcon-rw-1b between SimulMask and ExPosST on En-De.
Figure 7: Ablation study of ExPosST components on the En-De task using Llama-3.1-8B-Instruct . The curves represent the quality-latency trade-off for the full model versus variants where masking or slot allocation is removed.
Lslot
wait-1
wait-3
wait-5
wait-7
4
12.94
19.81
13.31
13.23
8
16.84
25.92
29.18
26.81
16
18.16
26.97
30.94
32.15
32
17.82
27.11
31.08
32.39
64
15.94
26.79
31.07
32.46
Table 1: Sensitivity analysis of the mismatch between training and inference slot lengths ( Lslot ) on the IWSLT 2017 En-De dataset using Llama-3.1-8B-Instruct . The model is fine-tuned with a fixed training slot length of Lslot=16 and evaluated under various wait-k policies with different inference slot lengths.
Figure 8: Box plots of cumulative computational cost (GFLOPs) during inference on the IWSLT 2017 En-De test set using Llama-3.1-8B-Instruct . We report results under the wait-3 and read-5 policies.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Group 1
Weight Precision
bfloat16
Optimizer
AdamW
Learning Rate
2⋅10−4
LR Scheduler
Inverse Sqrt
Weight Decay
0.1
Warmup Steps
500
Appendix
Table 2: Fine-tuning hyperparameters for all models.
Figure 9: COMET results on IWSLT 2017 tasks on Llama-3.1-8B-Instruct . The curves demonstrate the trade-off between translation quality (COMET) and latency (LAAL) for ExPosST against various baselines.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
Simultaneous speech translation (SimulST) produces translations incrementally while processing partial speech input. Although large language models (LLMs) have shown strong capabilities in offline translation tasks, applying them to SimulST poses notable challenges. Existing LLM-based SimulST approaches either incur significant computational overhead due to repeated encoding of bidirectional speech encoder, or they depend on a fixed read/write policy, limiting the efficiency and performance. In this work, we introduce Efficient and Adaptive Simultaneous Speech Translation (EASiST) with fully unidirectional architecture, including both speech encoder and LLM. EASiST includes a multi-latency data curation strategy to generate semantically aligned SimulST training samples and redefines SimulST as an interleaved generation task with explicit read/write tokens. To facilitate adaptive inference, we incorporate a lightweight policy head that dynamically predicts read/write actions. Additionally, we employ a multi-stage training strategy to align speech-text modalities and optimize both translation and policy behavior. Experiments on both in-domain (MuST-C) and out-of-domain (Europarl-ST) En-De and En-Es datasets demonstrate that EASiST offers superior latency-quality trade-offs compared to several strong baselines.
Biao Fu, Donglei Yu, Minpeng Liao +5
School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · Alibaba Group Tongyi Lab
Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit alignment signals. In contrast, Speech Large Language Models (SpeechLLMs) are decoder-only architectures relying solely on self-attention. This raises a central question: whether decoder self-attention contains sufficiently stable alignment signals to guide the streaming policy. Moreover, existing approaches typically rely on training-based adaptations or heuristic wait-k policies and have not been validated in long-form settings. To fill these gaps, we propose Decoder-Only Attention (DOA), a training-free policy that enables long-form simultaneous translation with off-the-shelf SpeechLLMs by deriving a proxy alignment from self-attention. Experiments on Phi4-Multimodal and Qwen3-Omni show that DOA provides an effective alignment signal for supporting streaming decisions, enabling low-latency long-form SimulST with quality close to offline decoding without retraining.
Simultaneous speech translation (SimulST) requires incremental translation under strict latency constraints, yet remains challenging for decoder-only LLM systems due to limited context and cross-lingual reordering. Recent approaches often introduce architectural changes or explicit read/write policies to control output timing, which can be brittle in conversational speech where segmentation boundaries are ambiguous. We present a simple data-driven alternative: fixed-length chunks for cumulative streaming decoding with a rewind-based committed prefix, and teacher-labeled prefix-to-prefix (P2P) targets with bounded waiting for fine-tuning, yielding CSSEL-P2P, where CSSEL is our proposed chunked streaming speech encoder LLM. In our in-house conversational speech evaluation, CSSEL-P2P improves streaming quality by +1.54 COMETKiwi over the CSSEL streaming baseline at comparable latency (+0.15s Average Lagging), suggesting effective SimulST without architectural changes via P2P supervision.