Simultaneous speech translation (SimulST) aims to translate ongoing speech with low latency while preserving translation quality. Recent studies have adopted large language models (LLMs) for SimulST and formulated it as a multi-turn dialogue task for efficient generation, achieving state-of-the-art performance. However, existing dialogue-style SimulST approaches typically rely on specialized training schemes to acquire simultaneous translation behavior, either using synthesized read/write trajectories, which may provide suboptimal training signals, or jointly optimizing translation quality and latency, whose competing objectives may cause optimization instability. In this paper, we introduce SimPOTrans, a reinforcement learning (RL) framework that enables offline speech translation models to self-adapt to dialogue-style SimulST. Through a simple two-stage RL post-training pipeline, SimPOTrans rewards an offline model for exhibiting translation behaviors beneficial to simultaneous scenarios when given partial speech inputs, thereby enabling offline-to-simultaneous self-adaptation without synthesized read/write trajectories or explicit latency optimization. At inference time, SimPOTrans requires no external decision policy and enables flexible quality--latency control by simply adjusting the chunk size of speech input. Experiments on the ACL 60/60 and MCIF benchmarks across three language directions (En→De, En→Zh, and En→Ja) demonstrate that SimPOTrans outperforms several strong baselines, achieving a superior quality--latency trade-off.
Figures & tables
Figure 1: Overview of the SimPOTrans training pipeline. (1) Stage 0: Offline Translation Pretraining (SFT). SFT establishes the offline translation capability of the LLM-based speech translation model. (2) Stage 1: Offline Translation Optimization (RL). RL-Stage I uses GRPO with a translation-quality reward to further enhance offline translation quality on complete utterances. (3) Stage 2: Offline-to-Simultaneous Translation Adaptation (RL). RL-Stage II exposes the offline-trained model to partial speech inputs in a multi-turn setting and optimizes it with quality and monotonicity rewards together with a repetition penalty, enabling the model to self-adapt to dialogue-style SimulST.
Figure 2: Comparison of quality–latency trade-offs on the ACL 60/60 benchmark for En → De, En → Zh, and En → Ja. Rows report COMET, MetricX-24, and BLEU, respectively, with latency measured by LongYAAL. MetricX-24 scores are negated so that higher values consistently indicate better translation quality across all metrics. The SimPOTrans curves are obtained using chunk sizes of {1.0,1.5,2.0,3.0,4.0,5.0} seconds.
Lang.
System
COMET
LongYAAL
PDF
En → De
NeMo *
0.93
2.0
√
CUHKSZ
0.85
1.5
√
PINCH-AST *
0.84
1.8
×
CUNI-POCKET
0.77
1.7
×
Ours
0.87
1.6
×
En → Zh
CUHKSZ
0.74
1.5
√
Table 1: Comparison with IWSLT 2026 Simultaneous Track systems on the MCIF development set in the low-latency regime (≤2s) . LongYAAL is reported in seconds. Asterisks indicate cascaded systems. PDF indicates whether PDF-derived extra context is used.
Figure 3: Ablation studies. (a) Comparison of quality–latency trade-offs across different training stages. (b) Effects of the monotonicity reward (Mono.) and repetition penalty (Rep.) in RL-Stage II. (c) Effect of the speech chunk size in RL-Stage II.
Figure 4: Monotonicity across SimPOTrans training stages under simultaneous inference with a fixed chunk size of 1.5 seconds.
Figure 5: An En → Zh example comparing translation behaviors before and after offline-to-simultaneous self-adaptation. Colored spans and connecting lines indicate source–target alignments. The offline model largely follows the reference word order, which involves substantial reordering relative to the source. In contrast, SimPOTrans conveys the core meaning in an order closer to the source, producing a more monotonic translation better suited to simultaneous generation.
School of Informatics, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · Alibaba Group Tongyi Lab