Simultaneous speech-to-text translation (Simul-S2TT) consists of generating partial translations while the incoming audio frames are processed by the system. However, the streaming nature of this setup creates the challenge of deciding the best moment to perform an accurate translation while minimizing the delay. To address this challenge, we utilize the cross-attention mechanism of the encoder-decoder architecture to find the right alignment between the input speech frames and the target text tokens. In this paper, we propose the Recent Frame Attention Policy (RFAP) and the Dual-Condition Attention Policy (DCAP) that allow offline trained speech-to-text translation models to be used in streaming scenarios without requiring additional training. Results on three different language translation pairs over the CVSS-C corpus show that the RFAP is able to surpass other policies with gains of up to 4.0 BLEU while reducing the translation delay by almost 1 second. Moreover, the DCAP is able to preserve a high translation quality when the latency is very low.
Figures & tables
Figure 1: READ Decision : High recent attention score ( λrecentt=0.55>α,α=0.2 ) indicates the model requires more source context. Recent frames are denoted in red, previous frames with lambda over threshold in blue and previous frames that invoked WRITE decision in black. Future source frames in light gray. Next target token prediction marked with *.
Figure 2: WRITE Decision : Focus shifts to earlier context ( λrecentt=0.15<α,α=0.2 ), signaling sufficient information to emit the target text token ’monte’.
Figure 3: BLEU score vs. Average Lagging (AL) for the (a) FR-EN, (b) DE-EN, and (c) ES-EN translation pairs.
Language Pair
Split
No. Samples
Hours
Avg. SRC Duration (s)
Avg. TGT Tokens Count
FR-EN
Train
207,364
174.0
4.57
11.6
Dev
14,759
13.0
5.28
12.4
Test
14,759
13.3
5.66
12.7
DE-EN
Train
127,822
112.4
5.17
12.3
Dev
13,511
12.5
5.48
13.0
Test
13,504
12.1
5.72
12.7
Table 1: CVSS-C Dataset Statistics
AL range (s)
Policy
< 0
0–0.5
0.5–1
1–1.5
1.5–2
2–2.5
2.5–3
> 3
French-English (FR-EN)
RFAP
–
23.97
26.12
27.73
28.25
–
28.53
29.20
DCAP
–
23.77
24.48
24.96
25.04
–
–
–
EDAtt
–
19.75
22.08
25.22
27.24
28.55
–
29.31
Wait-K
–
–
21.95
24.50
27.17
28.35
28.64
–
Table 2: Best BLEU reached by each policy within a given AL range. – marks a range where policy has no result.
Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit alignment signals. In contrast, Speech Large Language Models (SpeechLLMs) are decoder-only architectures relying solely on self-attention. This raises a central question: whether decoder self-attention contains sufficiently stable alignment signals to guide the streaming policy. Moreover, existing approaches typically rely on training-based adaptations or heuristic wait-k policies and have not been validated in long-form settings. To fill these gaps, we propose Decoder-Only Attention (DOA), a training-free policy that enables long-form simultaneous translation with off-the-shelf SpeechLLMs by deriving a proxy alignment from self-attention. Experiments on Phi4-Multimodal and Qwen3-Omni show that DOA provides an effective alignment signal for supporting streaming decisions, enabling low-latency long-form SimulST with quality close to offline decoding without retraining.
Simultaneous speech-to-speech translation (SimulS2S) is essential for real-time multilingual communication, with increasing integration into meeting and streaming platforms. Despite this, SimulS2S remains underexplored in research, where current solutions often rely on resource-intensive training procedures and operate on short-form, pre-segmented utterances, failing to generalize to continuous speech. To bridge this gap, we propose SimulU, the first training-free policy for long-form SimulS2S. SimulU adopts history management and speech output selection strategies that exploit cross-attention in pre-trained end-to-end models to regulate both input history and output generation. Evaluations on MuST-C across 8 languages show that SimulU achieves a better or comparable quality-latency trade-off against strong cascaded models. By eliminating the need for ad-hoc training, SimulU offers a promising path to end-to-end SimulS2S in realistic, long-form scenarios.
Simultaneous Speech Translation (SimulST) requires balancing high translation quality with low latency. Recent work introduced REINA, a method that trains a Read/Write policy based on estimating the information gain of reading more audio. However, we find that information-based policies often lack temporal context, leading the policy to bias itself toward reading most of the audio before starting to write. We improve REINA using two distinct strategies: a supervised alignment network (REINA-SAN) and a timestep-augmented network (REINA-TAN). Our results demonstrate that while both methods significantly outperform the baseline and resolve stability issues, REINA-TAN provides a slightly superior Pareto frontier for streaming efficiency, whereas REINA-SAN offers more robustness against 'read loops'. Applied to Whisper, both methods improve the pareto frontier of streaming efficiency as measured by Normalized Streaming Efficiency (NoSE) scores up to 7.1% over existing competitive baselines.