Dual-Stream Simultaneous Translation via 2D Grid Attention

Yu Pu; Wei-Qiang Zhang

cs.AISep 28, 2026

Dual-Stream Simultaneous Translation via 2D Grid Attention

Authors: Yu Pu, Wei-Qiang Zhang

Organizations: Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China

Abstract

Simultaneous machine translation must generate target tokens before the source input is complete. Existing approaches address this through post-hoc read-write policies, leaving the attention mechanism unaware of bidirectional stream dependencies. We propose a dual-stream attention framework that represents source and target streams as a two-dimensional grid of hidden states and models their interaction through four structurally distinct attention types merged via joint QK Softmax normalization. Two approximations---broadcast and Hadamard---reduce the per-layer complexity from O(X^2Y+XY^2) to O(X^2+Y^2+XY) with provably decaying error. Training uses a self-guided loop: a per-cell loss heatmap drives dynamic-programming path recovery, which generates read/write decision supervision labels without external alignment. An incremental KV cache with anchored rotary position embeddings enables efficient streaming inference. On Chinese-to-English simultaneous translation, the proposed model outperforms the Wait-k baseline by +5.66 BLEURT and +10.36 COMET at comparable latency, and surpasses the non-streaming reference on COMET at a fraction of the response delay.

Figures & tables

Symbol	Meaning	Typical Value
$B$	Batch size	—
$X$	Input sequence length	—
$Y$	Output sequence length	—
$D$	Model hidden dimension	896
$H$	Number of attention heads	14
$H_{k}$	Number of Key/Value heads (GQA)	2

TABLE I: Principal Notation

Fig. 1: Illustration of the 2D grid hidden state representation. Each grid cell $(x,y)$ maintains hidden states for both the input stream $\mathbf{I}^{(l)}_{x,y}$ and the output stream $\mathbf{O}^{(l)}_{x,y}$ . The horizontal axis corresponds to input positions and the vertical axis to output positions.

Type	Query	Key/Value	Causal Constraint
I $\to$ I	$\mathbf{Q}_{I}$	$\mathbf{K}_{I},\mathbf{V}_{I}$	Causal in $x$ ( $x^{\prime}\leq x$ )
O $\to$ O	$\mathbf{Q}_{O}$	$\mathbf{K}_{O},\mathbf{V}_{O}$	Causal in $y$ ( $y^{\prime}\leq y$ )
I $\leftarrow$ O	$\mathbf{Q}_{I}$	$\mathbf{K}_{O},\mathbf{V}_{O}$	Prefix in $y$ ( $y^{\prime}\leq y$ )
O $\leftarrow$ I	$\mathbf{Q}_{O}$	$\mathbf{K}_{I},\mathbf{V}_{I}$	Prefix in $x$ ( $x^{\prime}\leq x$ )

TABLE II: Four Types of Duplex Attention and Their Properties

Fig. 2: Grid illustration of input stream attention. Left: I $\to$ I self-attention, where each cell attends causally along $x$ and results are broadcast along $y$ . Right: I $\leftarrow$ O cross-attention, where each input cell aggregates from the output prefix $y^{\prime}\leq y$ .

Fig. 3: Grid illustration of output stream attention. Left: O $\to$ O self-attention, broadcast along $x$ . Right: O $\leftarrow$ I cross-attention, where each output cell aggregates from the input prefix $x^{\prime}\leq x$ .

Attention Type	Exact	Approximated
I $\to$ I self-attention	$O(X^{2}Y)$	$O(X^{2})$
O $\to$ O self-attention	$O(XY^{2})$	$O(Y^{2})$
I $\leftarrow$ O cross-attention	$O(XY^{2})$	$O(XY)$
O $\leftarrow$ I cross-attention	$O(X^{2}Y)$	$O(XY)$
Total	$O(X^{2}Y+XY^{2})$	$O(X^{2}+Y^{2}+XY)$

TABLE III: Computational Complexity Per Layer Before and After Approximation

Fig. 4: Illustration of the loss heatmap and the optimal DP path. Color intensity indicates the grid loss $\mathcal{L}(x,y)$ (darker = higher loss). The black staircase line is the optimal monotone path: horizontal segments are WAIT steps and vertical segments are EMIT steps.

Cache Entry	Purpose	Update Step
$\mathbf{K}_{I}^{y=0}$	I $\to$ I broadcast keys	WAIT
$\mathbf{V}_{I}^{y=0}$	I $\to$ I broadcast values	WAIT
$\mathbf{K}_{O}^{x=0}$	O $\to$ O broadcast keys	EMIT
$\mathbf{V}_{O}^{x=0}$	O $\to$ O broadcast values	EMIT
$m^{\mathrm{ii}},Z^{\mathrm{ii}},S^{\mathrm{ii}}$	I $\to$ I joint Softmax statistics	WAIT
$m^{\mathrm{oo}},Z^{\mathrm{oo}},S^{\mathrm{oo}}$	O $\to$ O joint Softmax statistics	EMIT

TABLE IV: Per-layer KV cache entries and update rules

Initialize cache with

i_{1:1}

and optional seed output

o_{1:Y_{\mathrm{seed}}}

. Set

x_{\mathrm{vis}}=1

y_{\mathrm{gen}}=Y_{\mathrm{seed}}

While not terminated:

Compute new O-stream column for current

y_{\mathrm{gen}}

using cached I

\leftarrow

O and O

\to

O statistics.

Sample next token

o_{y_{\mathrm{gen}}+1}

If EMIT head indicates WAIT and

x_{\mathrm{vis}}<X_{\mathrm{total}}

, read next input token and update I-stream row cache.

Otherwise continue EMIT.

Return generated output sequence.

TABLE V: Pseudocode for the duplex inference main loop

Fig. 5: Incremental KV cache update. Left: EMIT step appends a new O-stream column and updates cross-stream statistics. Right: WAIT step appends a new I-stream row. Gray regions are cached; colored regions are newly computed.

Hyperparameter	Value
Base model	Qwen2.5-0.5B
LoRA rank / scale	64 / 128
LoRA modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Text decoder	Full fine-tuning
EMIT decision head	Full fine-tuning
Learning rate	$1\times 10^{-4}$

TABLE VI: Dual-stream model training hyperparameters

Source	站在那边的男人曾是棒球手。
Reference	The man standing there was a baseball player.

TABLE VII: Chinese-to-English translation case study

Fig. 6: Grid forward propagation visualization. Horizontal axis: input sequence (left to right). Vertical axis: output sequence (bottom to top). Cell text: highest-probability token. Cell color: EMIT probability intensity.

$\theta$	站在	那边	的男人	曾	是	棒	球	手	。	<eos>
0.3	standing		there, the man	had	been	a	pitcher			.
0.4		the		man standing there had	been	a	pitcher			.
0.5			the	man standing there	was	a		baseball	player	.
0.6				the man standing there	was	a		baseball	player	.
0.7				the man standing there	was		a		baseball player	.
0.8				the	man standing there	was		a	baseball	player .

TABLE VIII: Simultaneous inference under different EMIT thresholds

System	Param	BLEURT $\uparrow$	COMET $\uparrow$	AL $\downarrow$	AP $\downarrow$	FRL $\downarrow$
Base model	—	54.03	63.85	9.39	1.00	18.06
Wait- $k$	$k=3$	43.20	52.06	4.98	0.79	3.00
Wait- $k$	$k=5$	45.20	54.93	5.99	0.84	4.96
Ours	$\theta=0.5$	47.01	59.59	0.23	0.49	2.76
	$\theta=0.6$	48.86	62.42	1.14	0.53	3.31
	$\theta=0.7$	49.10	62.79	1.61	0.55	3.79

TABLE IX: Translation quality and latency on the zh

\to

en test set. The baseline reads the complete input before translating (

\mathrm{AP}=1.00

). Bold indicates the best value in each column.

Fig. 7: BLEURT–AL quality–latency trade-off on the zh $\to$ en test set.

Explore similar work

DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
May 29, 2026Sara Papi, Luisa BentivogliSpeech Translation Large Language Model Decoding
Fondazione Bruno Kessler, Italy
Streaming Translation and Transcription Through Speech-to-Text Causal Alignment
Date pendingRoman Koshkin, Jeon Haesung, Lianbo Liu +4Speech Translation Transcription
SB Intuitions, Tokyo, Japan
SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision
Jul 22, 2026Rongshen He, Xinyu Liang, Dekun Chen +3Speech Translation Speech Language Models
1The Chinese University of Hong Kong, Shenzhen

May 29, 2026, cs.CLJ/K move · Enter open · S save

DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs

Sara Papi, Luisa Bentivogli

Fondazione Bruno Kessler, Italy

Speech Translation Large Language Model Decoding

Open paper AskPDF arXiv

Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit alignment signals. In contrast, Speech Large Language Models (SpeechLLMs) are decoder-only architectures relying solely on self-attention. This raises a central question: whether decoder self-attention contains sufficiently stable alignment signals to guide the streaming policy. Moreover, existing approaches typically rely on training-based adaptations or heuristic wait-

k

policies and have not been validated in long-form settings. To fill these gaps, we propose Decoder-Only Attention (DOA), a training-free policy that enables long-form simultaneous translation with off-the-shelf SpeechLLMs by deriving a proxy alignment from self-attention. Experiments on Phi4-Multimodal and Qwen3-Omni show that DOA provides an effective alignment signal for supporting streaming decisions, enabling low-latency long-form SimulST with quality close to offline decoding without retraining.

Abstract

Explore similar work

DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs

Streaming Translation and Transcription Through Speech-to-Text Causal Alignment

SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision