Top-down string-to-dependency Neural Machine Translation

Shuhei Kondo; Katsuhito Sudoh; Yuji Matsumoto

cs.CLMar 30, 2026

Top-down string-to-dependency Neural Machine Translation

Authors: Shuhei Kondo, Katsuhito Sudoh, Yuji Matsumoto

Organizations: Unaffiliated · RIKEN Center for Advanced Intelligence Project · Nara Women’s University

Abstract

Most of modern neural machine translation (NMT) models are based on an encoder-decoder framework with an attention mechanism. While they perform well on standard datasets, they can have trouble in translation of long inputs that are rare or unseen during training. Incorporating target syntax is one approach to dealing with such length-related problems. We propose a novel syntactic decoder that generates a target-language dependency tree in a top-down, left-to-right order. Experiments show that the proposed top-down string-to-tree decoding generalizes better than conventional sequence-to-sequence decoding in translating long inputs that are not observed in the training data.

Figures & tables

Figure 1: Basic unit of our model that generates a node and a subtree that belongs to it. Each triangle in the figure represents a subtree.

Figure 2: An example dependency tree generated by the procedure in Fig. 3 . Most empty symbols are omitted for brevity.

	input	output	stack
1	ROOT	VBZ	$L\|W\|R$
2	LEFT::VBZ	IN	$L\|W\|R\|S\|W\|R$
3	LEFT::IN	$\phi_{L}$	$W\|R\|S\|W\|R$
4	WORD:: $\phi_{L}$	Until	$W\|R\|S\|W\|R$
5	WORD::Until	$\phi_{W}$	$R\|S\|W\|R$
6	RIGHT:: $\phi_{W}$	RB	$L\|W\|R\|S\|S\|W\|R$

Figure 3: An example action sequence that generates the tree in Fig. 2 . “::” stands for a concatenation operation.

L,W,S

and

R

stand for LEFT, WORD, SIBLING and RIGHT actions, respectively.

Figure 4: Illustration of the proposed decoder. Dotted lines represent hidden states.

	Topdown	Transformer
dev	35.78	35.76
test	34.17	33.93

Table 1: Results on WMT16 Ro-En dataset.

Num. verbs	Topdown	Transformer
0	39.58 $\pm$ 0.49	40.00 $\pm$ 1.48
1	39.42 $\pm$ 1.02	38.56 $\pm$ 0.66
2	38.76 $\pm$ 0.64	38.62 $\pm$ 0.59
3	37.62 $\pm$ 0.78	37.39 $\pm$ 0.73
4	37.15 $\pm$ 0.57	37.11 $\pm$ 0.72
5	36.16 $\pm$ 0.79	35.94 $\pm$ 0.95

Table 2: Results on Tilde-subset based on number of verbs in the source sentence, averaged over five test sets with different random seeds.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Ensemble Self-Training for Unsupervised Machine Translation
Mar 17, 2026Ido Aharon, Jonathan Shaki, Sarit KrausMachine Translation
Bar-Ilan University
Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation
Apr 20, 2026Abhishek Purushothama, Emma Thronson, Alexia Guo +1In-Context Learning Low-Resource MT
Corpling Lab Georgetown University
Separating Syntax from Language: A Mechanistic Account of Translation in Multilingual LLMs
Sep 1, 2026Mikhail Sonkin, Tanja Baeumel, Daniil Gurgurov +2Multilingual Language Models Mechanistic Interpretability
Saarland University · University of Göttingen · German Research Center for Artificial Intelligence (DFKI) +1

Mar 17, 2026J/K move · Enter open · S save

Ensemble Self-Training for Unsupervised Machine Translation

Ido Aharon, Jonathan Shaki, Sarit Kraus

Bar-Ilan University

Machine Translation

Open paper AskPDF arXiv

We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.

Abstract

Machine Translation

Explore similar work

Ensemble Self-Training for Unsupervised Machine Translation

Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation

Separating Syntax from Language: A Mechanistic Account of Translation in Multilingual LLMs