Just Use XML: Revisiting Joint Translation and Label Projection

Thennal DK; Chris Biemann; Hans Ole Hatzel

cs.CLMar 12, 2026

Just Use XML: Revisiting Joint Translation and Label Projection

Authors: Thennal DK, Chris Biemann, Hans Ole Hatzel

Organizations: Language Technology Group, University of Hamburg

Abstract

Label projection is an effective technique for cross-lingual transfer, extending span-annotated datasets from a high-resource language to low-resource ones. Most approaches perform label projection as a separate step after machine translation, and prior work that combines the two reports degraded translation quality. We re-evaluate this claim with LabelPigeon, a novel framework that jointly performs translation and label projection via XML tags. We design a direct evaluation scheme for label projection, and find that LabelPigeon outperforms baselines and actively improves translation quality in 11 languages. We further assess translation quality across 203 languages and varying annotation complexity, finding consistent improvement attributed to additional fine-tuning. Finally, across 27 languages and three downstream tasks, we report substantial gains in cross-lingual transfer over comparable work, up to +40.2 F1 on NER. Overall, our results demonstrate that XML-tagged label projection provides effective and efficient label transfer without compromising translation quality.

Figures & tables

Figure 1: An example taken from XQuAD ( Artetxe et al., 2020 ) , where LabelPigeon accurately and seamlessly handles translating English to German while transferring 7 labeled spans with nesting.

Figure 2: Examples of labeled English sentences with two equally valid translations, where the labeled span is preserved in one and split, omitted, or ambiguous in the other.

Figure 3: An example showcasing the tag swap that we conduct on training data in order to make it generally applicable.

XQUAD
Language	COMET Score				Label Match F1 (%)
Language	Awes.	Gemma	EProj.	LP	Awes.	Gemma	EProj.	LP
Arabic	83.4	54.3	81.3	82.2	49.8	69.8	80.9	75.3
Chinese	80.8	64.9	79.4	77.9	46.4	70.6	70.9	72.9
German	83.0	73.7	81.1	82.7	60.0	86.4	84.6	86.8
Greek	84.2	71.4	84.3	87.3	44.8	71.4	65.8	75.8

Table 1: Direct label projection results on XQuAD and MLQA. COMET scores and the label match F1 scores are both provided. Sentences are translated from English to the corresponding language. We compare four label projection methods: a) Awesome-align (Awes.), b) Gemma 3 27B (Gemma), c) EasyProject (EProj.), and d) LabelPigeon (LP). Awesome-align is used as the baseline, and differences are highlighted via color.

Metrics	No Markers				Single		Simple		Complex
Metrics	Baseline	EProj.	LP	NF	EProj.	LP	EProj.	LP	EProj.	LP
BLEU	17.4	17.7	17.6	17.9	16.8	17.6	15.3	16.1	14.9	15.5
chrF++	42.9	43.5	43.4	43.8	42.8	43.7	41.4	42.3	40.8	41.7
Proj. Rate	—	—	—	—	85.9	92.5	68.2	81.0	47.7	69.3

Table 2: Metrics across different models and with different marker-insertion strategies on the Flores-200 dataset. We compare the baseline NLLB-200 3.3B model (Baseline), EasyProject (EProj.), LabelPigeon (LP), and the Non-marker Fine-tuned model (NF). BLEU and chrF++ are lexical measures for translation quality, while the projection rate measures how many of the original labels are included in the translation. Differences with respect to the baseline are highlighted via color.

UNER (Named Entity Recognition)
Language	Dataset	EProj.	Ours
Cebuano	ceb_gja	47.6	78.3
Chinese	zh_gsd	53.9	46.4
	zh_gsdsimp	52.9	47.4
	zh_pud	62.2	54.5
Croatian	hr_set	77.4	85.6

Table 3: Downstream F1 scores for UNER, CorefUD, and MLQA, comparing EasyProject (EProj.) and LabelPigeon (Ours). Differences are highlighted via color, and instances of exceptionally low scores (F1

<1

) are noted in gray.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Learning rate	1e -3
Batch size	8
Grad. Accumulation	2
Scheduler	Inverse square root
Weight Decay	0.01
Warmup	5% steps

Appendix

Table 4: Relevant hyperparameters for LabelPigeon fine-tuning.

	de-en		ru-en		zh-en
	Train	Valid	Train	Valid	Train	Valid
Samples (N)	24311	1262	24243	1301	24173	1248
Total Tags	41569	2179	41542	2250	41881	2122
Max Tags / Example	50	9	50	14	50	13
Max Unique Tags / Example	6	5	6	5	6	5
Avg. # Tags / Example	1.71	1.73	1.71	1.73	1.73	1.70

Appendix

Table 5: Statistics for our training data.

XQUAD
Language	COMET Score				Label Matches (F1, %)
Language	Base	One	Some	All	Base	One	Some	All
Arabic	79.9	80.5	81.4	80.2	2.3	71.9	75.7	74.7
Chinese	79.8	79.2	80.3	80.3	4.5	70.3	76.5	75.0
German	81.6	82.4	83.2	82.2	6.7	83.9	86.2	84.8
Greek	82.6	83.1	84.5	83.1	3.4	73.8	76.6	75.4

Appendix

Table 6: Ablations on the set of languages used for training, using our direct label projection evaluation schema in § 5 . XML is used as the marker, with both the EN→XX and XX→EN directions evaluated and the results averaged. Base refers to the original unmodified model. We compare with models trained on three language sets: 1) one high resource language (One), 2) three high resource languages (Some), and 3) all seven languages (All). Differences with Base are highlighted in color.

	XQuAD	MLQA
Samples (N)	2539	5414
Total Tags	23764	12265
Min Tags / Example	2	2
Max Tags / Example	24	8
Avg. # Tags / Example	9.36	2.27

Appendix

Table 7: Tag statistics for evaluation datasets.

	ar	de	es	hi	vi	zh
Length	843	395	1152	908	1325	791

Appendix

Table 8: Data statistics after filtering in MLQA.

XQUAD
Language	COMET Score				Label Matches (F1, %)
Language	Awes.	Gemma	EProj.	Ours	Awes.	Gemma	EProj.	Ours
Arabic	79.1	81.6	77.8	80.6	36.0	69.4	59.9	76.0
Chinese	80.3	81.9	78.7	82.7	45.0	64.7	67.2	80.0
German	82.3	85.2	81.3	83.8	59.8	80.3	81.0	85.6
Greek	80.9	84.9	80.3	81.7	51.9	73.4	73.2	77.3

Appendix

Table 9: Additional direct label projection results on XQuAD and MLQA, with sentences translated from the corresponding language to English. We compare four label projection methods: a) Awesome-align (Awes.), b) Gemma 3 27B (Gemma), c) EasyProject (EProj.), and d) LabelPigeon (LP). Awesome-align is used as the baseline, and differences are highlighted via color.

Figure 4: Translation performance of our model on Flores-200 as measured by chrF++ across different values of $P_{close}$ and $P_{open}$ under the Complex marker insertion scheme.

	BLEU										chrF++
Language	No Markers				Single		Simple		Complex		No Markers				Single		Simple		Complex
	Baseline	EProj.	LP	NF	EProj.	LP	EProj.	LP	EProj.	LP	Baseline	EProj.	LP	NF	EProj.	LP	EProj.	LP	EProj.	LP
ace_Arab	0.8	0.7	0.8	0.7	0.6	0.6	0.4	0.5	0.3	0.5	18.1	18.2	17.0	17.2	18.1	16.2	17.3	16.9	17.2	17.1
ace_Latn	9.7	9.4	9.1	9.3	8.7	8.8	7.8	7.7	6.7	7.5	37.0	37.1	36.2	36.2	36.2	35.3	33.8	33.1	32.7	32.9
acm_Arab	10.6	7.1	12.6	12.8	6.1	12.2	5.0	10.8	4.4	10.5	39.2	28.5	42.5	43.0	26.1	42.6	24.3	41.3	23.3	40.7
acq_Arab	13.6	15.2	15.2	15.8	14.6	15.1	13.1	14.0	12.4	13.0	43.2	44.7	45.2	45.5	44.3	45.0	42.8	43.7	42.2	43.0

Appendix

Table 10: Full results across different marker insertion configurations on the Flores-200 dataset (Languages 0–99).

	BLEU										chrF++
Language	No Markers				Single		Simple		Complex		No Markers				Single		Simple		Complex
	Baseline	EProj.	LP	NF	EProj.	LP	EProj.	LP	EProj.	LP	Baseline	EProj.	LP	NF	EProj.	LP	EProj.	LP	EProj.	LP
kor_Hang	11.6	12.7	10.4	10.8	10.6	12.1	9.5	10.1	9.3	9.4	33.1	34.0	32.7	33.1	32.3	32.6	29.7	29.9	29.5	29.0
lao_Laoo	7.8	8.0	7.1	7.7	6.2	8.3	5.3	7.0	4.9	5.7	42.9	44.7	43.3	44.6	43.8	45.1	43.1	44.3	42.7	43.7
lij_Latn	20.4	20.7	21.2	21.3	20.0	20.2	17.8	18.2	17.5	18.0	46.4	47.1	47.4	48.0	46.3	47.7	44.6	46.2	44.3	45.7
lim_Latn	13.8	13.4	12.5	12.3	13.3	12.0	11.5	10.8	11.5	10.7	43.3	43.1	42.6	42.6	43.2	42.5	41.9	41.4	41.6	40.8

Appendix

Table 11: Contd. results across different marker insertion configurations on the Flores-200 dataset (languages 100–203).

Explore similar work

XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics
Apr 16, 2026Jingxuan Liu, Zhi Qu, Jin Tei +3MT Evaluation
Nara Institute of Science and Technology, Japan. · Fudan University, China.
LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation
Apr 20, 2026Samar M. Magdy, Fakhraddin Alwajih, Abdellah El Mekki +2Arabic NLP Machine Translation
λThe University of British Columbia · ξMinia University · γCanada Research Chair in NLP and ML
Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
Mar 19, 2026Yana Veitsman, Yihong Liu, Hinrich SchützeCross-Lingual Representation Alignment Cross-Lingual Transfer
University of Göttingen · LMU Munich · Munich Center for Machine Learning (MCML)

Apr 16, 2026J/K move · Enter open · S save

XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics

Jingxuan Liu, Zhi Qu, Jin Tei, Hidetaka Kamigaito+2

Nara Institute of Science and Technology, Japan. · Fudan University, China.

MT Evaluation

Open paper AskPDF arXiv

Automatic evaluation metrics are essential for building multilingual translation systems. The common practice of evaluating these systems is averaging metric scores across languages, yet this is suspicious since metrics may suffer from cross-lingual scoring bias, where translations of equal quality receive different scores across languages. This problem has not been systematically studied because no benchmark exists that provides parallel-quality instances across languages, and expert annotation is not realistic. In this work, we propose XQ-MEval, a semi-automatically built dataset covering nine translation directions, to benchmark translation metrics. Specifically, we inject MQM-defined errors into gold translations automatically, filter them by native speakers for reliability, and merge errors to generate pseudo translations with controllable quality. These pseudo translations are then paired with corresponding sources and references to form triplets used in assessing the qualities of translation metrics. Using XQ-MEval, our experiments on nine representative metrics reveal the inconsistency between averaging and human judgment and provide the first empirical evidence of cross-lingual scoring bias. Finally, we propose a normalization strategy derived from XQ-MEval that aligns score distributions across languages, improving the fairness and reliability of multilingual metric evaluation.

Abstract

Explore similar work

XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics

LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders