EuroAlpaca: Task-Preserving Localisation of Instruction Data for European Languages

Aleix Sant; Jordi Luque; Carlos Escolano

cs.CLSep 4, 2026

EuroAlpaca: Task-Preserving Localisation of Instruction Data for European Languages

Authors: Aleix Sant, Jordi Luque, Carlos Escolano

Organizations: Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya Barcelona, Spain

Abstract

Machine translation (MT) offers a scalable way to extend English instruction-tuning data to multiple languages, but it can distort task-critical constraints and required outputs, creating corrupted training examples and degrading models trained on such data. We introduce EuroAlpaca, a task-preserving localisation pipeline and near-parallel resource covering 50 European languages and regional varieties, together with European-IFEval, a multilingual benchmark for verifiable instruction following. Depending on the example, our pipeline applies field-wise MT while preserving task-critical content or reconstructs a task-equivalent target-language instance, followed by validation of cross-field coherence and target-language consistency. Across LoRA experiments with four LLMs, training on directly translated data improves ROUGE-L and F-BERT on the Aya Evaluation Suite, but reduces accuracy on European-IFEval by 29.8% relative to the unadapted baseline. In contrast, adaptation with EuroAlpaca improves accuracy by 12.9% over the same baseline, reversing the degradation caused by direct MT, while also achieving the highest ROUGE-L and F-BERT scores on Aya. These results show that preserving task semantics is essential for multilingual instruction tuning.

Figures & tables

Figure 1: Task-preserving instruction-data generation pipeline. Blue nodes are data artefacts, purple are LLM stages, orange is policy-guided MT, and green is task-level rewriting. Gated routing directs samples to either per-field translation ( translate / preserve ) or task-level localisation, yielding Task-Preserved which is validated to produce EuroAlpaca for language $\ell$ .

Handling regime	Count	%
Direct translation	27,946	54.0
Field-preserving translation	12,726	24.6
Task-level localisation	11,088	21.4
Total	51,760	100.0

Table 1: Handling regimes for the 51,760 source examples. Direct translation translates all non-empty fields. Field-preserving translation retains at least one field unchanged. Task-level localisation reconstructs examples for which field-wise MT would invalidate the task.

Failure type	Typical cases	Handling policy
Source-language form dependence	Grammar or spelling correction, punctuation judgements, tense/voice edits, alphabetic ordering	Generate an analogous target-language task and recompute the output.
Lexical or phonological constraints	Letter/character counts, word counts, rhymes, acrostics, mnemonics, puns, anagrams	Replace the constrained material with target-language content satisfying the same constraint.
Language- or culture-specific expressions	Idioms, proverbs, fixed expressions, English-only prompts, language-specific examples	Substitute a natural target-language analogue while preserving task intent and difficulty.
Protected literals	Code, URLs, paths, identifiers, formulas, quoted strings, names, titles, table columns	Preserve fields containing task-critical literal artefacts.
Fixed output vocabulary or schema	Classification labels, JSON keys, structured outputs, canonical options	Preserve fields defining or containing the required labels or output format.
Field-role dependence	Translation tasks, demonstrations, input–output examples, source/target language roles	Preserve the role of each field rather than translating all fields uniformly.

Table 2: Operational taxonomy of examples requiring non-uniform translation across fields.

Source task	Uniform MT failure	Task-preserving construction
Correct the grammar in an intentionally erroneous sentence.	Translation may silently remove the error, leaving nothing to correct.	Create a comparable target-language error and provide its correction.
Return a word that rhymes with a given word.	Rhyme depends on target-language phonology. Translated words may not rhyme.	Select a target-language cue and answer satisfying the same relation.
Write exactly ten words about a topic.	Word counts are not preserved under translation.	Generate target-language text satisfying the count.
Classify with exact labels.	Translating labels can break the required output vocabulary or automatic checker.	Translate the content but preserve the labels verbatim.
Interpret an idiom.	Literal translation may produce a nonsensical expression or a different idiom.	Replace it with a natural target-language idiom and explain that expression.
Translate a sentence from language A to language B.	Translating every field can invalidate the specified language pair.	Translate the instruction, but preserve the source and target texts.

Table 3: Schematic examples where direct field-wise translation can break the target-language instruction task.

Figure 2: Approximate traditional geographic areas associated with the 50 European languages and regional varieties covered by EuroAlpaca and European-IFEval . Boundaries are schematic and do not represent exclusive or current speaker distributions. The map is based on FUEN, 2013 .

		Aya ROUGE-L $\uparrow$		Aya F bert $\uparrow$		European-IFEval Acc. $\uparrow$
Model	Training data	Mean	$\Delta$ OTS	Mean	$\Delta$ OTS	Mean	$\Delta$ OTS
Tiny-Aya-Global	Off-the-shelf	0.103	–	0.824	–	0.421	–
	Direct MT	0.228	+0.125	0.872	+0.049	0.295	$-$ 0.126
	Task-Preserved	0.232	+0.129	0.874	+0.050	0.446	+0.025
	EuroAlpaca	0.238	+0.135	0.877	+0.054	0.499	+0.078
Phi-4-Mini-Instruct	Off-the-shelf	0.091	–	0.827	–	0.258	–

Table 4: Language-averaged performance for each model, with macro-averages across the four models. Aya ROUGE-L and F bert are averaged over the 36 languages with available reference data, whereas European-IFEval accuracy is averaged over all 50 evaluated languages. The

\Delta

OTS columns report absolute score differences relative to the corresponding off-the-shelf checkpoint.

Figure 3: Macro-average $\Delta$ OTS values across the four evaluated models

		Training data			Training data			Training data
Eval metric (Macro Avg)	$n$	Okapi	EuroAlpaca	$n$	MITS	EuroAlpaca	$n$	Bactrian-X	EuroAlpaca
Aya ROUGE-L $\uparrow$	16	0.231	0.239	35	0.212	0.223	19	0.217	0.232
Aya F bert $\uparrow$	16	0.879	0.879	35	0.864	0.870	19	0.871	0.878
European-IFEval Acc. $\uparrow$	17	0.377	0.397	38	0.289	0.359	20	0.401	0.385

Table 5: Pairwise comparison of LoRA fine-tuning runs on Alpaca-derived multilingual instruction resources and on our EuroAlpaca . For each resource and benchmark, scores are macro-averaged over a fixed shared-language set within each model and then averaged across the four models.

n

denotes the number of languages shared by the resource and EuroAlpaca that are also covered by the corresponding evaluation benchmark.

Resource	Sample-level alignment	Embedding similarity
Okapi	Yes	0.867
EuroAlpaca	Yes	0.860
Bactrian-X	Yes	0.800
MITS	No	–

Table 6: Sample-level alignment indicates whether corresponding records can be matched across languages. Embedding similarity is the mean cosine similarity between BGE-M3 representations of aligned instruction–input–output records over the common 11-language intersection of EuroAlpaca , Okapi and Bactrian-X.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Languages	Hugging Face Dataset ID
Native-speaker or professional translator (human-curated)	ca	projecte-aina/IFEval_ca
	da	danish-foundation-models/ifeval-da
	de	jzhang86/de_ifeval
	el	ilsp/ifeval_greek
	es	BSC-LT/IFEval_es
	et	tartuNLP/ifeval_et

Appendix

Table 14: Language-level provenance summary for European-IFEval data. The released manifest stores the exact URL, source tier, missing-sample count and source-language answer-constraint count for each language.

Parameter	Value
Method	LoRA
Rank / alpha / dropout	16 / 32 / 0.05
Target modules	q , k , v , and o projections
Maximum sequence length	1,300 tokens by default
Optimiser	AdamW
Learning rate	$1\times 10^{-5}$

Appendix

Table 34: Fine-tuning configuration shared by all language-specific runs. Based on token-length analysis, we use a 1,600-token limit for ga , gd , is , mk , mt and sq to reduce truncation in the upper tail of the sequence-length distribution.

Component	Configuration
Decision model	Gemma-4-31B-IT, Q8_K_XL GGUF
Draft model	Gemma-4-E2B-IT, Q4_K_XL GGUF
Decoding	Temperature $=0$ , top- $p=0.95$ , min- $p=0.01$
Penalties	None explicitly configured
Reasoning	Disabled
Speculative decoding	5–16 draft tokens

Appendix

Table 35: Inference and decoding configuration of the Gemma-4 decision module.

Instruction Tuning

Instruction tuning adapts pretrained language models to follow natural-language requests using curated or synthesized instruction-response examples.

See all 81 papers

Task-Preserving Multilingual Instruction Tuning for Open-Ended QA in WMT 2026 MIST
Oct 21, 2026Aleix Sant, Jordi Luque, Carlos EscolanoMultilingual QA Fine-Tuning Language Models
Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Sep 23, 2026Niklas Scholz, David Thulke, Abdallah Nasir +3LLM Fine-Tuning Elastic Weight Consolidation
AppTek GmbH, Aachen, Germany · Machine Learning and Human Language Technology, RWTH Aachen University, Germany · Applied Science Private University, Amman, Jordan
IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
May 27, 2026Mingrui Sun, Mao Zheng, Zheng Li +1Multilingual Language Model Evaluation Machine Translation
Large Language Model Department, Tencent

Oct 21, 2026J/K move · Enter open · S save

Task-Preserving Multilingual Instruction Tuning for Open-Ended QA in WMT 2026 MIST

Aleix Sant, Jordi Luque, Carlos Escolano

Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya

Multilingual QA Fine-Tuning Language Models Instruction Tuning Open-Domain QA Data Augmentation

Open paper AskPDF arXiv

This paper describes our submission to the WMT 2026 Multilingual Instruction Shared Task. We adapt Qwen3-8B with LoRA using organiser-provided data, EuroAlpaca, a multilingual instruction dataset created through task-preserving localisation, and CrossEuroAlpaca, a cross-lingual augmentation that assigns different source and target languages to instructions, contexts and responses. Our system targets open-ended question answering (QA-OEG) in ten shared-task languages represented in the auxiliary data. Under matched decoding, adaptation improves the mean Gemma-4 judge score on target-language QA-OEG by 7.8% relative to the base model. On an additional evaluation using the Aya Evaluation Suite across all shared-task languages, it yields relative gains of 41.5% in macro-averaged ROUGE-L and 2.9% in BERTScore-F1, with larger improvements in target languages. However, these gains are task- and language-specific. Gemma-4 scores improve for QA-OEG and summarisation in the target languages but decline for context-grounded QA there and across all three tasks in the non-target languages.

Abstract

Instruction Tuning

Explore similar work

Task-Preserving Multilingual Instruction Tuning for Open-Ended QA in WMT 2026 MIST

Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following

IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following