Supervised fine-tuning datasets remain overwhelmingly English-centric and commercially restricted, as their content is frequently generated by proprietary models whose terms of service forbid training competing systems. We introduce Multi-SFT, a commercially permissive collection of 502,818 instruction-response pairs across five European languages, built via translation and regeneration pipelines for tasks where translation is insufficient. Human evaluation yields inter-annotator agreement of at least 0.87 across all datasets and languages. Fine-tuning nine models from three different families (EuroLLM, OLMo 2, Qwen3; 1B-32B) and evaluating them on 13 benchmarks shows that Multi-SFT alone is competitive with larger augmented mixes and official instruction-tuned releases, with decisive advantages on safety tasks.
Figures & tables
Project
Multilingual
Dataset License
Data Provenance
Apertus
✓
✓
✗
EuroLLM
✓
~
✗
Nemotron
✗
✓
✓
OLMo
✗
✓
✗
Salamandra
✓
✗
✗
Multi-SFT
✓
✓
✓
Table 1: Comparison of SFT mixtures used in recent open LLM projects. ✓ denotes full compliance, ✗ denotes non-compliance, and ~ denotes partial compliance. Dataset License indicates whether all training data is released under permissive licenses. Data Provenance indicates whether synthetic data is generated exclusively with permissive or open-weight models, excluding proprietary sources such as GPT-4 or Gemini.
Dataset
Domain
Strategy
License
EN
DE
ES
FR
IT
Total
Multi-AdvBench
Safety
Regeneration
MIT
104
104
104
104
104
520
Multi-AyaRedteaming
Safety
Regeneration
Apache 2.0
812
835
330
754
913
3,644
multi-dialogues
Conversation
Regeneration
ODC-BY-1.0
-
12,001
11,999
12,000
12,001
48,001
multi-dolly-15k
General QA
Translation
CC BY-SA 3.0
6,004
2,250
2,251
2,252
2,222
14,979
Multi-FLAN
Reasoning
Translation
Apache 2.0
36,069
13,495
13,441
13,492
13,485
89,982
multi-mentorES
General QA
Translation
CC BY 4.0
-
2,541
2,552
2,534
2,548
10,175
Table 2: Overview of the multi- dataset collection. Source datasets and pipelines: AdvBench ( Zou et al., 2023 ) , Aya Red-teaming ( Ahmadian et al., 2024 ) , SODA ( Kim et al., 2023 ) , Dolly ( Conover et al., 2023 ) , FLAN ( Longpre et al., 2023 ) , Persona IF ( Lambert et al., 2024 ) , RAG Multilingual ( Projecte AINA, 2026 ) , SciRIFF ( Wadden et al., 2025 ) , SmolTalk ( Allal et al., 2025 ) , and Table-GPT ( Li et al., 2024 ) .
Dataset
DE
EN
ES
FR
IT
multi-dialogues
98
–
99
100
95
multi-dolly-15k
100
–
93
100
94
Multi-FLAN
90
–
87
84
88
multi-mentorES
95
–
–
96
93
Multi-PersonaIF
87
89
83
80
72
Multi-RAG-Multilingual
100
–
–
96
90
Table 3: Percentage of positively-rated entries by dataset and language. Values represent the average of the two independent annotators.
Dataset
DE
EN
ES
FR
IT
multi-dialogues
0.99
–
0.99
1
0.97
multi-dolly-15k
1
–
0.95
1
0.98
Multi-FLAN
0.90
–
0.97
0.93
0.93
multi-mentorES
0.99
–
–
0.96
0.98
Multi-PersonaIF
0.98
0.94
0.98
0.88
0.94
Multi-RAG-Multilingual
1
–
–
0.98
0.97
Table 4: Inter-annotator agreement (Gwet’s AC2) by dataset and language.
Multilingual
English
Family
Model
MARCO
FLORES
Belebele
XQuAD
M-ALERT
MMLU
ARC
XStory
Avg
XQuAD
DROP
IFE
MMLU
ARC
Avg
EuroLLM
1.7B ml
31.7
49.0
23.4
33.4
43.6
24.5
32.0
58.4
37.0
32.8
23.0
32.3
23.5
34.0
29.1
1.7B full
21.1
45.3
23.0
25.7
21.6
24.4
32.2
61.2
31.8
24.6
21.1
27.9
23.4
34.1
26.2
1.7B bl
27.1
52.2
25.9
42.6
29.0
24.5
32.1
58.6
36.5
42.5
25.2
39.9
24.3
37.5
33.9
1.7B sft †
16.5
59.1
25.4
25.0
44.4
26.3
33.3
60.8
36.4
37.1
25.1
20.3
26.7
37.4
29.3
9B ml
50.2
62.0
64.6
75.8
56.3
50.7
48.9
64.5
59.1
73.9
46.6
59.5
53.0
52.9
57.2
Table 5: Benchmark results divided between Multilingual (left) and English (right). ml : Multi-SFT ; full : extended mix; bl : baseline SFT; sft † : official provider model. Average performance is given per group (see Avg columns).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Type
Description
messages
list[{role: str, content: str}]
Conversation turns in chat format. Roles are system , user , and assistant .
source_data
string
Identifier of the originating dataset or repository.
category
string
Task category: Chat , Instruction following , Reasoning , Code , Knowledge , or Safety .
language
string
Language code of the example (e.g., en , de , fr , it , es ).
token_count
int32
Number of tokens in the conversation, computed using the tokenizer of the target model.
strategy
string
Origin or transformation status of the example: original , translated , regenerated , or translated+regenerated .
Appendix
Table 6: Schema of the Multi-SFT dataset.
Instruction-following datasets
A
Correct and complete response fully satisfying the instruction.
B
Generally correct response with minor issues (e.g., slight inaccuracies, small omissions, or minor linguistic problems).
C
Response attempts to follow the instruction but contains major content or linguistic issues reducing training usefulness.
D
Irrelevant, off-topic, or otherwise unusable response.
Appendix
Table 7: Annotation schema used for instruction-following and conversational datasets.