Wolof-Adapted Industry-Ready Open-Weight Language Models

Gwénolé Lecorvé; Delphine Charlet; Pierre Adam; Floriane Behanzin; Lionel Delphin-Poulat; Johannes Heinecke; Géraldine Damnati; Pelin Kaytmaz

cs.CLOct 14, 2026

Wolof-Adapted Industry-Ready Open-Weight Language Models

Authors: Gwénolé Lecorvé, Delphine Charlet, Pierre Adam, Floriane Behanzin, Lionel Delphin-Poulat, Johannes Heinecke, Géraldine Damnati, Pelin Kaytmaz,

Organizations: Orange Research, France

Abstract

Wolof is spoken by 23M people, mostly in Senegal, but remains weakly supported by open-weight language models, while API models raise cost, governance, and sovereignty constraints for local deployments. In this paper, we present 0.6B-14B open-weight models adapted to Wolof through instruction tuning, preference alignment, and model merging. After presenting the pipeline and data, extensive experiments on 63~public and in-house tasks show good performance on Wolof understanding and generation while remaining effective on other languages and cross-lingual skills, with the 8B and 14B models being particularly competitive with substantially larger commercial systems. The models are currently being evaluated in internal customer-care pilots and are publicly available at https://huggingface.co/collections/Orange/wolof-lm .

Figures & tables

	Dataset source	Samples	Tokens	Lang.	N. turns
Biling.	Alwaly FR $\rightarrow$ WO	91K	5.0M	FR/WO	Single
	Alwaly WO $\rightarrow$ FR	91K	5.0M	WO/FR	Single
	Total	182K	10.0M
Translated general	Alpaca	52K	10.4M	WO	Single
	Dolly-15k	15K	4.4M	WO	Single
	SAMSum	15K	4.0M	WO	Single

Table 1: Training data. WO, FR, and EN stand for Wolof, French, and English, resp.. Turns are user-assistant pairs of messages (+ optional system message).

Model	Overall	Correctness	Formatting	Wolof quality
Gemini-2.5-flash	1112 (.67)	1104 (.66)	1072 (.61)	1066 (.60)
Gemini-2.5-pro	1093 (.64)	1082 (.63)	1090 (.64)	1062 (.60)
Wolof-Qwen3-8B	1051 (.57)	1035 (.55)	1030 (.54)	1058 (.59)
Oolel-v0.1	1002 (.48)	994 (.48)	1030 (.54)	1038 (.55)
Qwen3-8B	742 (.14)	785 (.18)	777 (.17)	777 (.16)

Table 4: Human evaluation Elo scores and win probability by criterion. The Wolof quality column excludes tasks whose expected output is English or French or a classification label.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Aggregate scores as a function of model size. Orange squares denote our Wolof-adapted models. Circles denote unadapted baselines, and diamonds denote Wolof-aware models from the literature. Commercial API systems are shown using purely indicative, prospectively estimated parameter ranges based on unofficial online information, rather than vendor-confirmed specifications.

Dataset	Split	Examples	Tokens	Avg Len	Language	Type
Translated General Instruction (Wolof)
Alpaca (Wolof)	Train	51,939	10,420,068	200.6	Wolof	Single turn
	Validation	30	6,456	215.2	Wolof	Single turn
Dolly-15k (Wolof)	Train	14,985	4,352,286	290.4	Wolof	Single turn
	Validation	30	9,723	324.1	Wolof	Single turn
SAMSum (Wolof)	Train	14,696	4,006,869	272.7	Wolof	Single turn

Appendix

Table 5: Dataset statistics by split. Avg Len is the average number of tokens per example. Wolof-containing data (monolingual and bilingual) accounts for approximately 90% of training examples.

Category	Details
Hardware	Single node; 2 $\times$ NVIDIA H100 80GB and H200 140GB GPUs
Training duration	3 epochs; a few hours to a few days depending on model and corpus size
Optimizer	Paged AdamW ( paged_adamw_32bit )
Learning rate schedule	Cosine decay with 5% warmup
Peak learning rate	$2\times 10^{-5}$
Regularization	Gradient clipping (max-norm 1.0); weight decay 0.1

Appendix

Table 6: Instruction tuning (SFT) training setup.

Instruction-Following
Dataset	Split	Examples	Avg Len Prompt	Avg Len Chosen	Avg Len Rejected
Chat & WildChat	Train	27,284	404.3	383.4	803.9
	Validation	2896	366.5	378.3	895.1
Dolci Instruct Precise IF	Train	31,449	197.1	555.3	537.2
	Validation	3534	194.1	558.0	554.7
Dolci Instruct Persona Precise IF	Train	5,994	58.9	413.1	322.9

Appendix

Table 7: Composition and Qwen3 token-length statistics for Dolci-Instruct-DPO. Token averages are computed using Qwen3 tokenizer.

Category	Details
Hardware	Single node; 2 $\times$ NVIDIA H100 80GB and 1 $\times$ H200 140GB GPUs
Training duration	2 epochs; from a day to a few days depending on model size
Beta	0.2
Learning rate schedule	Cosine decay with 5% warmup
Peak learning rate	$5\times 10^{-6}$
Regularization	Gradient clipping (max-norm 1.0); weight decay 0.01

Appendix

Table 8: Preference alignment (DPO) training setup.

Language Modeling

Language modeling develops systems that learn patterns in text and other sequences to understand, predict, and generate language.

See all 519 papers

Building a Strong Instruction Language Model for a Less-Resourced Language
Mar 2, 2026Domen Vreš, Tjaša Arčon, Timotej Petrič +3Low-Resource Language Processing LLM Fine-Tuning
University of Ljubljana, Faculty of Computer and Information Science, Ljubljana, Slovenia
Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models
Jul 29, 2026Parishruthi Ganesh, Gerry Dozier, Cheryl SealsLLM Evaluation Zero-Shot Text Classification
Department of Computer Science and Software Engineering Auburn University Auburn, Alabama, USA
SomaliBench Eval: Measuring English-to-Somali Refusal Gaps in Open-Weight Language Models
May 25, 2026Khalid Yusuf DahirLLM Safety Benchmarks Multilingual Language Model Evaluation
Independent researcher

Mar 2, 2026J/K move · Enter open · S save

Building a Strong Instruction Language Model for a Less-Resourced Language

Domen Vreš, Tjaša Arčon, Timotej Petrič, Dario Vajda+2

University of Ljubljana, Faculty of Computer and Information Science, Ljubljana, Slovenia

Low-Resource Language Processing LLM Fine-Tuning Fine-Tuning Language Models Domain Adaptation for LLMs

Open paper AskPDF arXiv

Large language models (LLMs) have become an essential tool for natural language processing and artificial intelligence in general. Current open-source models are primarily trained on English texts, resulting in poorer performance on less-resourced languages and cultures. We present a set of methodological approaches necessary for the successful adaptation of an LLM to a less-resourced language, and demonstrate them using the Slovene language. We present GaMS3-12B, a generative model for Slovene with 12 billion parameters, and demonstrate that it is the best-performing open-source model for Slovene within its parameter range. We adapted the model to the Slovene language using three-stage continual pre-training of the Gemma 3 model, followed by two-stage supervised fine-tuning (SFT). We trained the model on a combination of 140B Slovene, English, Bosnian, Serbian, and Croatian pretraining tokens, and over 200 thousand English and Slovene SFT examples. We evaluate GaMS3-12B on the Slovenian-LLM-Eval datasets, English-to-Slovene translation, and the Slovene LLM arena. We show that the described model outperforms 12B Gemma 3 across all three scenarios and performs comparably to much larger commercial GPT-4o in the Slovene LLM arena, achieving a win rate of over 60 %.

Abstract

Language Modeling

Explore similar work

Building a Strong Instruction Language Model for a Less-Resourced Language

Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models

SomaliBench Eval: Measuring English-to-Somali Refusal Gaps in Open-Weight Language Models