Text Generation in Low-Resource Languages Using Human-Curated Vocabularies

Michael Christlambert Sinanta; Muhammad Dehan Al Kautsar; Amanda Nurul Izzah; Thanh-Nhi Nguyen; Phan Phuong Mai Chau; Zheng Wei Lim; Fajri Koto

cs.CLOct 27, 2026

Text Generation in Low-Resource Languages Using Human-Curated Vocabularies

Authors: Michael Christlambert Sinanta, Muhammad Dehan Al Kautsar, Amanda Nurul Izzah, Thanh-Nhi Nguyen, Phan Phuong Mai Chau, Zheng Wei Lim, Fajri Koto

Organizations: Independent Researcher · Mohamed bin Zayed University of Artificial Intelligence · University of Amsterdam

Abstract

Synthetic text generation offers a promising approach to alleviating data scarcity in low-resource languages. However, its effectiveness is inherently limited for languages that are scarcely represented, or entirely absent, in an LLM's pretraining data. At the same time, many such languages have human-curated vocabularies that provide rich lexical and semantic information, often accompanied by grammatical information such as part-of-speech labels. In this work, we ask whether LLMs can leverage these resources to generate coherent text in extremely low-resource and unseen languages. We introduce a framework that combines human-curated vocabularies with a small number of human-written exemplars that provide syntactic signals, while sampling candidate words according to a Zipfian distribution. Across languages with varying levels of resource availability, we evaluate the generated text using lexical diversity, off-target ratio, and human judgments of fluency. Our results show that reasoning-enabled LLMs can generate fluent text even in extremely low-resource languages, with particularly strong results at the sentence level and promising performance at the paragraph level. Our findings highlight a practical pathway for bootstrapping synthetic data in low-resource languages, expanding the amount of usable text even when existing digital resources are extremely limited. Our source code is available at https://github.com/michaelsinanta/vocab2text.

Figures & tables

Figure 1: Overview of our vocabulary-guided synthetic text generation pipeline. Human-curated vocabulary is sampled using an open-source word-frequency list, while we then sample the high- and low- frequency words using our method described in Section 3 . We then combine it with few-shot human-written exemplars to construct the generation prompt.

Language	Pearson	Spearman
Indonesian	0.564	0.243
Vietnamese	0.554	0.516
Javanese	0.365	0.331
Minangkabau	0.747	0.736
Acehnese	0.774	0.771
Hitu	0.466	0.494

Table 1: Inter-annotator Pearson and Spearman correlations of human fluency ratings between the two annotators, by language.

Language	Prompt	Fluency $\uparrow$			Off-Target Rate $\downarrow$	Lexical Diversity $\uparrow$
Language	Prompt	5	15	Para.	Off-Target Rate $\downarrow$	5	10	15	20	Para.
Medium-Resource Languages
Indonesian	Zero-shot	4.989	4.950	4.761	0.028	0.542	0.573	0.570	0.588	0.546
	Few-shot	4.994	4.994	4.994	0.018	0.430	0.520	0.568	0.608	0.548
	Ours	4.972	4.911	4.811	0.014	0.868	0.859	0.834	0.817	0.721
Vietnamese	Zero-shot	4.667	4.744	4.722	0.015	0.459	0.546	0.551	0.548	0.508

Table 2: Fluency, off-target rate, and lexical diversity per language and prompt. Each value is the average over models and thinking mode (for fluency, only models with thinking mode enabled). Best per language in bold .

Language	Spearman	MAE	Bias
Indonesian	0.37	0.12	-0.04
Vietnamese	0.39	0.44	+0.40
Javanese	0.34	0.87	+0.44
Minangkabau	0.47	0.91	+0.60
Acehnese	0.69	0.86	+0.58
Hitu	0.15	1.70	+1.19

Table 3: Reliability of the LLM judge (Gemini 2.5 Flash) on fluency evaluation, measured against human ratings.

Model	Thinking	Off-Target Rate $\downarrow$	Lexical Diversity $\uparrow$
GPT	Off	0.347	0.311
	On	0.322	0.341
Gemini	Off	0.294	0.336
	On	0.233	0.371
Claude	Off	0.378	0.300
	On	0.394	0.290

Table 4: Results per model and thinking mode, averaged over three languages (Hitu, Minangkabau, and Acehnese) and three prompts. Lexical diversity is averaged over the 5-, 10-, 15-, 20-word and paragraph settings. The best score between the thinking modes is shown in bold .

Panel of Figure 2 — Figure 2: Average percentage of unique valid n-grams as the number of generated Minangkabau sentences increases.

Panel of Figure 3 — Figure 3: Average percentage of unique valid n-grams as the number of generated Hitu sentences increases.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Language	Number of Words
Indonesian	71,017
Javanese	65,568
Vietnamese	33,968
Minangkabau	33,149
Acehnese	10,638
Hitu	1,347

Appendix

Table 5: Number of words in the dictionary of each language.

Param.	Description	Values
$k$	Number of word candidates	$[100,300,500]$
$n$	Number of few-shot examples	$[3,5,7,10]$
$b$	Rank boundary	$[100,500,1000]$
$c$	Composition	$[10\%,20\%,50\%]$
$p$	Top- $p$ similarity fraction	$[25\%,50\%]$

Appendix

Table 6: Hyperparameter settings.

Language	Generated Text	Human
		Score
5-word sentences
Zero-shot	I don’t have reliable knowledge of the Hitu language to generate an accurate sentence. Hitu is a small Austronesian language spoken in Indonesia, and …	1
Ours	Malole ma laka tarus nela.	5
Paragraphs
Zero-shot	I appreciate your request, but I must be honest: I don’t have reliable knowledge of the Hitu language to generate authentic text in it. Hitu is a language spoken in Indonesia’s Maluku Islands …	1

Appendix

Table 7: The sample outputs of Claude Haiku 4.5 in zero-shot and our method at the 5-word sentence and paragraph level.

				Fluency $\uparrow$			Off-Target Rate $\downarrow$	Lexical Diversity $\uparrow$
Language	Model	Thinking	Prompt	5	15	Para.		5	10	15	20	Para.
Medium-Resource Languages
Indonesian	GPT	On	Zero-shot	4.967	4.933	4.400	0.036	0.577	0.541	0.578	0.606	0.565
			Few-shot	4.983	5.000	5.000	0.014	0.533	0.556	0.546	0.618	0.569
			Ours	4.967	4.917	4.700	0.015	0.888	0.857	0.824	0.817	0.715
		Off	Zero-shot	—	—	—	0.029	0.579	0.543	0.573	0.569	0.549

Appendix

Table 8: Fluency, off-target rate, and lexical diversity per language, model, thinking mode, and prompt. Fluency was only human-evaluated for models with thinking mode enabled (— = not evaluated). Best per model and thinking mode in bold .

Text Generation

Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.

See all 423 papers

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation
Jun 16, 2026Jan Cegin, Daniil Gurgurov, Yusser Al Ghussin +1Language Model Steering Synthetic Data Generation for Language Models
Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · German Research Institute for Artificial Intelligence (DFKI), Saarbrücken, Germany
A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books
Jul 24, 2026Varun Ghat Ravikumar, Sina Ahmadi, Lena Jäger +1LLM Fine-Tuning Synthetic Data Generation
Department of Informatics, University of Zurich · Department of Computational Linguistics, University of Zurich
Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data
Mar 4, 2026Zaruhi Navasardyan, Spartak Bughdaryan, Bagrat Minasyan +1Text Embeddings Low-Resource Language Processing
Metric AI Lab

Jun 16, 2026J/K move · Enter open · S save

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

Jan Cegin, Daniil Gurgurov, Yusser Al Ghussin, Simon Ostermann

Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · German Research Institute for Artificial Intelligence (DFKI), Saarbrücken, Germany

Language Model Steering Synthetic Data Generation for Language Models Low-Resource Language Processing Activation Steering Large Language Model-Based Data Augmentation

Open paper AskPDF arXiv

Large language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated data can improve downstream task performance. Current best-performing approaches typically rely on few-shot prompting with target-language examples, which increases inference costs and may reduce diversity through lexical anchoring. In this work, we investigate activation steering as an alternative for low-resource synthetic data generation. We study two steering strategies: Language Steering, which targets the linguistic identity of a language, and Quality Steering, which captures well-formedness by contrasting human-written and backtranslated text representations. We evaluate these methods across four open-source LLMs, multiple layers, and 11 typologically diverse languages by generating sentiment and topic classification data and finetuning smaller classifiers. Steering is applied in both zero-shot and few-shot prompting settings and compared against non-steered counterparts. Our results show that steering on early layers consistently improves the diversity of generated data while often yielding stronger downstream model performance, particularly for low-resource languages.

Abstract

Text Generation

Explore similar work

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books

Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data