Finding the passage that answers a question is the first step of most practical systems built on documents, and for Tajik we found no openly described model for it. We present Saidzoda-Tajik-E5-Large, a Tajik text embedder built from multilingual-e5-large-instruct (560M parameters). We asked our own Tajik language model to write questions for passages from Tajik laws, tax documents, Wikipedia and web text, and trained on 235,140 question and passage pairs for one epoch, with hard negatives chosen by another embedder and English and Russian pairs mixed in so the model would not forget those languages. On laws the model had never seen, the right passage is among the top ten results for 93.6% of questions, up from 80.2%; on tax documents, 94.5% against 82.6%. On Belebele, the one test we have that was written by people, the gain is smaller and not confirmed (92.2% to 95.2%, with touching intervals), and Russian and English do not change. On law and tax questions written in the same style as its training questions, it is tied with Giga-Embeddings-3B, a model five times its size; on Belebele the 3B and 8B models score slightly higher, and only after we fine-tuned them on our data are they clearly ahead. We also report what failed: training in bfloat16 broke the model, and replacing Tajik letters with similar Russian ones did not help. Our law, tax and Wikipedia test questions come from the same model that wrote the training questions, so we treat those scores as an upper limit until questions written by Tajik speakers are available.
Figures & tables
Domain
Pairs
With negative
Tajik law
52,758
40,259
Tajik tax
1,033
875
Tajik Wikipedia
21,123
20,374
Tajik web and literature
59,985
39,140
Customer service (synthetic)
15,744
6,313
English replay (NQ)
49,987
n/a
Table 1: Training pairs by domain, as counted by the training run. Replay pairs use in-batch negatives only.
Figure 1: Top: training loss of Saidzoda-Tajik-E5-Large (logged every 10 steps). Bottom: two short test runs with the same settings; in bfloat16 the logged loss drops to exactly zero at step 8 and stays there, in fp32 it does not.
Test set (queries)
e5 base
Ours
Change
Law (1,000)
80.2 [77.6, 82.6]
93.6 [92.0, 95.1]
+13.4
Tax (109)
82.6 [75.2, 89.9]
94.5 [89.9, 98.2]
+11.9
Wikipedia (500)
96.4 [94.8, 97.8]
97.4 [96.0, 98.6]
+1.0
Belebele Tajik (900)
92.2 [90.4, 93.9]
95.2 [93.9, 96.6]
+3.0
Belebele Russian (900)
97.9 [96.8, 98.8]
98.6 [97.8, 99.2]
+0.7
Belebele English (900)
98.9 [98.2, 99.6]
98.9 [98.1, 99.6]
+0.0
Table 2: Recall@10 (top block, with 95% bootstrap intervals) and recall@1 (bottom block) in percent, for the starting model and our fine-tuned model. Change is in percentage points. Law, tax and Wikipedia queries were written by an LLM on held-out documents; only the Belebele sets are human-written.
Model
Size
Used as
Law
Tax
Wiki
Bel. tg
Bel. ru
Bel. en
EmbeddingGemma-300M
0.31B
zero-shot
49.3
45.9
89.6
69.8
98.6
99.2
gte-multilingual-base
0.31B
zero-shot
62.2
66.1
92.0
81.3
97.3
98.8
Qwen3-Embedding-0.6B
0.6B
zero-shot
35.9
51.4
86.6
66.1
97.7
98.9
BGE-M3
0.57B
zero-shot
77.6
78.9
95.0
83.8
98.9
99.0
multilingual-e5-large-instruct
0.56B
zero-shot
80.2
82.6
96.4
92.2
97.9
98.9
F2LLM-v2-4B
4B
zero-shot
69.7
69.7
94.8
94.0
98.3
99.0
Table 3: Recall@10 (percent) of every embedder we scored on the same frozen test sets. Bold marks the highest value in each column, not a significant difference. Rows above the line are used as released; rows below were trained on our pairs (the LoRA runs used 40% of them). Belebele Russian and English check that the other languages are not lost.
Variant
Law
Tax
Wiki
Bel. tg
e5 base
80.2
82.6
96.4
92.2
e5 base, letters folded
78.0
78.9
96.2
92.0
Ours, step 300
93.6
95.4
97.0
95.7
Ours, step 400
93.5
94.5
97.4
96.1
Ours, final (step 466)
93.6
94.5
97.4
95.2
WiSE, 0.7 tuned + 0.3 base
93.2
92.7
97.2
95.3
Table 4: Recall@10 (percent) for the variants we tried. Folding maps the Tajik letters , ӣ, ӯ (and capitals) to the Russian letters ч, и, у before encoding.
Why did the National Party and the Labour Party win the 1924 election?
Appendix
Table 5: Example test questions from the frozen law, tax and Wikipedia sets, exactly as written by the generator, without editing. The English glosses are ours. The law collection holds Tajik laws and codes, including the Tax Code; the tax collection holds tax-service orders, decrees and declaration forms. The full sets are available on request.
We release Saidzoda-Tajik-LFM-2.6B, a Tajik adaptation of the 2.6B-parameter hybrid model LFM2.5-2.6B-Base, built for tool calling in Tajik on modest hardware. The recipe has two stages: LoRA continued pretraining (CPT) on 1,485,319 cleaned Tajik web documents for 3 epochs, then one epoch of supervised fine-tuning (SFT) on a mix of machine-translated Tajik tool calls, English tool calls kept as replay, and Tajik instructions. On a 174-item tool-calling test set built from BFCL v3 and machine-translated to Tajik, strict full-call accuracy on Tajik prompts rises from 20.69 for the base model to 64.14 (paired difference +43.45 points, 95% interval +35.2 to +51.7); a lenient parser that also accepts the base model's own call formats puts the base at 31.72. English shows no detectable change (76.55 and 77.24). Every CPT checkpoint scores zero: CPT removes the native tool-call format and, as the lenient re-scoring shows, part of the skill itself. Outside tool calling, accuracy on a Tajik knowledge benchmark rises from 28.0 to 35.0, Belebele Tajik shows no significant change, and English MMLU falls from 60.8 to 52.5. The model is not a general Tajik assistant. It almost never declines when no tool fits (6.9% correct on Tajik, against 31.0% for the base under the lenient parser), it is weak at multi-turn calls, and its calls should be checked before they are executed. We describe the data, the training settings and every evaluation in enough detail to repeat them.
Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov +5
Saidzoda AI Lab, Saidzoda Engineering Company, Dushanbe, Tajikistan
We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, we perform Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, we introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and we open-source them on Hugging Face. Across these Tajik benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance on standard datasets. We further show that FP8 and INT4 quantization of Soro preserves most Tajik-language gains while reducing memory requirements for edge deployment, supporting an ongoing education-sector pilot and planned scale-out across schools in Tajikistan.
Stanislav Liashkov, Haitz Sáez de Ocáriz Borde, Azizjon Azimi +3
Kazakh, a Turkic language spoken by over 22 million people, remains underserved by existing multilingual language models, which allocate minimal capacity to low-resource languages and employ tokenizers ill-suited to agglutinative morphology. We present SozKZ, a family of Llama-architecture language models (50M-600M parameters) trained entirely from scratch on 9 billion tokens of Kazakh text with a dedicated 50K BPE tokenizer. We evaluate all models on three Kazakh benchmarks -- multiple-choice cultural QA, reading comprehension (Belebele), and topic classification (SIB-200) -- alongside five multilingual baselines ranging from 500M to 3B parameters. Our 600M model achieves 30.3% accuracy on Kazakh cultural QA, approaching the 32.0% of Llama-3.2-1B (2x larger), and 25.5% on SIB-200 topic classification, surpassing all evaluated multilingual models up to 2B parameters. We observe consistent scaling from 50M to 600M, with MC QA accuracy rising from 22.8% to 30.3%, suggesting that further scaling remains beneficial. These results demonstrate that small, dedicated models trained from scratch with a language-appropriate tokenizer offer a viable path for low-resource language technology, achieving competitive performance at a fraction of the computational cost. All models and the tokenizer are released under open licenses.