Saidzoda-Tajik-EmbeddingGemma-300M: Knowing Every Letter Is Not Reading Tajik
Organizations: Saidzoda AI Lab, Saidzoda Engineering Company, Dushanbe, Tajikistan
Abstract
EmbeddingGemma-300M is a small open embedder whose tokenizer represents every Tajik letter, and as released it is weak at Tajik retrieval: on our Tajik law test the right passage is in its top ten results for 49.3% of questions, on tax documents for 45.9%, and on the human-written Belebele Tajik set for 69.8%, against 98.6% on the Russian version of the same set. We present Saidzoda-Tajik-EmbeddingGemma-300M, the same model fully fine-tuned for one epoch on 235,140 question and passage pairs written by our own Tajik language model for laws, tax documents, Wikipedia and web text, with the same mined hard negatives, English and Russian replay, loss, global batch, learning rate and epoch as our earlier Saidzoda-Tajik-E5-Large (arXiv:2611.00592) and BGE-M3 models. After training, law reaches 92.4% and tax 91.7%; on Belebele, the one test written by people, recall@10 rises from 69.8% to 92.6% and the first result is right for 76.6% of questions instead of 51.7%. A checkpoint at step 100 of 466 already holds 88% of the law gain at recall@10. At recall@10 the result is no more than 2.8 points below the two 560M models trained on the same pairs on law, tax and Wikipedia, within the intervals, with 768-dimensional vectors and about half their parameters; at recall@1, for which we have no intervals, it stays 7.0 points behind the E5 model on Belebele. The cost is a drop of 3.0 and 2.8 points in Russian and English recall@1, larger than for the two earlier models. We also report that, according to our run notes, a bfloat16 test run produced a non-finite loss at step 3 which the training framework logged as zero, so training was done in fp32. Our generated test questions come from the same model as the training questions, so we treat those scores as an upper limit until questions written by Tajik speakers exist.
Figures & tables
| Domain | Pairs | With negative |
|---|---|---|
| Tajik law | 52,758 | 40,259 |
| Tajik tax | 1,033 | 875 |
| Tajik Wikipedia | 21,123 | 20,374 |
| Tajik web and literature | 59,985 | 39,140 |
| Customer service (synthetic) | 15,744 | 6,313 |
| English replay (NQ) | 49,987 | n/a |
| Test set (queries) | EmbeddingGemma-300M | Ours | Change |
|---|---|---|---|
| Law (1,000) | 49.3 [46.2, 52.3] | 92.4 [90.6, 94.0] | +43.1 |
| Tax (109) | 45.9 [36.7, 55.0] | 91.7 [86.2, 97.2] | +45.9 |
| Wikipedia (500) | 89.6 [86.8, 92.2] | 97.2 [95.8, 98.6] | +7.6 |
| Belebele Tajik (900) | 69.8 [66.7, 72.8] | 92.6 [90.9, 94.2] | +22.8 |
| Belebele Russian (900) | 98.6 [97.7, 99.3] | 97.9 [96.9, 98.8] | -0.7 |
| Belebele English (900) | 99.2 [98.7, 99.8] | 98.1 [97.1, 98.9] | -1.1 |
| Model | Size | Used as | Law | Tax | Wiki | Bel. tg | Bel. ru | Bel. en |
|---|---|---|---|---|---|---|---|---|
| EmbeddingGemma-300M | 0.31B | zero-shot | 49.3 | 45.9 | 89.6 | 69.8 | 98.6 | 99.2 |
| Ours, step 100 | 0.31B | full FT | 87.4 | n/a | n/a | 87.7 | n/a | n/a |
| Ours (EmbeddingGemma, fine-tuned) | 0.31B | full FT | 92.4 | 91.7 | 97.2 | 92.6 | 97.9 | 98.1 |
| BGE-M3 | 0.57B | zero-shot | 77.6 | 78.9 | 95.0 | 83.8 | 98.9 | 99.0 |
| Saidzoda-Tajik-BGE-M3 | 0.57B | full FT | 91.6 | 92.7 | 97.2 | 92.2 | 98.1 | 98.1 |
| multilingual-e5-large-instruct | 0.56B | zero-shot | 80.2 | 82.6 | 96.4 | 92.2 | 97.9 | 98.9 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Set | Question (Tajik) | English gloss |
|---|---|---|
| Law | Мувофии моддаи 43-и Кодекси андози Тоикистон, объектои муоинаи хронометражӣ кадомоянд? | Under Article 43 of the Tax Code of Tajikistan, what are the objects of a timekeeping inspection? |
| Law | Тиби Моддаи 346-и Кодекси андози умурии Тоикистон, манбаи андоз чӣ тавр муайян карда мешавад? | Under Article 346 of the Tax Code, how is the tax base determined? |
| Tax | Муаррароти Кодекси одоби касбии маомоти андоз ба кадом категорияи кормандон татби мегардад? | To which categories of staff do the rules of the tax authorities’ code of professional ethics apply? |
| Tax | Корманди андоз бояд чӣ гуна амал кунад, то ба ошкоркунии сирри давлатӣ ро надиад? | How must a tax officer act so as not to disclose state secrets? |
| Wikipedia | Дар замони Имперотурии Юан окимияти имперотурӣ дар ар як вилоят аз ониби кадом намояндагон ташкил карда мешуд? | In the Yuan Empire, which representatives exercised imperial authority in each province? |
| Wikipedia | Чаро дар интихоботи соли 1924 изби Миллӣ ва изби Менатӣ пирӯз шуданд? | Why did the National Party and the Labour Party win the 1924 election? |
Text Embedding Models
Text embedding models map words, sentences, or documents to vectors for semantic search, retrieval, clustering, and downstream prediction.
See all 136 papers