MITRA-lexicon: Lexica of attested translations from Sanskrit into English, Tibetan and Chinese
Authors: Sebastian Nehrdich, Kengo Harimoto, Sven Sellmer, Oliver Hellwig
Organizations: Center for Integrated Japanese Studies Tohoku University Sendai, Japan · Institute of Oriental Studies Adam Mickiewicz University Poznań, Poland · ISLE University of Zurich Zurich, Switzerland
The standard Sanskrit lexica are monumental and indispensable, but they cite their attestations as abbreviated references to editions, without links to the digitized texts now widely available and without a translation of the passage, and they naturally do not reflect translations made after them. To address these gaps, we present MITRA-lexicon, a family of corpus-grounded lexica based on span annotations of translation equivalents with Sanskrit as the pivot: Sanskrit-English, derived from the Mitrasamgraha corpus, as well as Sanskrit-Tibetan and Sanskrit-Chinese, derived from the MITRA-parallel corpus of Buddhist literature. In total, the resource comprises 2.18 million headword entries in six directional lexica. Headwords are lemmas and compounds from silver annotations that follow the Digital Corpus of Sanskrit (DCS). Senses are ranked by frequency, and each of the 10.4 million attestations includes its sentence pair, with character offsets of the word and its equivalent. We propose a multi-step pipeline based on grammatical annotation models, fine-tuned large language models, and a teacher-student setup to produce the alignments. We evaluate the lexica against reference dictionaries and manually corrected gold alignments. We release the lexica and the fine-tuned span annotation model, together with a web interface for looking up headwords and their attested senses.
Figures & tables
Figure 1: Two sentence pairs labeled by the teacher model (Section 3.4 ): the Sanskrit is segmented and lemmatized with ByT5-Sanskrit, and each content word is linked to the span of the translation that renders it, while function words such as ca and pronouns such as sa and vaḥ remain unlinked.
Pair
Corpus
Sentence pairs
Sanskrit words
Sanskrit–English
Mitrasaṃgraha
395,528
5.87M
Sanskrit–Tibetan
MITRA-parallel v3
566,068
5.57M
Sanskrit–Chinese
MITRA-parallel v3
340,691
4.48M
Table 1: Parallel corpora used for word alignment in this work. Sanskrit words are counted after segmentation with ByT5-Sanskrit, compound members counted separately.
Direction
Teacher calls
Verbatim
Words / sent.
Tagged
Seeds kept
Sanskrit–English
1,200
99.8%
13.8
52.2%
1,000
Sanskrit–Tibetan
3,542
99.6%
9.5
63.8%
2,000
Sanskrit–Chinese
2,518
96.0%
11.4
56.7%
2,000
Chinese–Tibetan (aux.)
2,522
99.2%
11.4
44.0%
2,000
English–Tibetan (aux.)
2,469
99.5%
17.8
27.1%
1,786
Table 2: Teacher labeling. “Tagged” is the mean fraction of source words that received a span. The two auxiliary directions are part of the training mixture of the student (Section 3.5 ) but their lexica are not the subject of this paper.
Direction
Sent. pairs
Verbatim
Ungrounded
Coverage
Links
Sanskrit–English
395,528
97.4%
0.20%
0.52
2,952,451
Sanskrit–Tibetan
566,068
95.9%
0.01%
0.70
3,634,841
Sanskrit–Chinese
340,691
98.8%
0.02%
0.50
1,942,754
Total
1,302,287
8,530,046
Table 3: Corpus-scale inference. Verbatim: share of outputs that reproduce the target exactly once the tags are removed. Ungrounded: mean share of the tagged spans of an output that do not occur in the target. Coverage: mean fraction of Sanskrit words per sentence that received a grounded span.
Direction
Word spans
Phrase spans
Total
Sanskrit–English
2,749,903
566,022
3,315,925
Sanskrit–Tibetan
3,634,045
1,090,654
4,724,699
Sanskrit–Chinese
1,942,753
532,475
2,475,228
Table 4: Spans extracted from the alignments. Phrase spans comprise full compounds and their sandhied surface forms.
Pair
Direction
Headwords
Senses
Senses / headword
Attestations
Sanskrit–English
sa → en
418,927
950,845
2.27
3,293,993
en → sa
396,289
2.40
Sanskrit–Tibetan
sa → bo
529,599
895,229
1.69
4,665,998
bo → sa
445,176
2.01
Sanskrit–Chinese
sa → zh
223,081
422,074
1.89
2,447,898
zh → sa
171,780
2.46
Table 5: The released lexica. Senses (headword–gloss pairs) and attestations are properties of a pair and identical in both of its directions. Headwords comprise lemmas as well as compound and phrase entries.
Attestations
1
2–4
5–19
20–99
100–999
≥ 1000
Total
Sanskrit–English
159,756
29,615
12,918
6,511
3,382
491
212,673
Sanskrit–Tibetan
124,176
60,734
17,776
6,397
3,159
614
212,856
Sanskrit–Chinese
42,567
29,254
10,120
4,041
2,008
350
88,340
Table 6: Single-word Sanskrit headwords by number of attestations. The remaining headwords of Table 5 are compound and surface entries.
Genre
Vedic
Epic
Classical
Medieval
Total
%
Veda
339,419
339,419
10.3
Epic
634,817
634,817
19.3
Purāṇa
1,187,279
1,187,279
36.0
Religious scripture
58,196
231,568
61,703
351,467
10.7
Treatises
34,261
530,746
14,993
580,000
17.6
Poetry
26,013
174,998
201,011
6.1
Table 7: Attestations of the Sanskrit–English lexicon, by genre and period of the Sanskrit text as reported in ? ). Periods are Vedic before 500 BCE, Epic 500 BCE–200 CE, Classical 200–1000 CE, Medieval 1000–1800 CE.
Sanskrit–Tibetan
Sanskrit–Chinese
Collection
Category
Attestations
%
Attestations
%
Scriptures
Sūtra
602,459
12.9
541,232
22.1
Prajñāpāramitā
708,552
15.2
536,847
21.9
Tantra
431,906
9.3
208,568
8.5
Vinaya
338,486
7.3
163,853
6.7
Other
126,426
2.7
308,665
12.6
Table 8: Attestations of the Sanskrit–Tibetan and Sanskrit–Chinese lexica by the DharmaNexus collection and category of the Sanskrit text. Other scriptures are Ratnakūṭa, Avataṃsaka (for Chinese 219,201 attestations) and miscellaneous sūtras. Other treatises include hymns, Prajñāpāramitā and sūtra commentaries, letters, Jātaka, Vinaya commentary and nīti.
Pair
Reference, matching
∣H∣
P@1
P@5
Att.-weighted prec.
Recall
Sanskrit–Tibetan
Negi, strict
24,858
58.2
71.4
76.6
46.0
Negi, containing
60.5
73.1
79.9
50.9
Negi, relaxed
68.0
78.6
85.7
60.9
Sanskrit–Chinese
Hirakawa, strict
15,783
53.0
69.4
75.2
44.1
Hirakawa, containing
57.2
72.1
80.4
47.8
Hirakawa, relaxed
68.4
80.3
88.3
63.4
Table 9: Agreement of the mined Sanskrit-source lexica with reference dictionaries (in %), over all single-word headwords that the reference can judge. Strict and containing figures are lower bounds on precision (see text).
Attestations of headword
1
2–4
5–19
20–99
100–999
≥ 1000
Sanskrit–Tibetan, P@1
45.4
50.6
56.9
71.9
84.3
90.0
Sanskrit–Tibetan, P@5
45.4
64.2
80.0
91.7
96.6
96.6
Sanskrit–Chinese, P@1
39.9
42.3
49.3
65.7
77.0
88.1
Sanskrit–Chinese, P@5
39.9
56.2
74.0
87.5
93.6
98.8
Sanskrit–English, P@1
20.4
24.8
38.1
54.4
75.0
90.2
Sanskrit–English, P@5
20.4
39.0
64.0
81.6
93.9
98.5
Table 10: Agreement by frequency of the headword (strict matching for Tibetan and Chinese; MW ∪ Apte for English). With a single attestation there is a single sense, so that P@1 and P@5 coincide.
Sense of the lexicon
31.8
Co-occurring in a sentence pair
28.0
Composable from entries
9.8
covered
69.7
Chinese not in our sentence pairs
7.5
Chinese not in the Chinese collection
7.0
Both in the corpus, never in one pair
14.1
Table 11: The 123,248 pairs of Hirakawa (Chinese of at least two characters) by the evidence of MITRA-lexicon (in %, each pair in the first row that applies).
DCS lemmas
MITRA-lexicon headwords, n≥20
Reference
≥ 1000
20–999
verbs
sa–en
sa–bo
sa–zh
Monier-Williams 1872
95.7
83.7
64.2
86.8
74.0
75.6
Apte 1890
93.6
63.4
54.3
71.3
61.4
63.6
MW ∪ Apte
96.4
86.2
65.4
89.1
76.0
77.3
Edgerton BHS
32.7
12.6
1.0
16.2
27.3
32.7
MW ∪ Apte ∪ BHS
96.9
86.8
65.5
89.7
82.7
84.6
Table 12: Lemma coverage of the reference dictionaries (in %): the share of DCS lemmas, by number of occurrences in the DCS corpus, and of the single-word headwords of MITRA-lexicon with at least 20 attestations, that have an entry in the reference after normalization to DCS-style stems. Headwords are restricted to lemmas produced by the analyzer, i.e. without the sandhied surface entries of compounds.
Link F1 vs. teacher labels
Expert gold
Aligner
sa–bo
sa–zh
sa–en
sa–en
fast_align
51.7
54.4
42.0
48.3
GIZA++
62.2
59.2
41.8
49.6
eflomal
62.8
61.6
46.6
53.9
Students fine-tuned on the same teacher labels
Qwen3.5-2B, MITRA pretraining
79.3
79.5
76.9
74.9
Table 13: Statistical aligners and students with different base models: link-level F1 against 1,489 held-out sentence pairs labeled by the teacher (Section 3.5 ). The students are trained on the same five-direction data with the same recipe (one epoch unless stated otherwise, full fine-tuning, learning rate 2⋅10−5 ) and differ only in the base model. † marks the checkpoint that aligned the respective corpus (Section 3.5 ). Last column: link F1 against the expert gold alignments of Section 6.3 .
P@1, n≥20
P@1, n≤4
Aligner
sa–bo
sa–zh
sa–en
sa–bo
sa–zh
sa–en
fast_align
53.0
51.4
59.5
9.1
7.5
11.4
GIZA++
58.0
49.0
57.2
8.7
4.8
4.8
eflomal
62.6
55.0
62.4
7.1
7.1
14.3
Student (this paper)
77.5
71.1
63.9
45.1
39.1
22.8
Table 14: Strict P@1 against Negi, Hirakawa and Monier-Williams ∪ Apte (Section 6.1 ) for lexica built from each aligner’s links, pooled over headwords with at least 20 and at most 4 attestations ( n ).
Span F1
Link F1
Pair
Sent. pairs
Teacher
Student
Teacher
Student
Sanskrit–Tibetan
47
88.1
83.5
94.5
90.9
Sanskrit–Chinese
65
93.8
90.3
96.6
93.3
Sanskrit–English
43
97.0
89.5
96.6
89.0
Table 15: Agreement with the held-out teacher labels on a subset of 155 sentence pairs: a second run of the teacher under identical settings, and the student (the checkpoint that aligned the respective corpus), both scored on the same sentences. Span F1 as in Section 3.5 , link F1 as in Table 13 .
Gold
Sent.
Units
P
R
F1
Unch.
Annotator A
100
1,444
91.3
80.6
85.6
84.0
Annotator B
100
1,444
89.0
69.7
78.2
74.4
Annotator C
100
1,444
89.4
87.5
88.4
87.8
Majority (2 of 3)
100
1,404
92.3
84.8
88.4
87.0
All three agree
100
1,037
95.7
94.9
95.3
93.8
Majority, by unit kind
Table 16: Sanskrit–English alignments of the model against the expert gold alignments of the 100 sentences of the agreement study: precision, recall and F1 of the spans under exact match, and the share of units the annotator left unchanged. “Majority” takes the decision of at least two of the three annotators (units without one are left out), “all three agree” only the units on which the three annotations are identical.
We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a sloka-translation pair, formalizing the traditional patha commentary practice as an evaluable NLP objective. We construct a benchmark of 31,316 sloka-translation-glossary triples from the Valmiki Ramayana and Srimad Bhagavatam, paired with two metrics: Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Across zero-shot, few-shot, and instruction fine-tuned variants of Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B, instruction fine-tuning substantially outperforms prompting, while explicit segmentation yields gains. Error analysis identifies over-segmentation of sandhi and samasa compounds as the dominant failure mode, pointing to morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition.
Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal
We release Samasāmayik, a novel, meticulously curated, large-scale Hindi-Sanskrit corpus, comprising 92,196 parallel sentences. Unlike most data available in Sanskrit, which focuses on classical era text and poetry, this corpus aggregates data from diverse sources covering contemporary materials, including spoken tutorials, children's magazines, radio conversations, and instruction materials. We benchmark this new dataset by fine-tuning three complementary models - ByT5, NLLB and IndicTrans-v2, to demonstrate its utility. Our experiments demonstrate that models trained on the Samasamayik corpus achieve significant performance gains on in-domain test data, while achieving comparable performance on other widely used test sets, establishing a strong new performance baseline for contemporary Hindi-Sanskrit translation. Furthermore, a comparative analysis against existing corpora reveals minimal semantic and lexical overlap, confirming the novelty and non-redundancy of our dataset as a robust new resource for low-resource Indic language MT.
N J Karthika, Keerthana Suryanarayanan, Jahanvi Purohit +3
Indian Institute of Technology Bombay · Geakminds Technologies Private Limited · Indian Institute of Technology Roorkee
We present a computational stylometric analysis of the Tipitaka across all three Pitakas in English translation, extending earlier work on the Sutta Pitaka alone. The corpus spans 134,831 segments from Bhikkhu Sujato's Sutta Pitaka (114,591 segments, CC0), Bhikkhu Brahmali's Vinaya Pitaka (7,923 segments, CC0 2026), I.B. Horner's 1938 Vinaya translation (2,826 segments), three English translations of the Abhidhammattha Sangaha compendium (2,077 segments), and cross-tradition Vinaya texts from the Dharmaguptaka and Mulasarvastivada schools. We compute Zipf rank-frequency distributions with OLS-fitted exponents, Moving Average TTR (MATTR-500), numeral-word density, and vocabulary overlap (Jaccard and Szymkiewicz-Simpson coefficients). Main findings: (1) all corpora show Zipf-consistent distributions (R2 > 0.989); the Vinaya is closest to ideal Zipf slope -1 and the Sangaha corpus deviates most, with 'consciousness' displacing grammatical particles at rank 8; (2) MATTR-500 shows the Sutta and Vinaya Theravada are nearly identical in lexical diversity (0.399 and 0.400), while the Sangaha corpus is genuinely more diverse (0.560), confirmed by size-controlled subsampling; (3) the Sangaha corpus has the highest numeral-word density (3.26%), consistent with its systematic enumeration of mental and material categories; (4) the Mulasarvastivada Vinaya shares 20.0% vocabulary (Jaccard) and 49.1% (overlap coefficient) with the Theravada Vinaya, reflecting shared legal heritage across two millennia; (5) two English translations of the same Vinaya source text share only 24.2% of their vocabulary across 88 years, with 'musing' versus 'absorption' for jhana and 'defeat' versus 'expulsion' for parajika as the most diagnostic shifts. All results are point estimates; no significance testing is conducted. Code and data are released as open-source extensions to the Darshana Graph corpus (arXiv:2606.18222).