MITRA-lexicon: Lexica of attested translations from Sanskrit into English, Tibetan and Chinese

Sebastian Nehrdich; Kengo Harimoto; Sven Sellmer; Oliver Hellwig

cs.CLOct 19, 2026

MITRA-lexicon: Lexica of attested translations from Sanskrit into English, Tibetan and Chinese

Authors: Sebastian Nehrdich, Kengo Harimoto, Sven Sellmer, Oliver Hellwig

Organizations: Center for Integrated Japanese Studies Tohoku University Sendai, Japan · Institute of Oriental Studies Adam Mickiewicz University Poznań, Poland · ISLE University of Zurich Zurich, Switzerland

Abstract

The standard Sanskrit lexica are monumental and indispensable, but they cite their attestations as abbreviated references to editions, without links to the digitized texts now widely available and without a translation of the passage, and they naturally do not reflect translations made after them. To address these gaps, we present MITRA-lexicon, a family of corpus-grounded lexica based on span annotations of translation equivalents with Sanskrit as the pivot: Sanskrit-English, derived from the Mitrasamgraha corpus, as well as Sanskrit-Tibetan and Sanskrit-Chinese, derived from the MITRA-parallel corpus of Buddhist literature. In total, the resource comprises 2.18 million headword entries in six directional lexica. Headwords are lemmas and compounds from silver annotations that follow the Digital Corpus of Sanskrit (DCS). Senses are ranked by frequency, and each of the 10.4 million attestations includes its sentence pair, with character offsets of the word and its equivalent. We propose a multi-step pipeline based on grammatical annotation models, fine-tuned large language models, and a teacher-student setup to produce the alignments. We evaluate the lexica against reference dictionaries and manually corrected gold alignments. We release the lexica and the fine-tuned span annotation model, together with a web interface for looking up headwords and their attested senses.

Figures & tables

Panel of Figure 1 — Figure 1: Two sentence pairs labeled by the teacher model (Section 3.4 ): the Sanskrit is segmented and lemmatized with ByT5-Sanskrit, and each content word is linked to the span of the translation that renders it, while function words such as ca and pronouns such as sa and vaḥ remain unlinked.

Pair	Corpus	Sentence pairs	Sanskrit words
Sanskrit–English	Mitrasaṃgraha	395,528	5.87M
Sanskrit–Tibetan	MITRA-parallel v3	566,068	5.57M
Sanskrit–Chinese	MITRA-parallel v3	340,691	4.48M

Table 1: Parallel corpora used for word alignment in this work. Sanskrit words are counted after segmentation with ByT5-Sanskrit, compound members counted separately.

Direction	Teacher calls	Verbatim	Words / sent.	Tagged	Seeds kept
Sanskrit–English	1,200	99.8%	13.8	52.2%	1,000
Sanskrit–Tibetan	3,542	99.6%	9.5	63.8%	2,000
Sanskrit–Chinese	2,518	96.0%	11.4	56.7%	2,000
Chinese–Tibetan (aux.)	2,522	99.2%	11.4	44.0%	2,000
English–Tibetan (aux.)	2,469	99.5%	17.8	27.1%	1,786

Table 2: Teacher labeling. “Tagged” is the mean fraction of source words that received a span. The two auxiliary directions are part of the training mixture of the student (Section 3.5 ) but their lexica are not the subject of this paper.

Direction	Sent. pairs	Verbatim	Ungrounded	Coverage	Links
Sanskrit–English	395,528	97.4%	0.20%	0.52	2,952,451
Sanskrit–Tibetan	566,068	95.9%	0.01%	0.70	3,634,841
Sanskrit–Chinese	340,691	98.8%	0.02%	0.50	1,942,754
Total	1,302,287				8,530,046

Table 3: Corpus-scale inference. Verbatim: share of outputs that reproduce the target exactly once the tags are removed. Ungrounded: mean share of the tagged spans of an output that do not occur in the target. Coverage: mean fraction of Sanskrit words per sentence that received a grounded span.

Direction	Word spans	Phrase spans	Total
Sanskrit–English	2,749,903	566,022	3,315,925
Sanskrit–Tibetan	3,634,045	1,090,654	4,724,699
Sanskrit–Chinese	1,942,753	532,475	2,475,228

Table 4: Spans extracted from the alignments. Phrase spans comprise full compounds and their sandhied surface forms.

Pair	Direction	Headwords	Senses	Senses / headword	Attestations
Sanskrit–English	sa $\rightarrow$ en	418,927	950,845	2.27	3,293,993
	en $\rightarrow$ sa	396,289		2.40
Sanskrit–Tibetan	sa $\rightarrow$ bo	529,599	895,229	1.69	4,665,998
	bo $\rightarrow$ sa	445,176		2.01
Sanskrit–Chinese	sa $\rightarrow$ zh	223,081	422,074	1.89	2,447,898
	zh $\rightarrow$ sa	171,780		2.46

Table 5: The released lexica. Senses (headword–gloss pairs) and attestations are properties of a pair and identical in both of its directions. Headwords comprise lemmas as well as compound and phrase entries.

Attestations	1	2–4	5–19	20–99	100–999	$\geq$ 1000	Total
Sanskrit–English	159,756	29,615	12,918	6,511	3,382	491	212,673
Sanskrit–Tibetan	124,176	60,734	17,776	6,397	3,159	614	212,856
Sanskrit–Chinese	42,567	29,254	10,120	4,041	2,008	350	88,340

Table 6: Single-word Sanskrit headwords by number of attestations. The remaining headwords of Table 5 are compound and surface entries.

Genre	Vedic	Epic	Classical	Medieval	Total	%
Veda	339,419				339,419	10.3
Epic		634,817			634,817	19.3
Purāṇa			1,187,279		1,187,279	36.0
Religious scripture		58,196	231,568	61,703	351,467	10.7
Treatises		34,261	530,746	14,993	580,000	17.6
Poetry			26,013	174,998	201,011	6.1

Table 7: Attestations of the Sanskrit–English lexicon, by genre and period of the Sanskrit text as reported in ? ). Periods are Vedic before 500 BCE, Epic 500 BCE–200 CE, Classical 200–1000 CE, Medieval 1000–1800 CE.

		Sanskrit–Tibetan		Sanskrit–Chinese
Collection	Category	Attestations	%	Attestations	%
Scriptures	Sūtra	602,459	12.9	541,232	22.1
	Prajñāpāramitā	708,552	15.2	536,847	21.9
	Tantra	431,906	9.3	208,568	8.5
	Vinaya	338,486	7.3	163,853	6.7
	Other	126,426	2.7	308,665	12.6

Table 8: Attestations of the Sanskrit–Tibetan and Sanskrit–Chinese lexica by the DharmaNexus collection and category of the Sanskrit text. Other scriptures are Ratnakūṭa, Avataṃsaka (for Chinese 219,201 attestations) and miscellaneous sūtras. Other treatises include hymns, Prajñāpāramitā and sūtra commentaries, letters, Jātaka, Vinaya commentary and nīti.

Pair	Reference, matching	$\|H\|$	P@1	P@5	Att.-weighted prec.	Recall
Sanskrit–Tibetan	Negi, strict	24,858	58.2	71.4	76.6	46.0
	Negi, containing		60.5	73.1	79.9	50.9
	Negi, relaxed		68.0	78.6	85.7	60.9
Sanskrit–Chinese	Hirakawa, strict	15,783	53.0	69.4	75.2	44.1
	Hirakawa, containing		57.2	72.1	80.4	47.8
	Hirakawa, relaxed		68.4	80.3	88.3	63.4

Table 9: Agreement of the mined Sanskrit-source lexica with reference dictionaries (in %), over all single-word headwords that the reference can judge. Strict and containing figures are lower bounds on precision (see text).

Attestations of headword	1	2–4	5–19	20–99	100–999	$\geq$ 1000
Sanskrit–Tibetan, P@1	45.4	50.6	56.9	71.9	84.3	90.0
Sanskrit–Tibetan, P@5	45.4	64.2	80.0	91.7	96.6	96.6
Sanskrit–Chinese, P@1	39.9	42.3	49.3	65.7	77.0	88.1
Sanskrit–Chinese, P@5	39.9	56.2	74.0	87.5	93.6	98.8
Sanskrit–English, P@1	20.4	24.8	38.1	54.4	75.0	90.2
Sanskrit–English, P@5	20.4	39.0	64.0	81.6	93.9	98.5

Table 10: Agreement by frequency of the headword (strict matching for Tibetan and Chinese; MW

\cup

Apte for English). With a single attestation there is a single sense, so that P@1 and P@5 coincide.

Sense of the lexicon	31.8
Co-occurring in a sentence pair	28.0
Composable from entries	9.8
covered	69.7
Chinese not in our sentence pairs	7.5
Chinese not in the Chinese collection	7.0
Both in the corpus, never in one pair	14.1

Table 11: The 123,248 pairs of Hirakawa (Chinese of at least two characters) by the evidence of MITRA-lexicon (in %, each pair in the first row that applies).

	DCS lemmas			MITRA-lexicon headwords, $n\geq 20$
Reference	$\geq$ 1000	20–999	verbs	sa–en	sa–bo	sa–zh
Monier-Williams 1872	95.7	83.7	64.2	86.8	74.0	75.6
Apte 1890	93.6	63.4	54.3	71.3	61.4	63.6
MW $\cup$ Apte	96.4	86.2	65.4	89.1	76.0	77.3
Edgerton BHS	32.7	12.6	1.0	16.2	27.3	32.7
MW $\cup$ Apte $\cup$ BHS	96.9	86.8	65.5	89.7	82.7	84.6

Table 12: Lemma coverage of the reference dictionaries (in %): the share of DCS lemmas, by number of occurrences in the DCS corpus, and of the single-word headwords of MITRA-lexicon with at least 20 attestations, that have an entry in the reference after normalization to DCS-style stems. Headwords are restricted to lemmas produced by the analyzer, i.e. without the sandhied surface entries of compounds.

Students fine-tuned on the same teacher labels
	Link $F_{1}$ vs. teacher labels			Expert gold
Aligner	sa–bo	sa–zh	sa–en	sa–en
fast_align	51.7	54.4	42.0	48.3
GIZA++	62.2	59.2	41.8	49.6
eflomal	62.8	61.6	46.6	53.9
Qwen3.5-2B, MITRA pretraining	79.3	79.5	76.9	74.9

Table 13: Statistical aligners and students with different base models: link-level

F_{1}

against 1,489 held-out sentence pairs labeled by the teacher (Section 3.5 ). The students are trained on the same five-direction data with the same recipe (one epoch unless stated otherwise, full fine-tuning, learning rate

2\cdot 10^{-5}

) and differ only in the base model. † marks the checkpoint that aligned the respective corpus (Section 3.5 ). Last column: link

F_{1}

against the expert gold alignments of Section 6.3 .

	P@1, $n\geq 20$			P@1, $n\leq 4$
Aligner	sa–bo	sa–zh	sa–en	sa–bo	sa–zh	sa–en
fast_align	53.0	51.4	59.5	9.1	7.5	11.4
GIZA++	58.0	49.0	57.2	8.7	4.8	4.8
eflomal	62.6	55.0	62.4	7.1	7.1	14.3
Student (this paper)	77.5	71.1	63.9	45.1	39.1	22.8

Table 14: Strict P@1 against Negi, Hirakawa and Monier-Williams

\cup

Apte (Section 6.1 ) for lexica built from each aligner’s links, pooled over headwords with at least 20 and at most 4 attestations (

n

		Span $F_{1}$		Link $F_{1}$
Pair	Sent. pairs	Teacher	Student	Teacher	Student
Sanskrit–Tibetan	47	88.1	83.5	94.5	90.9
Sanskrit–Chinese	65	93.8	90.3	96.6	93.3
Sanskrit–English	43	97.0	89.5	96.6	89.0

Table 15: Agreement with the held-out teacher labels on a subset of 155 sentence pairs: a second run of the teacher under identical settings, and the student (the checkpoint that aligned the respective corpus), both scored on the same sentences. Span

F_{1}

as in Section 3.5 , link

F_{1}

as in Table 13 .

Majority, by unit kind
Gold	Sent.	Units	P	R	$F_{1}$	Unch.
Annotator A	100	1,444	91.3	80.6	85.6	84.0
Annotator B	100	1,444	89.0	69.7	78.2	74.4
Annotator C	100	1,444	89.4	87.5	88.4	87.8
Majority (2 of 3)	100	1,404	92.3	84.8	88.4	87.0
All three agree	100	1,037	95.7	94.9	95.3	93.8

Table 16: Sanskrit–English alignments of the model against the expert gold alignments of the 100 sentences of the agreement study: precision, recall and

F_{1}

of the spans under exact match, and the share of units the annotator left unchanged. “Majority” takes the decision of at least two of the three annotators (units without one are left out), “all three agree” only the units on which the three annotations are identical.

Padamitra: Grounded Glossary Generation for Classical Sanskrit
Aug 25, 2026Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan GoyalLLM Grounding Low-Resource Language Processing
Indian Institute of Technology, Kharagpur
Samasāmayik: A Parallel Dataset for Hindi-Sanskrit Machine Translation
Mar 25, 2026N J Karthika, Keerthana Suryanarayanan, Jahanvi Purohit +3Machine Translation Low-Resource MT
Indian Institute of Technology Bombay · Geakminds Technologies Private Limited · Indian Institute of Technology Roorkee
Three Buddhist Vocabularies: Computational Stylometry of the English Pali Canon across Sutta, Vinaya, and Abhidhamma
Jun 24, 2026Joy BoseComputational Literary Studies Zipf's Law
Independent Researcher, Bengaluru, India

Aug 25, 2026J/K move · Enter open · S save

Padamitra: Grounded Glossary Generation for Classical Sanskrit

Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal

Indian Institute of Technology, Kharagpur

LLM Grounding Low-Resource Language Processing

Open paper AskPDF arXiv

We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a sloka-translation pair, formalizing the traditional patha commentary practice as an evaluable NLP objective. We construct a benchmark of 31,316 sloka-translation-glossary triples from the Valmiki Ramayana and Srimad Bhagavatam, paired with two metrics: Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Across zero-shot, few-shot, and instruction fine-tuned variants of Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B, instruction fine-tuning substantially outperforms prompting, while explicit segmentation yields gains. Error analysis identifies over-segmentation of sandhi and samasa compounds as the dominant failure mode, pointing to morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition.

Abstract

Explore similar work

Padamitra: Grounded Glossary Generation for Classical Sanskrit

Samasāmayik: A Parallel Dataset for Hindi-Sanskrit Machine Translation

Three Buddhist Vocabularies: Computational Stylometry of the English Pali Canon across Sutta, Vinaya, and Abhidhamma