Organizations: Indian Institute of Technology Kharagpur, India · Central Sanskrit University, Devprayag, India · Manipal Academy of Higher Education, Manipal, India
Annotation schemas are not neutral. When applied to classical literature, tag sets developed for modern journalistic texts impose source-culture definitions on texts they were never designed to describe. We instead ground a schema in the tradition of the text itself introducing Padārtha, the first ontology-grounded fine-grained Named Entity Recognition (NER) benchmark for Sanskrit, built on the Mahābhārata epic. Our tag set derives from Nyāya-Vaiśesika, a classical Indian ontological system, yielding 18 fine-grained categories organized under 10 ontological nodes and mapped onto five standard coarse tags, ensuring interoperability with existing benchmarks. Expert annotators label over 12.6K entries from a scholarly index of named entities, linked to corresponding mentions in the Mahānāma corpus, producing fine-grained annotations for 108,335 entity mentions across 73,632 verses, along with a 5,000-verse expert-verified test set sampled to stress rare mentions. We present the first systematic benchmarking of generative NER against traditional architectures for Sanskrit, finding that fine-tuned generative models perform comparably to task-specific systems. However, all systems show a sharp decline from coarse to fine granularity and struggle with out-of-entity mentions unseen during training. The limitation is not due to data scarcity alone, as fine-tuned models recall unseen entities far worse than seen ones and tend to default to the majority sense under lexical ambiguity.
Figures & tables
SL
Coarse
Fine Tag
Gloss
Example
Sørensen Description
1
Person
īśvaraḥ
The Trimūrti (Trinity)
Śiva
Great god (Mahādeva)
2
Person
devatā
Gods
Indra
King of the gods
3
Person
ṛṣiḥ
Sages
Vasiṣṭha
Celebrated celestial sage
4
Person
devayoniḥ
Divine-origin beings
Citraratha
King of the Gandharvas
5
Person
manuṣyaḥ
Humans
Arjuna
Third Pāṇḍava prince
6
Person
jantuḥ
Animals
Sugrīva
One of Kṛṣṇa’s horses
Table 1: Complete coarse and mapped fine-grained Nyāya-Vaiśeṣika tagset, featuring examples and their canonical descriptions from the book Index to the Names in the Mahābhārata Sørensen (1904) .
Figure 1: Nyāya-Vaiśeṣika ontology hierarchy underlying the tagset. Leaf nodes correspond to fine-grained categories (glosses and examples in Table 1 ). Color indicates standard coarse category (legend).
Format
Used By
Example Output (Mahābhārata CE Volume 1 Chapter 1 Verse 2)
BIO Tags
CRF, MuRIL, ByT5
loma ( B-PER ) harṣaṇa ( I-PER ) putra ( I-PER ), ugraśravāḥ ( B-PER ), sautiḥ ( B-PER ), paurāṇikā ( B-PER ), u ( O ), naimiṣa ( B-LOC ), araṇye ( O ), śaunakasya ( B-PER ) …
Inline Bracketed
Qwen2.5-7B, Gemma-4-12B (fine-tuning)
[ loma harṣaṇa putra ∣ PER ] [ ugraśravāḥ ∣ PER ] [ sautiḥ ∣ PER ] [ paurāṇikā ∣ PER ] u [ naimiṣa ∣ LOC ] araṇye [ śaunakasya ∣ PER ] …
Table 2: Example verse and output representation formats by model and training paradigm, shown on the same example verse (“…sage Lomaharṣaṇa’s son Ugraśravas [=Sauti], well-read in the Purāṇas, [came to] the Naimiṣa forest, [during] Śaunaka’s sacrifice…”). All formats are converted to a unified BIO representation for scoring. In the Two-Stage Pipeline row, @@ … ## mark candidate mentions from the detection stage, following the cascade design of Luo et al. (2025) .
Level
Agreement (%)
κ
Coarse label
91.03
0.728
Fine label
79.71
0.746
Span detection
F1 = 0.811
Table 3: Annotation reliability between the primary annotator and a second reviewer, over 1000 doubly annotated test verses.
Category
Count
Category
Count
PERSON
88,930
MISC
4,537
NORP
9,416
LOCATION
4,798
TIME
654
Total
108,335 mentions
Table 4: Distribution of coarse-grained entity categories.
Tag
Count
Tag
Count
manuṣyaḥ
51,969
devatā
11,149
īśvaraḥ
11,271
samūhaḥ
9,416
ṛṣiḥ
6,424
devayoniḥ
6,022
prākṛtikasthānam
3,284
śabdaḥ
3,132
alaukikaprāṇī
1,939
alaukika_acalanirjīvavastu
856
alaukikasthānam
806
janapadaḥ
690
Table 5: Distribution of fine-grained entity categories.
Figure 2: Rank-frequency distribution of entity mentions
Methodology
Architecture
Model
Output Format
Micro Average (%)
Macro Average (%)
P
R
F1
P
R
F1
Coarse-Grained
Baseline
–
CRF
BIO
78.83
40.05
53.12
74.46
32.08
44.37
Fine-tuning
Encoder
MuRIL
BIO
79.84
52.09
63.04
75.46
48.77
58.56
Enc-Dec
ByT5
BIO
78.96
51.85
62.60
72.53
44.96
54.63
Decoder
Qwen-2.5-7B
In. Bracketed
79.38
53.37
63.83
58.75
41.75
48.63
Table 6: Precision, Recall, and F1 for all models at coarse-grained and fine-grained granularity, grouped by training paradigm (fine-tuning vs. few-shot) and architecture. Bold marks the best score in each column within a granularity.
The digitisation of classical Sanskrit literature is impeded by a scarcity of annotated resources, particularly for Named Entity Recognition. While recent methodologies utilise generic Large Language Models (LLMs) for data augmentation, these approaches remain prone to error and often lack the reasoning depth required for classical grammar. In this work, we introduce Naamah, a high quality silver standard Sanskrit NER dataset comprising 102,942 sentences. We propose a methodology that combines entity extraction from DBpedia with the generative capabilities of a 24B parameter hybrid reasoning model to create grammatically natural and synthetically diverse training data. We utilize this dataset to benchmark two transformer architectures: the massive multilingual XLM RoBERTa and the parameter efficient IndicBERTv2.
Akhil Rajeev P, Annarao Kulkarni
Centre for Development of Advanced Computing (C-DAC), Bangalore
We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a sloka-translation pair, formalizing the traditional patha commentary practice as an evaluable NLP objective. We construct a benchmark of 31,316 sloka-translation-glossary triples from the Valmiki Ramayana and Srimad Bhagavatam, paired with two metrics: Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Across zero-shot, few-shot, and instruction fine-tuned variants of Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B, instruction fine-tuning substantially outperforms prompting, while explicit segmentation yields gains. Error analysis identifies over-segmentation of sandhi and samasa compounds as the dominant failure mode, pointing to morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition.
Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal
Language is humanity's most consequential technology, yet for over a billion speakers across India's twenty-two constitutionally recognised languages, its digital layer remains structurally incomplete. Named Entity Recognition (NER), the foundational step in transforming raw text into machine-interpretable knowledge, has been studied exhaustively for English but remains largely unsolved across most Indic languages. This paper presents a rigorous comparative study of generative and encoder-based neural architectures for NER on all eleven languages of the Naamapadam benchmark. We evaluate five classic model families spanning sequence-to-sequence transformers and multilingual encoders; four decoder-only large language models (LLMs) fine-tuned with LoRA and 4-bit NF4 quantisation; and nine generative models in zero-to-5-shot inference. Under strict CoNLL span-level evaluation, encoder-based models (mBERT and XLM-R, both F1=0.675 on Hindi) substantially outperform every generative architecture in ten of eleven languages, with gaps of 7.5-40 percentage points against the strongest competitor (Gemma-2-2B: avg F1=0.427). The best few-shot result reaches only 28% of the encoder baseline. We identify three language clusters--encoder-dominant, partial-coverage, and failure-zone; and provide actionable deployment guidelines grounded in transfer learning and low-resource NLP principles.