Padārtha: Ontology-Grounded Fine-Grained NER Benchmark for Classical Sanskrit

Sujoy Sarkar; Pretam Ray; Paramhans Shah; Manoj Balaji Jagadeeshan; Akash Gairola; Arjuna S R; Pawan Goyal

cs.CLAug 29, 2026

Padārtha: Ontology-Grounded Fine-Grained NER Benchmark for Classical Sanskrit

Authors: Sujoy Sarkar, Pretam Ray, Paramhans Shah, Manoj Balaji Jagadeeshan, Akash Gairola, Arjuna S R, Pawan Goyal

Organizations: Indian Institute of Technology Kharagpur, India · Central Sanskrit University, Devprayag, India · Manipal Academy of Higher Education, Manipal, India

Abstract

Annotation schemas are not neutral. When applied to classical literature, tag sets developed for modern journalistic texts impose source-culture definitions on texts they were never designed to describe. We instead ground a schema in the tradition of the text itself introducing Padārtha, the first ontology-grounded fine-grained Named Entity Recognition (NER) benchmark for Sanskrit, built on the Mahābhārata epic. Our tag set derives from Nyāya-Vaiśesika, a classical Indian ontological system, yielding 18 fine-grained categories organized under 10 ontological nodes and mapped onto five standard coarse tags, ensuring interoperability with existing benchmarks. Expert annotators label over 12.6K entries from a scholarly index of named entities, linked to corresponding mentions in the Mahānāma corpus, producing fine-grained annotations for 108,335 entity mentions across 73,632 verses, along with a 5,000-verse expert-verified test set sampled to stress rare mentions. We present the first systematic benchmarking of generative NER against traditional architectures for Sanskrit, finding that fine-tuned generative models perform comparably to task-specific systems. However, all systems show a sharp decline from coarse to fine granularity and struggle with out-of-entity mentions unseen during training. The limitation is not due to data scarcity alone, as fine-tuned models recall unseen entities far worse than seen ones and tend to default to the majority sense under lexical ambiguity.

Figures & tables

SL	Coarse	Fine Tag	Gloss	Example	Sørensen Description
1	Person	īśvaraḥ	The Trimūrti (Trinity)	Śiva	Great god (Mahādeva)
2	Person	devatā	Gods	Indra	King of the gods
3	Person	ṛṣiḥ	Sages	Vasiṣṭha	Celebrated celestial sage
4	Person	devayoniḥ	Divine-origin beings	Citraratha	King of the Gandharvas
5	Person	manuṣyaḥ	Humans	Arjuna	Third Pāṇḍava prince
6	Person	jantuḥ	Animals	Sugrīva	One of Kṛṣṇa’s horses

Table 1: Complete coarse and mapped fine-grained Nyāya-Vaiśeṣika tagset, featuring examples and their canonical descriptions from the book Index to the Names in the Mahābhārata Sørensen (1904) .

Figure 1: Nyāya-Vaiśeṣika ontology hierarchy underlying the tagset. Leaf nodes correspond to fine-grained categories (glosses and examples in Table 1 ). Color indicates standard coarse category (legend).

Format	Used By	Example Output (Mahābhārata CE Volume 1 Chapter 1 Verse 2)
BIO Tags	CRF, MuRIL, ByT5	loma ( B-PER ) harṣaṇa ( I-PER ) putra ( I-PER ), ugraśravāḥ ( B-PER ), sautiḥ ( B-PER ), paurāṇikā ( B-PER ), u ( O ), naimiṣa ( B-LOC ), araṇye ( O ), śaunakasya ( B-PER ) …
Inline Bracketed	Qwen2.5-7B, Gemma-4-12B (fine-tuning)	[ loma harṣaṇa putra $\mid$ PER ] [ ugraśravāḥ $\mid$ PER ] [ sautiḥ $\mid$ PER ] [ paurāṇikā $\mid$ PER ] u [ naimiṣa $\mid$ LOC ] araṇye [ śaunakasya $\mid$ PER ] …
Two-Stage Pipeline	Qwen2.5-7B, Gemma-4-12B, Gemini 3 Flash (few-shot)	1. Mention Detection: @@loma harṣaṇa putra## @@ugraśravāḥ## @@sautiḥ## @@paurāṇikā## u @@naimiṣa## araṇye @@śaunakasya## … 2. Classification: [ "PER" , "PER" , "PER" , "PER" , "LOC" , "PER" , …]

Table 2: Example verse and output representation formats by model and training paradigm, shown on the same example verse (“…sage Lomaharṣaṇa’s son Ugraśravas [=Sauti], well-read in the Purāṇas, [came to] the Naimiṣa forest, [during] Śaunaka’s sacrifice…”). All formats are converted to a unified BIO representation for scoring. In the Two-Stage Pipeline row, @@ … ## mark candidate mentions from the detection stage, following the cascade design of Luo et al. (2025) .

Level	Agreement (%)	$\kappa$
Coarse label	91.03	0.728
Fine label	79.71	0.746
Span detection	F1 = 0.811

Table 3: Annotation reliability between the primary annotator and a second reviewer, over 1000 doubly annotated test verses.

Category	Count	Category	Count
PERSON	88,930	MISC	4,537
NORP	9,416	LOCATION	4,798
TIME	654
Total	108,335 mentions

Table 4: Distribution of coarse-grained entity categories.

Tag	Count	Tag	Count
manuṣyaḥ	51,969	devatā	11,149
īśvaraḥ	11,271	samūhaḥ	9,416
ṛṣiḥ	6,424	devayoniḥ	6,022
prākṛtikasthānam	3,284	śabdaḥ	3,132
alaukikaprāṇī	1,939	alaukika_acalanirjīvavastu	856
alaukikasthānam	806	janapadaḥ	690

Table 5: Distribution of fine-grained entity categories.

Figure 2: Rank-frequency distribution of entity mentions

Coarse-Grained
Methodology	Architecture	Model	Output Format	Micro Average (%)			Macro Average (%)
Methodology	Architecture	Model	Output Format	P	R	F1	P	R	F1
Baseline	–	CRF	BIO	78.83	40.05	53.12	74.46	32.08	44.37
Fine-tuning	Encoder	MuRIL	BIO	79.84	52.09	63.04	75.46	48.77	58.56
	Enc-Dec	ByT5	BIO	78.96	51.85	62.60	72.53	44.96	54.63
	Decoder	Qwen-2.5-7B	In. Bracketed	79.38	53.37	63.83	58.75	41.75	48.63

Table 6: Precision, Recall, and F1 for all models at coarse-grained and fine-grained granularity, grouped by training paradigm (fine-tuning vs. few-shot) and architecture. Bold marks the best score in each column within a granularity.

Model	P	R	F1
CRF	85.86	43.63	57.86
MuRIL	86.45	56.40	68.27
ByT5	85.99	56.48	68.18
Gemini-3 Flash	71.14	82.24	76.29
Qwen-2.5-7B (FT)	82.62	55.55	66.43
Gemma-4-12B (FT)	85.11	56.14	67.66

Table 7: Mention Detection performance across models (coarse-grained setup). (§ 6 ).

	Coarse (Recall %)		Fine (Recall %)
Model	Seen	Unseen	Seen	Unseen
CRF	71.9	8.8	63.7	4.6
MuRIL	80.6	24.0	72.5	18.3
ByT5	81.9	22.4	71.3	13.7
Qwen-FT	81.8	25.1	75.2	21.1
Gemma-FT	77.0	27.2	68.4	18.8

Table 8: Recall (%) for seen vs. unseen entities, based on exact surface-form overlap with training data.

Figure 3: Fine-grained error type distribution by model

Setting	Micro F1	Macro F1
Coarse, oracle detection	66.1	50.1
Fine, w/ coarse (oracle)	47.5	34.9
Fine, w/o coarse (oracle)	45.7	29.4

Table 9: Oracle-detection ablations for Gemma.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Coarse-Grained
Entity Class	Train	Test
person	72,503	16,427
norp	7,319	2,097
misc	3,577	960
location	2,879	1,919
time	558	96

Appendix

Table 10: Distribution of entity mentions across the training and test sets for both coarse-grained and fine-grained labeling schemas.

Panel of Figure 4 — Figure 4: Fine-tuning instruction templates for fine-grained (top) and coarse-grained (bottom) Sanskrit NER.

Panel of Figure 5 — Figure 5: Two-stage few-shot pipeline prompts: mention extraction (Stage 1) followed by classification (Stage 2).

Model	Gran.	Wrong	Contain	Contained	Overlap	Completely-O	OOD	Omitted	OOD
		Types	Gold	by Gold	with Gold		Ment.	Mentions	Types
CRF	Coarse	5.6%	0.3%	5.1%	0.0%	5.9%	0.0%	83.1%	0.0%
CRF	Fine	11.1%	0.2%	4.6%	0.0%	5.2%	0.0%	78.9%	0.0%
MuRIL	Coarse	8.2%	0.2%	6.6%	0.0%	9.8%	0.0%	75.2%	0.0%
MuRIL	Fine	17.9%	0.2%	5.5%	0.0%	8.6%	0.0%	67.8%	0.0%
ByT5	Coarse	8.6%	0.3%	6.7%	0.0%	10.2%	0.0%	74.2%	0.0%

Appendix

Table 11: Error distribution across models and granularities. Percentages are relative to total errors.

Panel of Figure 6 — Figure 6: Per-label F1 (%). Top: coarse-grained level. Bottom: fine-grained level (18 categories).

Granularity	Entity	Context
	$\mathcal{V}$ -Info	$\mathcal{V}$ -Info
Coarse (5 tags)	$-0.18$	$+0.41$
Fine-grained (18 tags)	$-0.31$	$+1.19$

Appendix

Table 12:

\mathcal{V}

-information for entity vs. context classifiers.

Model	Correct	Wrong	N.Det.
CRF	8.3%	25.7%	66.0%
Gemini-3 Flash	59.5%	27.1%	13.3%
Qwen-FT	31.8%	32.1%	36.0%
Qwen-FS	4.8%	6.7%	88.5%
MuRIL	24.2%	32.6%	43.2%
Gemma-FT	25.3%	27.4%	47.4%

Appendix

Table 13: Fine-grained accuracy on minority-sense homonyms. N.Det. = Not detected.

Named Entity Recognition

Named Entity Recognition identifies and classifies people, places, organizations, and other entities in text, speech, and multimodal data.

See all 110 papers

Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation
Apr 29, 2026Akhil Rajeev P, Annarao KulkarniNamed Entity Recognition Synthetic Data Generation
Centre for Development of Advanced Computing (C-DAC), Bangalore
Padamitra: Grounded Glossary Generation for Classical Sanskrit
Aug 25, 2026Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan GoyalLLM Grounding Low-Resource Language Processing
Indian Institute of Technology, Kharagpur
Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam
Aug 30, 2026Jakkala Mahesh, Jatavath Shravan Kumar, Komalla Shivani +1Named Entity Recognition Multilingual Language Model Evaluation
Rajiv Gandhi University of Knowledge Technologies, Basar, Telangana, India

Apr 29, 2026J/K move · Enter open · S save

Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation

Akhil Rajeev P, Annarao Kulkarni

Centre for Development of Advanced Computing (C-DAC), Bangalore

Named Entity Recognition Synthetic Data Generation LLM-Assisted Annotation Low-Resource Language Processing

Open paper AskPDF arXiv

The digitisation of classical Sanskrit literature is impeded by a scarcity of annotated resources, particularly for Named Entity Recognition. While recent methodologies utilise generic Large Language Models (LLMs) for data augmentation, these approaches remain prone to error and often lack the reasoning depth required for classical grammar. In this work, we introduce Naamah, a high quality silver standard Sanskrit NER dataset comprising 102,942 sentences. We propose a methodology that combines entity extraction from DBpedia with the generative capabilities of a 24B parameter hybrid reasoning model to create grammatically natural and synthetically diverse training data. We utilize this dataset to benchmark two transformer architectures: the massive multilingual XLM RoBERTa and the parameter efficient IndicBERTv2.

Abstract

Named Entity Recognition

Explore similar work

Naamah: A Large Scale Synthetic Sanskrit NER Corpus via DBpedia Seeding and LLM Generation

Padamitra: Grounded Glossary Generation for Classical Sanskrit

Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam