SinhaLegal: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Texts
Authors: Minduli Lasandi, Nevidu Jayatilleke
Organizations: School of Computing, Informatics Institute of Technology, Sri Lanka · Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka
SinhaLegal introduces a Sinhala legislative text corpus containing approximately 2 million words across 1,206 legal documents. The dataset includes two types of legal documents: 1,065 Acts dated from 1981 to 2014 and 141 Bills from 2010 to 2014, which were systematically collected from official sources. The texts were extracted using OCR with Google Document AI, followed by extensive post-processing and manual cleaning to ensure high-quality, machine-readable content, along with dedicated metadata files for each document. A comprehensive evaluation was conducted, including corpus statistics, lexical diversity, word frequency analysis, named entity recognition, and topic modelling, demonstrating the structured and domain-specific nature of the corpus. Additionally, perplexity analysis using both large and small language models was performed to assess how effectively language models respond to domain-specific texts. The SinhaLegal corpus represents a vital resource designed to support NLP tasks such as summarisation, information extraction, and analysis, thereby bridging a critical gap in Sinhala legal research.
Figures & tables
Figure 1: Distribution of document size of Acts and Bills
Figure 2: The flow of documents through each stage of the filtration process. *The initial dataset consists of documents that were available in the repository on the date of access (20th August 2025).
Figure 3: Structure of the SinhaLegal dataset
Statistic
Value
Average word tokens per document
1,677
Median word tokens per document
1,213
token count range
95 - 23,430
Total characters
12,877,970
Average characters per document
10,678
Table 1: Summary statistics of the SinhaLegal dataset
Statistic
Acts
Bills
Total Corpus
Total word tokens
1,778,265
243,942
2,022,207
Vocabulary size
37,326
10,390
39,169
TTR
0.7315
0.7456
0.7284
Table 2: The number of total word tokens, vocabulary size and type token ratio for Acts, Bills and the total dataset.
Documents
Hapax legomena
Count
Ratio
Acts
17,632
47.24%
Bills
4,026
38.75%
Total corpus
18,074
46.14%
Table 3: Distribution of hapax legomena across Acts, Bills, and the total corpus.
Entity type
Number of Entities
Date [DATE]
13,532
Title [TITLE]
8,736
Organisations [ORG]
4,281
Law [LAW]
2,255
Person [PERSON]
126
Amount [AMOUNT]
7
Table 4: Number of entities extracted from the SinhaLegal dataset
Model Name
MADLAD CulturaX
SinhaLegal
Llama-3.1
3.05
2.55
Deepseek-1.3B
3.30
2.94
Mistral - 7B
3.68
3.18
Falcon - 7B
2.77
2.61
DistilGPT - 2
6.45
5.77
Gemma - 2B
8.75
5.59
Table 5: Comparison of perplexity scores of the two datasets. Bold: indicates best performance and Underline: indicates the second best.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Relationship between the page count and OCR confidence
Figure 5: Distribution of Acts and Bills throughout the years from 1981-2025
Figure 6: Boxplot showing the distribution of page counts for legal documents by type.
Table 6: Examples of extra sentences in the scanned PDF and extracted text removed during post-processing.
Figure 7: Example of the metadata record for a document
Number of Word Covered
Acts
Bills
Top 20 words
23.00%
23.32%
Top 50 words
35.04%
35.53%
Top 100 words
45.89%
46.39%
Appendix
Table 7: Coverage of the most frequent words in Acts and Bills.
Table 8: The top 10 frequent words seen in Acts
Table 9: The top 10 frequent words seen in Bills
Table 10: The top 10 frequent bigrams seen in SinhaLegal
Table 11: The word distributions and their word probabilities for the identified topics from topic modelling.
Figure 8: Distribution of words per document before and after OCR
Statistic
Value
Documents analysed
100
Total words
184,011
Average words per document
1,840
Median words per document
1,295
Word count range
112 - 9,028
Total characters
911,524
Appendix
Table 12: Summary statistics of the evaluated document subset
Metric
Original Evaluation
Normalized Evaluation
CER
24.07%
24.06%
WER
26.87%
23.44%
Appendix
Table 13: Comparison of OCR evaluation metrics before and after text normalisation.
Feature
Raw OCR
Post-processed
Difference
Line breaks
39
22
17
Spaces
173
94
79
Appendix
Table 14: Structural comparison of raw OCR and post-processed text, showing the number of line breaks and spaces removed during post-processing.
Table 15: The character-level differences between a raw OCR and post-processed text in a document
Statistic
Raw OCR
Post-processed
Total words
211,317
184,011
Total characters
1,250,623
1,100,695
Average words per document
2,113
1,840
Average characters per document
1,2506
11007
Word count range
213–9891
112–9028
Appendix
Table 16: Values comparing raw OCR and post-processed documents on the word and character count.
Table 17: Example of scanned PDF, OCR text, and corrected text for word-level corrections.
SiPaKosa is a comprehensive corpus of Sinhala and Pali doctrinal texts comprising approximately 786K sentences and 9.25M words, incorporating 16 copyright-cleared historical Buddhist documents alongside the complete web-scraped Tripitaka canonical texts. The corpus was created through high-quality OCR using Google Document AI on historical manuscripts, combined with systematic web scraping of canonical repositories, followed by rigorous quality control and metadata annotation. The corpus is organised into language-specific subcorpora: Sinhala and Mixed Sinhala-Pali. We evaluate the performance of language models using ten pretrained models, with perplexity scores ranging from 1.09 to 189.67 on our corpus. This analysis shows that proprietary models significantly outperform open-source alternatives by factors of three to six times. This corpus supports the pretraining of domain-adapted language models, facilitates historical language analysis, and aids in the development of information retrieval systems for Buddhist scholarship while preserving Sinhala cultural heritage.
Ranidu Gurusinghe, Nevidu Jayatilleke
School of Computing, Informatics Institute of Technology, Sri Lanka · Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka
We present a collection of open, machine-readable document datasets covering parliamentary proceedings, legal judgments, government publications, news, and tourism statistics from Sri Lanka. The collection currently comprises of 299,131 documents (85.4 GB) across 26 datasets in Sinhala, Tamil, and English. The datasets are updated daily and mirrored on GitHub and Hugging Face. These resources aim to support research in computational linguistics, legal analytics, socio-political studies, and multilingual natural language processing. We describe the data sources, collection pipeline, formats, and potential use cases, while discussing licensing and ethical considerations. This manuscript is at version v2026-10-12-0839.
We present the Bangladesh Legal Acts Dataset (BLAD), a curated collection of 1{,}484 legislative acts enacted between 1799 and 2025. Each act is represented with its full text, structured sections and footnotes, repeal status, and metadata linking it to the governing regime, head of state, and prevailing legal framework at the time of enactment. The corpus spans English, Bengali, and mixed-language documents, supporting temporal and multilingual analysis of statutory law. BLAD addresses a persistent gap in legal natural language processing (NLP) resources for low-resource, civil-law jurisdictions in South Asia. We describe the acquisition and enrichment pipeline, report descriptive statistics over more than two centuries of legislation, and outline the research directions the corpus enables. The dataset is publicly available under the CCBY-SA4.0 license at https://www.kaggle.com/datasets/sakhadib/bangladesh-legal-acts-dataset.
Adib Sakhawat
Systems and Software Lab (SSL) · Department of Computer Science and Engineering · Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh