Reading Beyond OCR Noise: Robust Cross-Lingual RAG for Legislative Archives

Radha Krishna Deshpande; Ajoy Mondal; Akash Manna; C V Jawahar

cs.CVOct 26, 2026

Reading Beyond OCR Noise: Robust Cross-Lingual RAG for Legislative Archives

Authors: Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar

Organizations: CVIT, International Institute of Information Technology, Hyderabad, India

Abstract

India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrieval. We present an end-to-end framework that couples vision-based layout segmentation and multi-script OCR with an agentic retrieval-augmented generation (RAG) pipeline. Our approach specifically addresses how visual degradation and layout fragmentation propagate into text extraction errors, using an iterative agentic loop to navigate around OCR noise and layout ambiguities before generating answers. Evaluated on the 200,000-page Punjab Vidhan Sabha archive, our system demonstrates robust handling of historical document degradation, outperforming standard text-only baselines.

Figures & tables

Figure 1: Architecture of the proposed Agentic Retrieval-Augmented Generation (RAG) system. The framework features multi-lingual query processing, an iterative LLM-driven evaluation loop to ensure context sufficiency, and a grounded response generation pipeline.

Figure 2: Overview of dataset pages featuring tabular data, multi-scripts, and faded text characteristics.

Model & Pipeline Setup	Eval. Benchmark	MRR@1	Hit@3	Hit@5	Prec@3
MuRIL	1,000 Pages / 200 Queries	0.000	0.005	0.015	0.002
LaBSE	1,000 Pages / 200 Queries	0.320	0.510	0.590	0.170
Embedding-gemma (300M)	1,000 Pages / 200 Queries	0.525	0.735	0.775	0.245
NetraEmbed (dim=768)	1,000 Pages / 200 Queries	0.580	0.816	0.890	0.272
NetraEmbed (dim=1536)	1,000 Pages / 200 Queries	0.598	0.835	0.905	0.278
NetraEmbed (dim=2560)	1,000 Pages / 200 Queries	0.605	0.843	0.912	0.281

Table 1: PVS Retrieval Evaluation on the 1,000-page / 200-query stratified benchmark (indexed against the full 200,000-page archive).

Figure 3: End-to-end cross-lingual RAG pipeline for legislative documents, from the user query and document retrieval to grounded answer generation in Punjabi and translated to english for the reference.

Figure 4: Qualitative comparison of cross-lingual embedding models for historical legislative document retrieval. The same Punjabi query is given to multilingual-e5-large, BGE-M3, LaBSE, Netra embedding (2560 dimensions), and EmbeddingGemma-300M. The retrieved document from each model is compared with the ground-truth relevant document shown in the rightmost column. The retrieved results demonstrate differences in cross-lingual semantic matching, with several models returning visually and semantically unrelated legislative records. In contrast, our proposed system retrieves the relevant document containing the required information, demonstrating its effectiveness for retrieval from multilingual historical archives.

Layout Model	OCR Model	CRR (%)	WRR (%)
Gemini-3.1	Gemini-3.1	99.7	89.6
Surya2	Surya2	99.6	88.9
Sarvam	Sarvam	99.5	89.4
YOLO (Fine-tuned)	CRNN (Ours)	95.67	80.03
CRAFT	CRNN (Ours)	89.00	60.35

Table 2: OCR pipeline comparison on the 20-page ground-truth subset.

OCR Model	MRR@1	Hit@3	Hit@5	Prec@3
Gemini-3.1	0.825	0.920	0.960	0.3533
Sarvam	0.820	0.935	0.955	0.325
Surya2	0.815	0.925	0.950	0.305
Ourocr	0.810	0.925	0.955	0.325

Table 3: PVS Retrieval Evaluation using different OCR models on the 200-query benchmark.

Figure 5: Iterative retrieval correction through query rewriting. The first two iterations retrieve irrelevant same documents, while the third iteration retrieves the correct document containing the relevant information. The example demonstrates a cross-lingual setting, where the query is in Punjabi and the retrieved document is in English.

Historical Document Processing

Historical document processing uses AI to transcribe, structure, restore, and analyze manuscripts, books, newspapers, and archival records.

See all 76 papers

PereStruct: Multimodal Semantic Assembly for Robust Historical Document Parsing
Jun 3, 2026Maksim Shandybo, Ivan Bespalov, Daniil Yefimov +2Historical Document Analysis Multimodal Robustness
1IGIC RAS, Moscow, Russian Federation · 2Yandex Cloud, Moscow, Russian Federation · 3National University of Science and Technology MISIS, Moscow, Russian Federation +1
AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation
Feb 27, 2026Zhengren Wang, Dongsheng Ma, Huaping Zhong +4Multimodal RAG Visual Document Retrieval
Shanghai Artificial Intelligence Laboratory · Peking University · SenseTime
Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
Feb 18, 2026Ali Faraz, Raja Kolla, Ashish Kulkarni +1Multilingual OCR Optical Character Recognition
Krutrim AI, Bangalore, India

Jun 3, 2026J/K move · Enter open · S save

PereStruct: Multimodal Semantic Assembly for Robust Historical Document Parsing

Maksim Shandybo, Ivan Bespalov, Daniil Yefimov, Marina Kosheleva+1

1IGIC RAS, Moscow, Russian Federation · 2Yandex Cloud, Moscow, Russian Federation · 3National University of Science and Technology MISIS, Moscow, Russian Federation+1

Historical Document Analysis Multimodal Robustness Document Parsing Document Layout Analysis Historical Document Processing

Open paper AskPDF arXiv

Parsing historical documents with complex, non-standard layouts remains a fundamental bottleneck in large-scale archival digitization. Unlike modern typography, historical newspapers exhibit severe physical degradation and highly irregular page structures that confound even state-of-the-art vision-language models, presenting severe out-of-distribution challenges. We address this gap with an automated pipeline specifically designed for parsing historical newspapers, documents characterized by particularly intricate multi-column layouts. Our approach combines a fine-tuned YOLO architecture for layout analysis and block detection, trained on 1,426 fully human-annotated scanned pages, with a novel semantic assembly module that reconstructs articles by jointly modeling lexical-semantic similarity via TF-IDF, visual embeddings from our fine-tuned YOLO, and geometric layout constraints. This multi-modal integration yields state-of-the-art performance, achieving an F1 score of 0.904 on block-to-article mapping. Notably, end-to-end evaluation against vision-language models (Qwen3.6-35B-A3B and Qwen3.6-Plus) demonstrates that PereStruct achieves substantially higher fidelity (BLEU approximately 0.96 vs 0.34), validating that modular architectures excel where generic VLMs fail on complex historical layouts. To support reproducibility and advance research in this domain, we release both the training corpus of 599 annotated pages and a curated PereStruct benchmark of 93 pages with expert-verified ground-truth block-to-article mappings. This framework establishes a robust foundation for high-fidelity digitization and semantic reconstruction of complex archival materials.

Abstract

Historical Document Processing

Explore similar work

PereStruct: Multimodal Semantic Assembly for Robust Historical Document Parsing

AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation

Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems