cs.CLNov 9, 2026

BharatSynthQA: A Context-Grounded Question Answering Dataset on Indian Temples and Religious Heritage

Authors: Riya Naik, Shivam Sharma, Swati Agarwal

Organizations: BITS Pilani, K K Birla Goa Campus, India · Indian Institute of Technology Madras, India · PandaByte Innovations Pvt Ltd, India

Abstract

India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art. Due to the widespread curiosity, people increasingly turn to Artificial Intelligence (AI) assistants for information about them. Current models are rarely trained or evaluated on this domain. We present BharatSynthQA, a passage grounded question answering (QA) dataset focused on Indian temples, which also covers other religious sites connected to them. Our pipeline crawls Wikipedia from topics seeded by authentic heritage sources, filters articles for topical relevance, and locations in India. Further it generates Boolean, factoid, and explanatory QA pairs with a fewshot prompted LLM. An LLM judge filters the pairs on answer correctness, retaining 98,677 of 131,494 generated pairs. Additionally, 300 human-annotated pairs are released separately as a reference set. Human evaluation of the subsample of this dataset showed 89.83% of the questions were formed correctly. Benchmarking five current LLMs shows that factoid token F1 is only 0.18-0.30 without the source passage but 0.73-0.77 with it, indicating that this domain is poorly captured in model knowledge. The dataset is available at https://huggingface.co/datasets/theshivam7/bharatsynth-qa to support research on culturally grounded QA and retrieval-augmented generation for Indian heritage.

Figures & tables

Explore similar work

CardsList
  1. VIRAASAT: Traversing Novel Paths for Indian Cultural Reasoning

    Feb 20, 2026Harshul Raj Surana, Arijit Maji, Aryan Vats +3Cultural Knowledge in Language ModelsMulti-Hop QA

  2. L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages

    Aug 16, 2026Rinit Jain, Tirthraj Mahajan, Advait Joshi +1Multilingual QAFactual Knowledge in Language Models

  3. IndicIPR-QA: A Benchmark for Extractive Question Answering on Indian IPR Laws

    Oct 29, 2026M. Mohaiminul Islam, Muhammad AbulaishQuestion AnsweringLegal QA