Financial QA

QA: Question Answering

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 33

All topics

IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text
Apr 21, 2026Rajveer Singh PallLLM Evaluation Financial QA
Gyan Ganga Institute of Technology and Sciences, Jabalpur, India
SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
Apr 21, 2026Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl +12Financial Services Arabic NLP
1MBZUAI · 2INSAIT, Sofia University ”St. Kliment Ohridski” · 3The University of Manchester +1
Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off
Mar 26, 2026Zhiyuan Cheng, Longying Lai, Yue LiuRetrieval-Augmented Generation Financial QA
School of Engineering, Stanford University, Stanford, CA, USA · bSimon Business School, University of Rochester, Rochester, NY, USA · cAccounting & Information Systems, Rutgers University, Newark, NJ, USA

Apr 21, 2026, cs.CLJ/K move · Enter open · S save

IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text

Rajveer Singh Pall

Gyan Ganga Institute of Technology and Sciences, Jabalpur, India

LLM Evaluation Financial QA Finance

Open paper AskPDF arXiv

We introduce IndiaFinBench, to our knowledge the first publicly available evaluation benchmark for assessing large language model (LLM) performance on Indian financial regulatory text. Existing financial NLP benchmarks draw exclusively from Western financial corpora (SEC filings, US earnings reports, English-language financial news), leaving a significant gap in coverage of non-Western regulatory frameworks. IndiaFinBench addresses this gap with 406 expert-annotated question-answer pairs drawn from 192 documents sourced from the Securities and Exchange Board of India (SEBI) and the Reserve Bank of India (RBI), spanning four task types: regulatory interpretation (174 items), numerical reasoning (92 items), contradiction detection (62 items), and temporal reasoning (78 items). Annotation quality is validated through a model-based secondary pass (kappa=0.918 on contradiction detection; 90.7% overall agreement on a 150-item subset) and a 180-item human inter-annotator agreement study across three annotation rounds (kappa=0.645 on contradiction detection; 77.2% overall agreement; 44.3% benchmark coverage). We evaluate twelve models under zero-shot conditions, with accuracy ranging from 70.4% (Gemma 4 E4B) to 89.7% (Gemini 2.5 Flash). All models substantially outperform a non-specialist human baseline of 69.0%. Numerical reasoning is the most discriminative task, with a 35.9 percentage-point spread across models. Bootstrap significance testing (10,000 resamples) reveals three statistically distinct performance tiers. The dataset, evaluation code, and all model outputs are available at https://github.com/rajveerpall/IndiaFinBench

Momentum

Related topics

Latest papers 33

IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text

SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning

Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off