RenoBench: A Citation Parsing Benchmark

Parth Sarin; Juan Pablo Alperin; Adam Buttrick; Dione Mentis

cs.DLMar 26, 2026

RenoBench: A Citation Parsing Benchmark

Authors: Parth Sarin, Juan Pablo Alperin, Adam Buttrick, Dione Mentis

Organizations: Graduate School of Education, Stanford University, Stanford, California, USA · Public Knowledge Project, Simon Fraser University, Vancouver, British Columbia, Canada · California Digital Library, University of California Office of the President, Oakland, California, USA · DataCite, Hannover, Germany

Abstract

Accurate parsing of citations is necessary for machine-readable scholarly infrastructure. But, despite sustained interest in this problem, existing evaluation techniques are often not generalizable, based on synthetic data, or not publicly available. We introduce RenoBench, a public domain benchmark for citation parsing, sourced from PDFs released on four publishing ecosystems: SciELO, Redalyc, the Public Knowledge Project, and Open Research Europe. Starting from 161,000 annotated citations, we apply automated validation and feature-based sampling to produce a dataset of 10,000 citations spanning multiple languages, publication types, and platforms. We then evaluate a variety of citation parsing systems and report field-level precision and recall. Our results show strong performance from language models, particularly when fine-tuned. RenoBench enables reproducible, standardized evaluation of citation parsing systems, and provides a foundation for advancing automated citation parsing and metascientific research.

Figures & tables

Figure 1: An annotated citation from RenoBench

Model	Valid XML	Surname	Given	Title	Source	Volume	Issue	Year
GROBID	100.0	71.6	18.8	65.2	51.8	90.0	86.8	97.9
Qwen/Qwen2.5-3B-Instruct	83.5	93.0	64.1	93.9	54.3	95.5	83.8	93.8
Qwen/Qwen2.5-7B-Instruct	87.6	95.8	74.5	94.0	53.3	96.0	86.9	90.6
Qwen/Qwen3-0.6B	81.4	87.3	62.7	84.7	50.6	92.4	92.7	91.7
Qwen/Qwen3-0.6B + cometadata/citation-parsing-lora	95.0	92.6	77.9	93.4	73.3	98.6	95.3	97.2
Qwen/Qwen3-8B	78.2	86.6	70.9	82.0	51.2	90.6	89.8	80.7

Table 1: Recall Across Models and Fields

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Valid XML	Surname	Given	Title	Source	Volume	Issue	Year
GROBID	100.0	70.2	18.7	58.4	48.0	69.8	49.1	96.9
Qwen/Qwen2.5-3B-Instruct	83.5	86.5	62.5	56.0	53.4	86.7	88.0	97.7
Qwen/Qwen2.5-7B-Instruct	87.6	89.0	72.3	59.2	53.0	87.1	78.3	98.2
Qwen/Qwen3-0.6B	81.4	79.5	59.0	51.1	47.4	42.9	25.7	95.1
Qwen/Qwen3-0.6B + cometadata/citation-parsing-lora	95.0	83.6	73.5	55.7	66.9	43.5	25.2	96.4
Qwen/Qwen3-8B	78.2	91.8	76.1	60.2	59.1	82.3	74.5	91.4

Appendix

Table 2: Precision Across Models and Fields

Model	Valid XML	Surname	Given	Title	Source	Volume	Issue	Year
GROBID	100.0	71.6	18.8	65.2	51.8	90.0	86.8	97.9
Qwen/Qwen2.5-3B-Instruct	83.5	79.0	54.9	81.6	47.0	84.1	73.5	79.2
Qwen/Qwen2.5-7B-Instruct	87.6	83.7	65.2	83.4	47.1	85.0	76.7	79.8
Qwen/Qwen3-0.6B	81.4	72.5	52.9	72.7	44.2	80.8	81.8	75.8
Qwen/Qwen3-0.6B + cometadata/citation-parsing-lora	95.0	88.5	74.6	89.3	70.1	94.3	91.0	92.8
Qwen/Qwen3-8B	78.2	72.5	60.3	73.0	45.6	78.0	77.6	70.5

Appendix

Table 3: Recall (Strict) Across Models and Fields

Model	Valid XML	Surname	Given	Title	Source	Volume	Issue	Year
GROBID	100.0	70.2	18.7	58.4	48.0	69.8	49.1	96.9
Qwen/Qwen2.5-3B-Instruct	83.5	73.4	53.5	48.7	46.2	76.4	77.1	82.5
Qwen/Qwen2.5-7B-Instruct	87.6	77.8	63.2	52.5	46.8	77.1	69.1	86.5
Qwen/Qwen3-0.6B	81.4	66.1	49.8	43.9	41.4	37.5	22.6	78.6
Qwen/Qwen3-0.6B + cometadata/citation-parsing-lora	95.0	79.9	70.4	53.3	64.1	41.6	24.0	92.0
Qwen/Qwen3-8B	78.2	76.8	64.7	53.6	52.6	70.8	64.4	79.8

Appendix

Table 4: Precision (Strict) Across Models and Fields

Document Parsing Benchmarks

See all 15 papers

ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers
Jul 13, 2026Saba Imran, Debanjum Singh SolankyLLM Evaluation Citation Verification
Khoj Inc.
Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
Jun 21, 2025Yukun Huang, Sanxing Chen, Jian Pei +2Continual Learning for LLMs
Duke University · Meta
Local Citation Recommendation Without Graph Signals
Oct 11, 2026Benjamin Basseri, Pavlos Protopapas, Kemeng Zhang +3Information Retrieval Recommender Systems
Harvard University · Center for Astrophysics | Harvard & Smithsonian

Jul 13, 2026J/K move · Enter open · S save

ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers

Saba Imran, Debanjum Singh Solanky

Khoj Inc.

LLM Evaluation Citation Verification Scientific QA Knowledge-Intensive QA Factual QA

Open paper AskPDF arXiv

Large language models are increasingly used to assist scientific reading, but existing evaluation methods often fail to detect whether answers are supported by verifiable citations. We introduce ResearchQA, a benchmark of 6,211 single-paper question-answer pairs from 494 open-access papers spanning eight domains and four question types: lookup, comprehension, multi-hop, and adversarial. ResearchQA is designed for citation-grounded evaluation: it permits multiple valid supporting passages for a claim and rewards grounded refusal when the source paper does not support an answer. We evaluate eight leading closed- and open-weight models in a citation-grounded chat-with-paper setting using a deterministic citation matcher and an LLM-based rubric evaluator. Citation-based metrics separate systems more clearly than LLM-evaluator scores: section coverage and citation accuracy vary substantially across models, while evaluator scores remain tightly compressed. We further find that open-weight models approach the best closed-model citation accuracy while achieving 3 to 6 times lower per-example latency. We release the benchmark, evaluation harness, and evaluator prompt.

Abstract

Document Parsing Benchmarks

Explore similar work

ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models

Local Citation Recommendation Without Graph Signals