AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents

Chen Qian; Yimeng Wang; Yu Chen; Lingfei Wu; Andreas Stathopoulos

cs.CLSep 5, 2026

AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents

Authors: Chen Qian, Yimeng Wang, Yu Chen, Lingfei Wu, Andreas Stathopoulos

Organizations: William & Mary · Anytime AI

Abstract

Large language models answering questions over multi-page documents are expected to cite the supporting pages, yet supplied citations are sometimes inaccurate, and current evaluations score citations at generation time or against text passages: no existing benchmark evaluates whether a system can verify and correct a page-level citation already attached to an answer. We propose AtomCite, an agentic framework that parses an answer into claims, checks each claim against the image of its cited page, and applies a deterministic repair policy. To evaluate it, we introduce DocCite, to our knowledge the first benchmark for systems that verify and correct page-level citations in document images. Built on MP-DocVQA and DUDE, it combines 928 validated injected instances with 2,468 candidate natural errors harvested from frontier- and efficiency-tier models, of which a two-annotator audit confirms 1,909 as genuine errors. Primary labels are assigned deterministically, not by LLM judges, with the human audit as a separate validation layer. Across three model families (Gemini, Claude, and GPT), AtomCite reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition, including a compute-matched control, and exceeding every prior text-based baseline given the same OCR text. Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims. AtomCite also transfers: with frozen prompts and zero training, it raises the hallucination-detection scores of two open 7-8B models on five public benchmarks above the same models prompted as direct judges. Finally, the audit shows that noise in automatic labels biases measured verifier accuracy and can reverse system rankings, so evaluations relying only on synthetic or automatic labels risk mismeasuring verification capability.

Figures & tables

Figure 1: The AtomCite pipeline on a four-claim answer, the five stages grouped into parse, verify, and correct. Each parsed (claim, cited page) pair is checked by text and image sub-agents on the cited page only; window pages contribute correction proposals, never verdicts. The example covers all four actions: a true claim cited to the wrong page is repointed ( fix-page ), a verified claim kept, a claim supported nowhere removed, and a hard text–image conflict ( inconsistent ) kept and flagged, never deleted.

	Gemini		Claude		GPT
Condition	Acc	Sup. R	Acc	Sup. R	Acc	Sup. R
(A1) Tesseract flat	84.5	59.7	84.8	63.5	83.7	58.5
(A2) Tesseract layout	85.2	62.3	84.2	63.8	85.2	61.9
(A3) Tesseract matched	86.2	62.6	86.7	71.7	87.8	69.2
(A1 ′ ) MinerU flat	87.7	69.8	88.4	70.8	89.0	73.6
(A2 ′ ) MinerU layout	88.5	72.0	89.1	74.5	88.1	71.4

Table 1: DocCite-Syn conditions ladder (

n{=}928

; %). Acc = binary cited-page support accuracy (primary metric); Sup. R = recall on truth-supported instances. (C) is claim-level AtomCite (its decomposition variant is ablated with the results). Bold = per-family best among image-grounded conditions; the two (B1) Sup. R cells above (C) reflect lenient acceptance, not sharper discrimination (see Architecture vs. Prompting); paired differences with CIs in Table 2 . The matched control (A3) reaches its score with near-zero wrong-location recall.

Modality axis: what does the image channel contribute?
Difference	Gemini	Claude	GPT
(C) $-$ (A1 ′ )	+5.9 [3.8,8.0]	+4.5 [2.4,6.5]	+3.9 [2.0,5.8]
(C) $-$ (A2 ′ )	+5.1 [3.2,7.1]	+3.7 [1.8,5.6]	+4.8 [2.8,6.8]
(C) $-$ (A3 ′ )	+4.9 [3.0,6.9]	+2.3 [0.3,4.2]	+3.9 [2.1,5.8]
Prompting axis: does staged prompting close the gap?
(C) $-$ (B1)	+5.9 [3.4,8.5]	+11.2 [8.7,13.8]	+4.8 [2.6,6.9]

Table 2: Key paired differences in binary accuracy (pp; doc-cluster bootstrap,

B{=}10^{4}

, 95% CIs). Condition codes as in Table 1 ; the modality rows use the stronger MinerU engine (corpus-engine gaps are larger). All 18 differences are Holm-significant (CIs per-comparison).

System	Tier	Acc	Sup. R
CiteEval-Auto (Gem/Cl/GPT)	T2	89.0/88.2/88.9	—
HHEM	T1	83.9	64.5
MiniCheck	T1	82.9	57.5
LettuceDetect	T1	81.7	52.5
AlignScore	T1	76.0	33.0
DeBERTa-NLI	T1	72.8	81.4

Table 3: Text-Baseline Tier on DocCite-Syn (

n{=}928

; %; detector operating point

0.5

). T1 = specialized detectors run as intended; T2 = adapted protocols. Same-family paired differences, (C)

-

CiteEval-Auto:

+4.5

[2.0,7.1]

Gemini,

+4.9

[2.2,7.7]

Claude,

+4.1

[1.7,6.7]

GPT, all significant.

	Gemini			Claude			GPT
Condition	Det	FA	Acc	Det	FA	Acc	Det	FA	Acc
AtomCite (C)	97.7	5.5	97.1	97.3	5.4	96.8	98.6	12.3	96.5
Flat OCR (A1)	99.7	43.8	91.4	99.8	48.2	90.7	99.7	49.3	90.3
Seq. chain (B3)	91.1	16.9	89.6	87.2	10.0	87.7	96.3	12.4	94.6

Table 4: DocCite-Nat against audit-validated labels (%; best per column in bold). Partial support counts as an error. Det = detection of the

1{,}909

audit-validated errors; FA = false-alarm rate on the

450

human-confirmed supported claims; Acc = overall accuracy on the

2{,}359

audited instances. Rates are over the audited candidate pool, not all supplied citations. Balanced accuracy: C

96.1

96.0

93.1

, A1

78.0

75.8

75.2

, B3

87.1

88.6

91.9

(Gemini/Claude/GPT).

System	Precision	Retention	Coverage	Fix-reach
AtomCite (Gemini)	34.3 $\to$ 89.9	91.8	95.4	68.9
AtomCite (Claude)	34.3 $\to$ 87.7	91.8	94.1	72.9
AtomCite (GPT)	34.3 $\to$ 87.0	92.5	94.1	67.7
CiteFix (T3, reimpl.)	34.3 $\to$ 50.3	100.0 †	54.9	80.5
RARR (T2, adapted)	34.3 $\to$ 36.8	94.0	15.7	0.0

Table 5: Correction on DocCite-Syn (conservative mode; %). Precision = citation precision before

\to

after correction; Retention = correct claims kept; Coverage = error instances acted on; Fix-reach = exact gold-page recovery on window-reachable wrong-location citations. † CiteFix is replacement-only (cannot remove): its

100\%

retention is uninformative.

	Qwen2.5-7B		Llama-3-8B
Benchmark (metric)	direct	AtomCite	direct	AtomCite
RAGTruth (F1)	49.5	71.2	56.4	66.2
FaithBench (bAcc)	50.8	57.0	52.8	56.9
HalluMix (acc)	76.3	80.0	58.0	59.7
TofuEval (bAcc)	67.7	69.1	54.3	64.2
VeriGray resp. (F1)	29.8	44.6	39.7	54.7

Table 6: Transfer to five public hallucination benchmarks (%), under official test sets and metrics (

n{=}2{,}700

750

6{,}500

534

329

): F1 = response-level F1, bAcc = balanced accuracy, acc = accuracy. The direct arm is a same-backbone single-call judge; AtomCite is the framework in text mode.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Generator	AB claims	Cand. errors	Prev. (%)
Gemini frontier	954	426	44.7
Gemini efficiency	1,076	556	51.7
Claude frontier	1,037	590	56.9
Claude efficiency	1,098	601	54.7
GPT frontier	284	134	47.2
GPT efficiency	139	85	61.2

Appendix

Table A1: Layer-1 candidate-error prevalence by generator (AB = answer-bearing cited claims). Candidate status is the Layer-1 localization label; the two-annotator human audit validates

77.4\%

of pooled candidates as genuine errors (partial support counted as error).

Condition	Binary acc. (%)	$/item	Calls/item
(C) AtomCite	93.7	0.024 ∗	2.8
(C-atom)	91.9	0.030	3.5
(B3) seq. chain	87.5	0.029	3.5
(A3) Tesseract matched	86.2	0.028	4.0
(B2) checklist	88.1	0.020	1.0
(B1) direct	87.9	0.018	1.0

Appendix

Table A2: Cost-effectiveness on DocCite-Syn (Gemini backbone; list prices, uncached calls, per-item ledger averages). ∗ (C) measured on its two cache-disabled repeats (run-to-run variance paragraph); its original run shared cached grounding calls with (C-atom) on single-atom claims. Call/token structure is identical across families; absolute prices differ by provider list price.

	Condition	Bin	Sup. R	Gap to (C)	95% CI
Gem.	(A1) flat	87.7	69.8	$+5.9$	$[3.8,8.0]$
	(A2) layout	88.5	72.0	$+5.1$	$[3.2,7.1]$
	(A3) matched	88.7	72.6	$+4.9$	$[3.0,6.9]$
Cl.	(A1) flat	88.4	70.8	$+4.5$	$[2.4,6.5]$
	(A2) layout	89.1	74.5	$+3.7$	$[1.8,5.6]$
	(A3) matched	90.5	78.6	$+2.3$	$[0.3,4.2]$

Appendix

Table A3: Full MinerU 2.5 replication of the A ladder (

n{=}928

; %). Gap = paired difference (C)

-

rung, doc-cluster bootstrap,

B{=}10^{4}

; all nine are Holm-significant (

\alpha{=}0.05

, family of nine). Against the corpus-engine ladder, the stronger engine lifts every rung (supported recall by

+8

–

15

pp) without closing the image gap in any family.

	Condition	Binary	Sup. R	WL R
Gemini	(A1) Tesseract flat	84.5	59.7	56.9
	(A2) Tesseract layout	85.2	62.3	57.7
	(A3) Tesseract matched	86.2	62.6	0 0.6
	(B1) direct	87.9	81.4	59.9
	(B2) structured	88.1	80.8	55.8
	(B3) seq. matched	87.5	75.2	58.6

Appendix

Table A4: Full metric breakdown of the conditions ladder (

n{=}928

; %). WL R = recall on truth-wrong-location instances. The compute-matched OCR control (A3) reaches its binary score with no localization ability: its wrong-location recall collapses to

{\leq}1.4

in every family — its binary gains come entirely from coarse supported/unsupported calls. Wrong-location recognition is the capability the correction policy builds on.

Window	Bin	Prec.	Ret.	Reach	All	Pages
$\pm 1$	91.9	91.8	85.8	69.2	45.3	1.77
$\pm 2$ (default)	91.9	89.9	86.5	65.3	45.3	3.33
$\pm 3$	91.9	88.6	86.5	62.9	45.9	3.77
$\pm 4$	91.9	88.0	86.2	47.1	36.5	4.00
$\pm 5$	91.9	86.9	87.7	38.5	35.6	4.00

Appendix

Table A5: Correction-window sweep (Gemini,

n{=}928

; %; run under the decomposition-variant configuration — verification is window-invariant in both configurations, and the correction trade-off direction carries over). Bin = verification binary accuracy (invariant by design); Prec. = post-correction citation precision; Reach = exact recovery on wrong-location citations reachable within each window; All = exact recovery over all wrong-location citations; Pages = mean effective window size after document-boundary and evidence-cap truncation. Marginal re-run token cost rises monotonically with width (from

\sim

2.2k/item at

\pm 1

\sim

4.2k at

\pm 5

	Input: answer $A$ ; document $D$ (pages: OCR text $+$ image)
1:	$C\leftarrow\textsc{Parse}(A)$ $\rhd$ (claim, cited page) pairs
2:	for each $(x,p)\in C$ in parallel:
3:	$E\leftarrow\textsc{Evidence}(D,p)$ $\rhd$ cited page $+$ $\pm 2$ window
4:	$v_{\text{txt}}\leftarrow\textsc{Ground}(x,\mathrm{OCR}(p))$ $\rhd$ text channel
	$v_{\text{img}}\leftarrow\textsc{Ground}(x,\mathrm{Img}(p))$ $\rhd$ image channel
5:	$v\leftarrow\textsc{Merge}(v_{\text{txt}},v_{\text{img}})$ $\rhd$ hard $\to$ inc. ; soft $\to$ image wins

Appendix

Figure A1: Algorithm 1: AtomCite (conservative mode, claim-level default).

Panel of Figure 14 — Figure A2: DocCite-Nat audit funnel. The spot-check found $0\%$ noise ( $[0,7.3]$ ) in the undisputed region, whose Layer-1 labels stand.

Citation Verification

Citation verification checks whether cited sources exist, are accurately represented, and actually support the claims attached to them.

See all 49 papers

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence
May 13, 2026Dongsheng Ma, Jiayu Li, Zhengren Wang +9VLM Evaluation Citation Verification
Peking University · Shanghai Artificial Intelligence Laboratory
Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
Jun 21, 2025Yukun Huang, Sanxing Chen, Jian Pei +2Continual Learning for LLMs
Duke University · Meta
CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era
Feb 26, 2026Kaiwen Shi, Weixiang Sun, Zheyuan Zhang +3Citation Verification LLM Hallucination Detection
University of Notre Dame · Lehigh University

May 13, 2026J/K move · Enter open · S save

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang+8

Peking University · Shanghai Artificial Intelligence Laboratory

VLM Evaluation Citation Verification Document QA Citation Hallucination Evidence-Grounded QA

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) have significantly advanced document understanding, yet current Doc-VQA evaluations score only the final answer and leave the supporting evidence unchecked. This answer-only approach masks a critical failure mode: a model can land on the correct answer while grounding it in the wrong passage---a critical risk in high-stakes domains like law, finance, and medicine, where every conclusion must be traceable to a specific source region. To address this, we introduce CiteVQA, a benchmark that requires models to return element-level bounding-box citations alongside each answer, evaluating both jointly. CiteVQA comprises 1,897 questions across 711 PDFs spanning seven domains and two languages, averaging 40.6 pages per document. To ensure fidelity and scalability, the ground-truth citations are generated by an automated pipeline---which identifies crucial evidence via masking ablation and enforces multi-stage quality control. At the core of our evaluation is Strict Attributed Accuracy (SAA), which credits a prediction only when the answer and the cited region are both correct. Auditing 20 MLLMs reveals a pervasive Attribution Hallucination: models frequently produce the right answer while citing the wrong region. The strongest system (Gemini-3.1-Pro-Preview) achieves an SAA of only 76.0, and the strongest open-source MLLM reaches just 22.5. Ultimately, towards trustworthy document intelligence, CiteVQA exposes a reliability gap that answer-only evaluations overlook, providing the instrumentation needed to close it. Our repository is available at https://github.com/opendatalab/CiteVQA.

Abstract

Citation Verification

Explore similar work

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models

CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era