EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy

Kening Zheng; Aoying Zheng; Zhigang Chang; Yazhi Guo; Miaotian Guo; Qingwei Zong; Xianhai Xie; Weiqiang Jin

cs.CLSep 2, 2026

EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy

Authors: Kening Zheng, Aoying Zheng, Zhigang Chang, Yazhi Guo, Miaotian Guo, Qingwei Zong, Xianhai Xie, Weiqiang Jin,

Organizations: University of Illinois Chicago · Taobao & Tmall Group · Shandong University · Xi’an Jiaotong University ·

Abstract

Long-form factuality verification is commonly implemented as a static decompose-search-verify pipeline, with separately prompted modules processing claims and invoking external search. Treating claims independently makes LLM and search calls scale with claim count and causes repeated searches for overlapping evidence about related claims. We introduce EAVer, an End-to-end Agentic Verifier that learns to control the complete response-level verification workflow as a unified policy. EAVer groups semantically related claims, routes each group to direct verification or targeted search based on confidence, and keeps evidence returned by search in compact in-context memos for cross-claim reuse. To train this policy, we develop a privileged-teacher synthesis pipeline that converts gold claim annotations into executable multi-turn tool-interaction trajectories with live search rather than post-hoc rationales. Structural, label-alignment, tool-use, search-budget, and leakage checks yield 1,447 quality-controlled trajectories. We further construct 794 bidirectional same-trajectory preference pairs that keep claim grouping, search, and evidence fixed, enabling decision-focused Direct Preference Optimization (DPO) over factuality-decision tokens. The results with Qwen3-8B show that EAVer outperforms the strongest search-based baseline on each benchmark by 2.88 Macro-F1 points on VeriFastScore and 4.73 points on the out-of-distribution FaStFact-Bench, while using about 80% fewer searches than the most search-efficient baseline. Moreover, EAVer consistently improves performance across models ranging from 4B to 32B parameters, demonstrating its strong generalizability.

Figures & tables

Figure 1: Search efficiency of EAVer. Left: Existing per-claim evaluators repeatedly search for overlapping evidence; EAVer adaptively searches and reuses evidence across related claims. Right: Mean searches per sample and link redundancy on an independent VeriFastScore validation split. Link redundancy is the fraction of links repeated within the same evaluation sample.

Figure 2: Fixed verification pipelines versus EAVer. Left: FActScore, SAFE, and VeriScore verify claims independently; VeriFastScore batches sentence-level search. Center: Fixed pipelines have four limitations: high API and LLM costs, no end-to-end training, cross-claim search redundancy, and static claim processing. Right: EAVer groups related claims, routes them to direct verification or search by confidence, and stores evidence for reuse.

Figure 3: A representative EAVer inference trajectory. Five claims form two entity groups. The high-confidence group is verified directly; the medium-confidence group uses two searches and shared evidence before all five verdicts are returned.

Stage	Items	Searches /traj.	Length (k chars)	Sup.	Unsup.
SFT trajectories	1,447	2.24	9.16	70.4%	29.6%
DPO pairs (chosen)	794	3.69	13.80	61.7%	38.3%

Table 1: Synthetic training-data overview. Trajectory length counts generated text and injected search-result observations after the initial prompt, giving a tokenizer-independent measure. Label shares are computed over 19,334 SFT decisions and 15,686 decisions in the DPO chosen completions. Each rejected DPO completion shares the chosen completion’s searches and evidence and differs only at corrected decision labels; the 794 pairs comprise 405 false-support-to-unsupported and 389 false-unsupported-to-supported pairs, respectively.

Zero-shot LLM verifiers
Method	Searches/sample	VeriFastScore (%)			FaStFact-Bench (%)
Method	Searches/sample	Macro F1	FSR $\downarrow$	claim_ext_F1	Macro F1	FSR $\downarrow$	claim_ext_F1
Qwen2.5-7B-Instruct	–	43.61	96.71	64.69	48.58	96.66	71.30
Qwen3-8B	–	49.54	89.18	65.63	54.30	89.72	71.83
Qwen3-max	–	60.07	74.65	73.75	70.93	58.64	70.12
GPT-4.1	–	61.30	74.23	69.14	70.73	61.45	72.11

Table 2: Main results. Results on VeriFastScore (in-domain) and FaStFact-Bench (out-of-domain, human-annotated). Searches/sample averages search-API requests over both datasets; LLM calls are excluded, and “–” denotes no external search. Evaluation metrics are percentages. Best reported values in each column are bold .

Figure 4: Decision-focused DPO lowers FSR for EAVer on VeriFastScore. Results are reported on VeriFastScore across three backbones. Circles show EAVer before DPO, and diamonds show EAVer after DPO. Arrows connect matched checkpoints; leftward movement indicates lower FSR, while upward movement indicates higher Macro-F1. Colors identify backbones.

Figure 5: Mechanism-level component analysis for Qwen3-8B. (a) On FaStFact-Bench, entity grouping reduces token-level trajectory length per sample. (b) On FaStFact-Bench, disabling memo-based evidence reuse increases searches per sample and repeated-link redundancy. (c) On VeriFastScore, bars show claim share and final verification accuracy by confidence level.

Policy variant	Macro F1 (%) $\uparrow$	FSR (%) $\downarrow$	Avg. traj. length $\downarrow$	Searches / sample $\downarrow$
Full policy	74.05	35.87	2,474	4.10
w/o entity grouping	71.82	39.41	4,821	6.35
w/o confidence gate	72.19	36.07	4,058	7.65
w/o memo reuse	72.41	40.29	4,260	5.72

Table 3: Component ablations. Entity grouping shortens trajectories, confidence routing avoids unnecessary search, and memo reuse limits redundant searches.

Figure 6: Cross-backbone generalization in Macro-F1 and FSR. Base and EAVer -trained models are evaluated on the out-of-distribution FaStFact-Bench across six Qwen and Llama backbones. Higher Macro-F1 and lower FSR indicate better performance; EAVer improves both metrics for every tested model and scale.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Setting	Value
Data	Demonstrations	1,447 multi-turn policy trajectories
Sequence	Maximum sequence length	8,192 tokens, no sample packing
Objective	Token supervision	Assistant content and assistant end-of-turn marker only
Optimization	Optimizer	AdamW
Optimization	Learning rate and schedule	$2\times 10^{-5}$ , cosine decay
Optimization	Warmup and weight decay	0.05 warmup ratio, 0.01 weight decay

Appendix

Table 4: Shared policy demonstration training configuration. The same semantic training corpus and objective are used for all backbones. Only the memory-sensitive settings listed in Table 5 vary.

Backbone	Epochs run	Selected	Grad. accum.	Global batch	ZeRO stage	Loader workers
Qwen2.5-7B-Instruct	3	2	2	16	2	2
Qwen3-4B	3	2	2	16	2	2
Qwen3-8B	3	2	2	16	2	2
Qwen3-14B	3	2	2	16	2	2
Qwen3-32B	2	2	1	8	3	0
Llama-3.1-8B-Instruct	3	2	2	16	2	2

Appendix

Table 5: Backbone-specific training settings. The selected epoch is used for the reported evaluations. The 32B changes are imposed by host and device memory constraints rather than backbone-specific tuning.

Setting	Value
Initialization	Selected policy checkpoint
Preference pairs	794
Epochs	3
Per-device batch size	1
Effective global batch	32
Learning rate	$5\times 10^{-7}$

Appendix

Table 6: Decision-focused preference training configuration.

LLM Answer Verification

Large language model answer verification checks whether generated answers are correct, supported by evidence, and safe to accept.

See all 150 papers

OpenFC: Learning Verification Policies towards Open-Search Fact Checking
Sep 27, 2026Xinming Wang, Kaixiang Qiu, Yansong Lin +5RL for Language Model Reasoning Automated Fact-Checking
Institute of Automation, Chinese Academy of Sciences · University of Electronic Science and Technology of China · Beijing Institute of Technology +1
From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification
Jun 11, 2026Rongxin Yang, Shenghong He, Siyuan Zhu +1Agentic RL Process Reward Models
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
Jun 29, 2026Aojie Yuan, Yi Nian, Haiyue Zhang +2AI Agent Reliability Process Reward Models
University of Southern California, Los Angeles, USA · University of Michigan, Ann Arbor, USA

Sep 27, 2026J/K move · Enter open · S save

OpenFC: Learning Verification Policies towards Open-Search Fact Checking

Xinming Wang, Kaixiang Qiu, Yansong Lin, Chunji Lv+4

Institute of Automation, Chinese Academy of Sciences · University of Electronic Science and Technology of China · Beijing Institute of Technology+1

RL for Language Model Reasoning Automated Fact-Checking LLM Tool Use

Open paper AskPDF arXiv

Open-search fact checking is not merely retrieval followed by classification, but a sequential decision problem in which every query, source visit, and stopping decision reshapes the evidence available for verification. Yet existing systems often distribute these decisions across predefined pipelines or separately prompted modules rather than learning them as a unified task-specific policy. We introduce OpenFC, a unified verification-policy training framework that post-trains Qwen3-8B as a compact next-action controller over reasoning, evidence acquisition, and stopping. OpenFC learns this policy in two stages. Stepwise-Calibrated Cold Start (SCCS) uses a strong training-time supervisor to review post-initial reasoning, tool-use, and stopping proposals before execution, producing reliable trajectories for supervised fine-tuning without access to gold verdicts. Verification-Aware Reinforcement Learning (VA-RL) then improves the cold-start policy on unresolved claims through budget-aware tool rewards, label-aware advantage reweighting, and localized response masking. Across six fact-checking benchmarks, OpenFC achieves 70.39% average accuracy and 63.30% macro-F1, the highest overall averages among the evaluated methods. Stage-wise ablations further show that SCCS and VA-RL provide complementary gains, supporting the design of the two-stage training framework. These results position OpenFC as a strong and effective framework for open-search fact-checking. We will open-source our code and release the model checkpoints to support reproducibility.

Abstract

LLM Answer Verification

Explore similar work

OpenFC: Learning Verification Policies towards Open-Search Fact Checking

From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification

SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution