EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy
Authors: Kening Zheng, Aoying Zheng, Zhigang Chang, Yazhi Guo, Miaotian Guo, Qingwei Zong, Xianhai Xie, Weiqiang Jin, Chengze Li, Hanrong Zhang, Jie Yang, Wei-Chieh Huang, Lingzhe Zhang, Liancheng Fang, Xin Zou, Hanqian Li, Jiahao Huo, Yibo Yan, Zizhuang Deng, Lei Miao, Wei Guo, Haihong Tang, Bo Zheng, Philip S. Yu
Organizations: University of Illinois Chicago · Taobao & Tmall Group · Shandong University · Xi’an Jiaotong University · Peking University · The Hong Kong University of Science and Technology (Guangzhou)
Long-form factuality verification is commonly implemented as a static decompose-search-verify pipeline, with separately prompted modules processing claims and invoking external search. Treating claims independently makes LLM and search calls scale with claim count and causes repeated searches for overlapping evidence about related claims. We introduce EAVer, an End-to-end Agentic Verifier that learns to control the complete response-level verification workflow as a unified policy. EAVer groups semantically related claims, routes each group to direct verification or targeted search based on confidence, and keeps evidence returned by search in compact in-context memos for cross-claim reuse. To train this policy, we develop a privileged-teacher synthesis pipeline that converts gold claim annotations into executable multi-turn tool-interaction trajectories with live search rather than post-hoc rationales. Structural, label-alignment, tool-use, search-budget, and leakage checks yield 1,447 quality-controlled trajectories. We further construct 794 bidirectional same-trajectory preference pairs that keep claim grouping, search, and evidence fixed, enabling decision-focused Direct Preference Optimization (DPO) over factuality-decision tokens. The results with Qwen3-8B show that EAVer outperforms the strongest search-based baseline on each benchmark by 2.88 Macro-F1 points on VeriFastScore and 4.73 points on the out-of-distribution FaStFact-Bench, while using about 80% fewer searches than the most search-efficient baseline. Moreover, EAVer consistently improves performance across models ranging from 4B to 32B parameters, demonstrating its strong generalizability.
Figures & tables
Figure 1: Search efficiency of EAVer. Left: Existing per-claim evaluators repeatedly search for overlapping evidence; EAVer adaptively searches and reuses evidence across related claims. Right: Mean searches per sample and link redundancy on an independent VeriFastScore validation split. Link redundancy is the fraction of links repeated within the same evaluation sample.
Figure 2: Fixed verification pipelines versus EAVer. Left: FActScore, SAFE, and VeriScore verify claims independently; VeriFastScore batches sentence-level search. Center: Fixed pipelines have four limitations: high API and LLM costs, no end-to-end training, cross-claim search redundancy, and static claim processing. Right: EAVer groups related claims, routes them to direct verification or search by confidence, and stores evidence for reuse.
Figure 3: A representative EAVer inference trajectory. Five claims form two entity groups. The high-confidence group is verified directly; the medium-confidence group uses two searches and shared evidence before all five verdicts are returned.
Stage
Items
Searches /traj.
Length (k chars)
Sup.
Unsup.
SFT trajectories
1,447
2.24
9.16
70.4%
29.6%
DPO pairs (chosen)
794
3.69
13.80
61.7%
38.3%
Table 1: Synthetic training-data overview. Trajectory length counts generated text and injected search-result observations after the initial prompt, giving a tokenizer-independent measure. Label shares are computed over 19,334 SFT decisions and 15,686 decisions in the DPO chosen completions. Each rejected DPO completion shares the chosen completion’s searches and evidence and differs only at corrected decision labels; the 794 pairs comprise 405 false-support-to-unsupported and 389 false-unsupported-to-supported pairs, respectively.
Method
Searches/sample
VeriFastScore (%)
FaStFact-Bench (%)
Macro F1
FSR ↓
claim_ext_F1
Macro F1
FSR ↓
claim_ext_F1
Zero-shot LLM verifiers
Qwen2.5-7B-Instruct
–
43.61
96.71
64.69
48.58
96.66
71.30
Qwen3-8B
–
49.54
89.18
65.63
54.30
89.72
71.83
Qwen3-max
–
60.07
74.65
73.75
70.93
58.64
70.12
GPT-4.1
–
61.30
74.23
69.14
70.73
61.45
72.11
Table 2: Main results. Results on VeriFastScore (in-domain) and FaStFact-Bench (out-of-domain, human-annotated). Searches/sample averages search-API requests over both datasets; LLM calls are excluded, and “–” denotes no external search. Evaluation metrics are percentages. Best reported values in each column are bold .
Figure 4: Decision-focused DPO lowers FSR for EAVer on VeriFastScore. Results are reported on VeriFastScore across three backbones. Circles show EAVer before DPO, and diamonds show EAVer after DPO. Arrows connect matched checkpoints; leftward movement indicates lower FSR, while upward movement indicates higher Macro-F1. Colors identify backbones.
Figure 5: Mechanism-level component analysis for Qwen3-8B. (a) On FaStFact-Bench, entity grouping reduces token-level trajectory length per sample. (b) On FaStFact-Bench, disabling memo-based evidence reuse increases searches per sample and repeated-link redundancy. (c) On VeriFastScore, bars show claim share and final verification accuracy by confidence level.
Figure 6: Cross-backbone generalization in Macro-F1 and FSR. Base and EAVer -trained models are evaluated on the out-of-distribution FaStFact-Bench across six Qwen and Llama backbones. Higher Macro-F1 and lower FSR indicate better performance; EAVer improves both metrics for every tested model and scale.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Setting
Value
Data
Demonstrations
1,447 multi-turn policy trajectories
Sequence
Maximum sequence length
8,192 tokens, no sample packing
Objective
Token supervision
Assistant content and assistant end-of-turn marker only
Optimization
Optimizer
AdamW
Optimization
Learning rate and schedule
2×10−5 , cosine decay
Optimization
Warmup and weight decay
0.05 warmup ratio, 0.01 weight decay
Appendix
Table 4: Shared policy demonstration training configuration. The same semantic training corpus and objective are used for all backbones. Only the memory-sensitive settings listed in Table 5 vary.
Backbone
Epochs run
Selected
Grad. accum.
Global batch
ZeRO stage
Loader workers
Qwen2.5-7B-Instruct
3
2
2
16
2
2
Qwen3-4B
3
2
2
16
2
2
Qwen3-8B
3
2
2
16
2
2
Qwen3-14B
3
2
2
16
2
2
Qwen3-32B
2
2
1
8
3
0
Llama-3.1-8B-Instruct
3
2
2
16
2
2
Appendix
Table 5: Backbone-specific training settings. The selected epoch is used for the reported evaluations. The 32B changes are imposed by host and device memory constraints rather than backbone-specific tuning.
Setting
Value
Initialization
Selected policy checkpoint
Preference pairs
794
Epochs
3
Per-device batch size
1
Effective global batch
32
Learning rate
5×10−7
Appendix
Table 6: Decision-focused preference training configuration.
Open-search fact checking is not merely retrieval followed by classification, but a sequential decision problem in which every query, source visit, and stopping decision reshapes the evidence available for verification. Yet existing systems often distribute these decisions across predefined pipelines or separately prompted modules rather than learning them as a unified task-specific policy. We introduce OpenFC, a unified verification-policy training framework that post-trains Qwen3-8B as a compact next-action controller over reasoning, evidence acquisition, and stopping. OpenFC learns this policy in two stages. Stepwise-Calibrated Cold Start (SCCS) uses a strong training-time supervisor to review post-initial reasoning, tool-use, and stopping proposals before execution, producing reliable trajectories for supervised fine-tuning without access to gold verdicts. Verification-Aware Reinforcement Learning (VA-RL) then improves the cold-start policy on unresolved claims through budget-aware tool rewards, label-aware advantage reweighting, and localized response masking. Across six fact-checking benchmarks, OpenFC achieves 70.39% average accuracy and 63.30% macro-F1, the highest overall averages among the evaluated methods. Stage-wise ablations further show that SCCS and VA-RL provide complementary gains, supporting the design of the two-stage training framework. These results position OpenFC as a strong and effective framework for open-search fact-checking. We will open-source our code and release the model checkpoints to support reproducibility.
Xinming Wang, Kaixiang Qiu, Yansong Lin +5
Institute of Automation, Chinese Academy of Sciences · University of Electronic Science and Technology of China · Beijing Institute of Technology +1
Recent approaches combining Large Language Models (LLMs) with retrieval-augmented reasoning have shown promise for automated fact verification. To process complex claims, these verification pipelines typically execute multi-stage workflows that coordinate tightly coupled modules, including claim decomposition, evidence gathering, and verdict prediction. However, existing methods optimize individual stages in isolation or rely on fixed heuristics, which limits adaptive coordination among stages and can lead to suboptimal outcomes. In this work, we propose ProFact, an agentic reinforcement learning framework for end-to-end optimization of multi-stage fact verification trajectories. ProFact trains a unified policy to coordinate claim decomposition, evidence seeking, answer generation, and verdict prediction. To address the sparse and delayed supervision provided by final veracity labels, ProFact introduces process-aware rewards that provide stage-level learning signals throughout the verification process. Empirical evaluation shows that ProFact consistently outperforms strong baselines in both verification performance and inference efficiency. These results highlight the effectiveness of process-aware trajectory optimization for multi-stage fact verification.
Rongxin Yang, Shenghong He, Siyuan Zhu +1
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China
Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit. We present SEVA, a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis with actionable fixes. Training such an agent with RL is non-trivial: standard binary reward on multi-component output triggers advantage collapse -- within-group reward variance vanishes and the GRPO gradient disappears. We resolve this with a process reward that decomposes verification quality into five independent components weighted 70/30 toward process signals, restoring the gradient and inducing an implicit curriculum -- the agent first masters verification behavior (alignment 0.917 -> 0.997, format 72% -> 100%), then outcomes (F1 64.9 -> 69.0). Structured output further enables a Verify -> Reflect -> Probe -> Refine self-evolution loop, which over four rounds on a 7B model surfaces an unexpected structural finding: each round produces a benchmark-specialist, not a generalist (+15 pp on HaluEval, -10 to -14 pp on TruthfulQA in the same model, persistent at 4x data). On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output -- confirming a principle that should generalize: for any RL task with multi-component generation, reward granularity must match output granularity.
Aojie Yuan, Yi Nian, Haiyue Zhang +2
University of Southern California, Los Angeles, USA · University of Michigan, Ann Arbor, USA