Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification

Yichi Zhang; Nabeel Seedat; Yinpeng Dong; Peng Cui; Jun Zhu; Mihaela van de Schaar

cs.AIMar 3, 2026

Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification

Authors: Yichi Zhang, Nabeel Seedat, Yinpeng Dong, Peng Cui, Jun Zhu, Mihaela van de Schaar

Organizations: Tsinghua University · University of Cambridge · Thomson Reuters Foundational Research

Abstract

As LLM-powered agents have been used for high-stakes decision-making, such as clinical diagnosis, it becomes critical to develop reliable verification of their decisions to facilitate trustworthy deployment. Yet, existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration. To address this, we establish GLEAN, an agent verification framework with Guideline-grounded Evidence Accumulation that compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals. GLEAN evaluates the step-wise alignment with domain guidelines and aggregates multi-guideline ratings into surrogate features, which are accumulated along the trajectory and calibrated into correctness probabilities using Bayesian logistic regression. Moreover, the estimated uncertainty triggers active verification, which selectively collects additional evidence for uncertain cases via expanding guideline coverage and performing differential checks. We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset, surpassing the best baseline by 12% in AUROC and 50% in Brier score reduction, which confirms the effectiveness in both discrimination and calibration. In addition, the expert study with clinicians recognizes GLEAN's utility in practice.

Figures & tables

Figure 1: Guideline-grounded verification in clinical diagnosis. For an agent clinician with access to different examinations (bottom), GLEAN verifies its diagnosis by assessing alignment with clinical guidelines (top) at each step. The example on a patient with acute diverticulitis illustrates how GLEAN accumulates guideline-grounded evidence into calibrated correctness probabilities along the execution (middle). While initial history at the first step aligns well with criteria, physical examination contradicts guidelines on abdominal tenderness and fever, dropping confidence to 0.5. Then, laboratory results recover the confidence, and CT imaging at the last step further confirms the diagnosis with higher confidence.

Figure 2: Properties of guideline-grounded signals. Top: Signals grounded in guidelines significantly discriminate correct from incorrect prefixes, while uninformed signals do not. Bottom: Guideline-grounded signals exhibit strong logit-linearity with correctness, but uninformed ones lack this property.

Figure 3: Pipeline of GLEAN for clinical diagnosis. GLEAN (i) retrieves guidelines for the final diagnosis, (ii) aggregates step-wise scores for alignment with multiple guidelines, (iii) accumulates them into $\beta$ -discounted evidence, which is (iv) calibrated to yield confidence and uncertainty. (v) High uncertainty then triggers active verification via guideline expansion and differential checks.

	Diverticulitis				Cholecystitis				Pancreatitis
Method	AUROC $\uparrow$	Risk @ 0.5	$\downarrow$ ECE $\downarrow$	Brier $\downarrow$	AUROC $\uparrow$	Risk @ 0.5	$\downarrow$ ECE $\downarrow$	Brier $\downarrow$	AUROC $\uparrow$	Risk @ 0.5	$\downarrow$ ECE $\downarrow$	Brier $\downarrow$
Agent Backbone: Qwen2.5-7B-Instruct
$P(\texttt{TRUE})$	0.7280	0.3642	0.3445	0.3379	0.6225	0.2810	0.3640	0.3715	0.6565	0.5610	0.4009	0.4103
LLM-as-a-Judge	0.7709	0.3457	0.2558	0.2859	0.6426	0.2557	0.0917	0.2259	0.6528	0.5523	0.3701	0.3549
Self-Consistency	0.9011	0.2160	0.1880	0.1677	0.7978	0.1646	0.0834	0.1484	0.7621	0.4709	0.2582	0.2446
Semantic Entropy	0.8573	0.2531	–	–	0.7772	0.1747	–	–	0.7174	0.4884	–	–

Table 1: Verification performance across three diseases for two agent backbones. We report AUROC, Risk @ 0.5, ECE, and Brier. We omit calibration metrics for Semantic Entropy since it is not a probability. For GLEAN, we report active verification with an entropy threshold of 0.5, and also a zero-threshold setting as an upper bound ( gray ). Best results are in bold , and second-best are underlined .

Strategy	AUROC $\uparrow$	Risk @ 0.5 $\downarrow$	ECE $\downarrow$	Brier $\downarrow$
Agent Backbone: Qwen2.5-7B-Instruct
GLEAN ( $K=3$ )	0.9137	0.1678	0.0418	0.1178
w. GE	0.9322	0.1422	0.0564	0.1050
w. DC	0.9304	0.1444	0.0565	0.1062
w. Both	0.9470	0.1300	0.0700	0.0947
Agent Backbone: Qwen3-30B-A3B-Instruct

Table 2: Ablation on active verification strategies.

Method	AUROC $\uparrow$	Risk @ 0.5 $\downarrow$	ECE $\downarrow$	Brier $\downarrow$
Components in GLEAN
$P(\texttt{TRUE})$	0.6497	0.3933	0.3719	0.3803
+ Process	0.6935	0.3500	0.3655	0.3721
+ Guideline	0.7944	0.2755	0.2613	0.2703
+ Accumulation	0.8547	0.2367	0.2233	0.2367
+ Calibration	0.8574	0.2378	0.0360	0.1554

Table 3: Ablations on component design and guideline quality.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Properties of guideline-grounded signals accumulated with $\beta$ -discounting.

Table 9

Table 10

Verifier	Agent
	Qwen2.5-7B				Qwen3-30B
	AUROC	Risk @ 0.5	ECE	Brier	AUROC	Risk @ 0.5	ECE	Brier
Qwen2.5-7B	0.9137	0.1678	0.0418	0.1178	0.9056	0.1667	0.0310	0.1209
Qwen3-30B	0.9200	0.1633	0.1165	0.1368	0.9389	0.1267	0.0550	0.0975

Appendix

Table 8: Cross verification across two agent backbones.

	Qwen2.5-7B (Pass @ 1: $52.16\%$ )			Qwen3-30B (Pass @ 1: $58.94\%$ )
Method	BoN-4	BoN-8	BoN-16	BoN-4	BoN-8	BoN-16
P(True)	0.5386	0.5402	0.5380	0.6322	0.6245	0.6133
LLM-as-a-Judge	0.5818	0.6062	0.6200	0.6424	0.6396	0.6467
Self-Consistency	0.6209	0.6617	0.6940	0.6525	0.6865	0.7000
RAG	0.6891	0.7140	0.7240	0.6901	0.6933	0.6900
Self-Verification	0.6366	0.6474	0.6440	0.6743	0.6617	0.6200

Appendix

Table 9: Accuracy with Best-of-N selection.

Figure 7: Annotation interface shown to clinicians showing the step by step agent trajectory (scroll bar) and GLEAN metrics.

LLM Agent Verification

LLM agent verification checks whether agents’ actions, evidence, policies, and completion claims satisfy explicit requirements.

See all 145 papers

VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks
Oct 1, 2026Caiqi Zhang, Rujun Han, Zifeng Wang +4LLM Agent Verification Long-Horizon LLM Agents
University of Cambridge · Google Cloud AI Research
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
Jun 29, 2026Aojie Yuan, Yi Nian, Haiyue Zhang +2AI Agent Reliability Process Reward Models
University of Southern California, Los Angeles, USA · University of Michigan, Ann Arbor, USA
LLM-as-a-Verifier: A General-Purpose Verification Framework
Jul 6, 2026Jacky Kwok, Shulu Li, Pranav Atreya +6LLM Answer Verification LLM Agent Evaluation
Stanford University · UC Berkeley · NVIDIA Research

Oct 1, 2026J/K move · Enter open · S save

VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks

Caiqi Zhang, Rujun Han, Zifeng Wang, Zoey CuiZhu+3

University of Cambridge · Google Cloud AI Research

LLM Agent Verification Long-Horizon LLM Agents LLM Agent Harnesses

Open paper AskPDF arXiv

As LLM agents undertake increasingly complex, long-horizon tasks, verifying their outputs becomes increasingly challenging. We study how verification capability can be strengthened with a fixed base model, without access to reference answers or grading rubrics at test time. Repeated sampling yields multiple rollouts that can contain complementary correct claims, but we need a reliable verification mechanism to determine which claims to trust. We first find that disagreement often exposes correct alternatives, while consensus can conceal errors. These observations motivate VeriHarness, which turns the underlying LLM a generator uses into an agentic verifier by giving it a workspace, evidence tools, and reusable verification skills. A disagreement resolver checks competing claims against environmental evidence, while a consensus challenger tests shared claims and searches for omitted requirements. Their findings guide the selection and revision of the final artifact. Across five long-horizon workspace benchmarks and two frontier models, VeriHarness achieves the highest selection scores among the evaluated baselines. Evidence-backed revision further improves average performance, bringing gains over a single rollout to 6.2 points with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. We further show that verification skills can self-improve from failure feedback, demonstrating VeriHarness as a novel and critical approach for scaling long-horizon agentic verification. We release the full pool of approximately 26,000 rollouts across all five benchmarks and both models, produced at a cost of over $100,000, to support future research on agentic verification.

Abstract

LLM Agent Verification

Explore similar work

VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks

SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution

LLM-as-a-Verifier: A General-Purpose Verification Framework