AI agents are governed by hundreds of natural language criteria, yet existing verifiers cannot efficiently evaluate an agent's output against a large set of such criteria and report which are satisfied, violated, or inapplicable. We introduce the VFM, a tree-attention verifier that encodes the shared context and target text, and scores N natural language criteria in parallel, returning independent per-criterion ternary verdicts. The VFM is pretrained on CriteriaBank, a corpus of 394K (context, target, criterion, verdict) tuples spanning over 321K distinct criterion strings. A finetuned VFM-4B outperforms GPT-5.4 on a synthetic governance benchmark. On the DynaBench multi-criterion benchmark, it outperforms DynaGuard-4B on trace-level failure detection, and its per-criterion verdicts localize the violated rule and support focused escalation to a larger chain-of-thought judge. We show that CriteriaBank pretraining improves cross-domain transfer, particularly in the low-data regime, while matching a strong direct-finetuning control at full data. On a single H100, VFM-4B evaluates 1,000 criteria against a 4K-token context in under 2s at <11 GB VRAM, making it suitable for runtime verification.
Figures & tables
Figure 1: Verifying Multiple Criteria with the VFM. Natural language criteria c1,…,cN from multiple sources are scored against the agent trace, consisting of context x and target text t . The VFM encodes the x⊕t prefix once, and each criterion ci then branches independently under a tree-attention mask that isolates per-criterion attention while sharing the prefix’s KV cache. The output is a compliance record of softmax probabilities over ternary verdicts ( yes / no / n/a ).
Accepts free-text criteria
Governance rules are expressed in natural language
Adapts to diverse contexts
Deployer policies and user instructions may change based on the interaction
Issues per-criterion verdicts
Monitoring and auditing require knowing which rules passed or failed
Scales efficiently for runtime
Hundreds of criteria must be evaluated with runtime latency
Table 1: Desiderata for an open-ended runtime verifier.
Table 3
Stream
Source
Tuples
Criteria
Role
A (Synthetic)
Concept-first generation
33,657
2,625
Broad criteria coverage
B (Real data)
WildChat/WildJailbreak [ 71 , 24 ]
245,615
237,471
Real user/model scenarios
C (Gap-filling)
Targeted synthetic scenarios
37,926
35,018
Tools, error-correction, business
C+ (Multi-turn)
LMSYS-Chat-1M [ 73 ]
55,662
46,884
Multi-turn conversations
D (Safety)
Synthetic (safety taxonomy)
20,671
2,204
Guardrail-style criteria
Total
393,531
321,419
Table 2: CriteriaBank composition by data stream.
Table 5
Method
CoT
FAIL F1
DynaGuard-4B [ 20 ]
yes
0.720
DynaGuard-8B [ 20 ]
yes
0.731
Gemma4-E4B-it (raw rules)
yes
0.710±0.003
Gemma4-26B-A4B-it (raw rules)
yes
0.922±0.006
Gemma4-26B-A4B-it (decomposed rules)
yes
0.807±0.009
Direct-FT-4B
no
0.843±0.003
Table 5: DynaBench. Finetuned rows show mean ± std across 3 seeds with dev-selected thresholds; the cascade uses the same 3 VFM seeds with deterministic judge calls. Gemma-only rows show mean ± std over 3 deterministic inference runs.
Figure 2: Sample efficiency at 0.6B : VFM (blue), VFM w/o KL (teal), and Direct-FT (red) across six benchmarks under a uniform multi-epoch early-stopping recipe. Bands show ± 1 std. over 3 seeds.
0.6B backbone
4B backbone
Prefix
Path
N=30
N=100
N=1 K
VRAM
N=30
N=100
N=1 K
VRAM
∼ 144
vfm-batched
32
50
311
36 GB
53
115
884
55 GB
llm-prefix
34
52
326
38 GB
57
120
929
57 GB
∼ 1K
vfm-pc, c =64
31
63
454
1.9 GB
63
147
1,223
8.8 GB
llm-prefix
52
122
1,036 †
78 GB
100
250
2,214 †
70 GB
∼ 4K
vfm-pc, c =64
56
106
687
3.2 GB
156
279
1,807
10.5 GB
Table 6: Latency on a single H100 (bf16) . Median wall-clock ms over 5 repeats and peak VRAM at the largest non-OOM batch, for a criterion length of 32 tokens, with HuggingFace Transformers.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Criterion
Context
Target
Label
Rationale
Does the response disclose that it is AI-generated?
User-facing chat: “Help me plan a 3-day trip to Lisbon.”
“Hi! I’m an AI assistant. Here are three options…”
yes
Disclosure required and present
Same
User-facing chat: “Help me plan a 3-day trip to Lisbon.”
“Here are three options…”
no
Required and absent
Same
Internal API call; no user-facing dialogue
{flight: ..., hotel: ...}
n/a
Precondition (consumer-facing context) does not hold
Same
Multi-agent log; “user” role unclear
“Here are three options…”
n/a
Trace insufficient to determine whether disclosure was required
Appendix
Table 7: Examples illustrating the YES/NO/N/A label semantics. The same property concept (criterion) yields different labels depending on whether the precondition holds and whether the trace contains enough evidence to decide.
Stream
Examples
Unique criteria
Avg. trace
Label dist.
A (Synthetic)
33,657
2,625
130 tok
45/50/4%
B (Distillation)
245,615
237,471
1,130 tok
38/51/11%
C (Gap-filling)
37,926
35,018
230 tok
37/50/13%
C+ (Multi-turn)
55,662
46,884
742 tok
42/44/14%
D (Safety)
20,671
2,204
135 tok
51/48/0%
Total
393,531
321,419
—
40/50/10%
Appendix
Table 8: CriteriaBank dataset statistics by stream. Unique criteria counts distinct goal strings within a stream; the total row reports cross-stream unique strings (smaller than the per-stream sum because a small number of exact-string duplicates occur across streams). Avg. trace reports the mean Qwen3-tokenizer length of the chat-templated (context,target) pair conditioned on by the verifier (5K-record sample per stream). Stream B is heavily right-skewed (median ≈612 , p95 ≈3,700 ). Label distribution is YES/NO/ n/a .
Subset
Δa (drop A)
Δb (drop B)
Δc (drop C)
Δd (drop D)
BeaverTails
+2.29±2.01
+3.77±3.48
+1.08±0.48
−4.19±0.93
ContractNLI
−5.44±0.83
−0.40±1.55
−3.47±3.64
+3.17±1.77
IFEval
−3.16±1.75
+2.35±0.94
+0.57±1.21
+0.23±1.36
PRM800K
−1.50±1.76
+0.67±2.59
+0.50±0.58
+3.17±2.40
Governance
+0.69±4.22
+1.02±3.37
−2.55±3.65
−2.29±3.23
Appendix
Table 9: Fixed-budget LOO at 30K samples. Paired-seed Δ vs. full (mean ± SEM, n=3 seeds, balanced accuracy, percentage points). Δx=loox−full ; positive means dropping stream x helped.
Subset
noB @ 788
noB @ 2787
full @ 2787
Δ matched-EP
Δ matched-ST
Stream A (val)
80.93
91.39
85.71
−4.78
+5.68
Stream B (val)
86.61
89.40
94.33
−7.71
−4.93
Stream C (val)
83.40
93.90
93.25
−9.85
+0.65
Stream D (val)
95.20
97.59
99.21
−4.01
−1.61
BeaverTails
68.78
77.42
74.40
−5.62
+3.02
ContractNLI
49.27
62.03
54.41
−5.13
+7.63
Appendix
Table 10: Full-corpus noB vs. full at 4B. Balanced accuracy on five downstream subsets and four contamination-filtered per-stream val subsets. Matched-EP: one epoch each. Matched-ST: 3.54 epochs of noB vs. one epoch of full. Bold : ∣Δ∣≥3 pp.
Parameter
CriteriaBank pretraining
Downstream finetuning
Head LR
5×10−4
1×10−4
Backbone LR
5×10−5
5×10−5
Schedule
cosine
cosine
Warmup ratio
3%
10%
Epochs
1
≤30 , early stopping (patience 5)
KL regularizer
λ=1 (omitted for VFM w/o KL)
none
Appendix
Table 11: Training hyperparameters for CriteriaBank pretraining and downstream finetuning. Shared settings: no LoRA, effective batch size 128, max sequence length 4096 (8192 for ContractNLI), AdamW optimizer, bf16, weight decay 0.01, max gradient norm 1.0.
Benchmark
Setting
Records †
Criteria
Labels
Metric
Governance (§ 4.1 )
synthetic compliance
920
32 regulatory rules
ternary
Acc
Governance transfer (§ 4.1 )
synthetic compliance
951
2 held-out regulations
ternary
Acc
INSURE-Dial (§ 4.2 )
insurance-benefit calls
417
6 - 9 procedural checks
ternary
Acc/F1
DynaBench (§ 4.3 )
agent guardrails
543
∼14 rules (max 92 )
binary
F1
BeaverTails (§ 4.4 )
response safety
1,000
1 harm label
binary
F1
IFEval (§ 4.4 )
instruction following
3,418
1,504 constraints
binary
BalAcc
Appendix
Table 12: Evaluation benchmarks. Top four are governance-related, and bottom five probe standard verification tasks. Records ( † ) counts test-time (criterion, trace) pairs except DynaBench, which scores at the trace level. DynaBench and ContractNLI use their full official test splits. BeaverTails, IFEval, PRM800K, and LLM-AggreFact use subsamples, detailed below.
Model
BT (F1)
IFEval (BalAcc)
PRM (BalAcc)
AggreF (BalAcc)
CNLI (mF1)
Qwen3Guard-Gen-4B
85.4 [83.0, 87.5]
—
—
—
—
Qwen2.5-Math-7B-PRM800K
—
—
81.6 [80.2, 83.0]
—
—
Qwen3-4B (direct)
49.8 [45.2, 54.1]
56.1 [55.0, 57.1]
74.2 [72.6, 75.7]
74.8 [72.8, 76.7]
51.5 [49.2, 53.6]
Qwen3-Next-80B-A3B (CoT)
80.0 [77.2, 82.6]
70.1 [68.5, 71.6]
84.5 [83.2, 85.8]
81.7 [80.1, 83.4]
71.2 [68.9, 73.3]
Linear Probe (Qwen3-4B)
81.2
83.5
76.7
—
47.6
Direct-FT-4B
83.5 ± 0.7
94.6 ± 0.7
82.1 ± 0.6
75.4 ± 1.8
84.1 ± 0.6
Appendix
Table 13: Full-data cross-domain results (higher is better). Best external and best finetuned results per benchmark in bold . External baselines: 95% bootstrap CIs. Finetuned: mean ± std over 3 seeds. LLM-judge cells score valid parses only. Under 1% of responses are unparsed, except for the direct Qwen3-4B judge on IFEval (10%), which often carries out the instruction instead of judging it.
Prefix ∼ 1K
Prefix ∼ 4K
Method
N=100
N=1 K
N=100
N=1 K
VFM backbone (1-tok readout)
284
2,437
871
8,138
1-tok judge (logprobs)
284
2,407
875
8,130
LMUnit-style scorer
816
4,621
1,665
12,943
CoT judge
908
5,321
1,819
14,004
VFM, HuggingFace
147
1,223
279
1,807
Appendix
Table 14: Latency under vLLM automatic prefix caching (single H100, 4B models, bf16). Median wall-clock ms over 5 repeats after 2 warmups under identical serving configurations. The HuggingFace row repeats the vfm-pc row of Table 6 .
Model
InfoB
Gov
CNLI
INSURE
BT
IFEval
PRM
AggreF
lm-init-0.6B
0.603
0.793
0.766
0.708
0.615
0.458
0.472
0.598
VFM-0.6B (w/o KL)
0.844
0.888
0.852
0.663
0.734
0.651
0.577
0.752
VFM-0.6B
0.887
0.880
0.857
0.713
0.722
0.654
0.635
0.736
lm-init-4B
0.591
0.791
0.821
0.654
0.692
0.554
0.524
0.763
VFM-4B (w/o KL)
0.891
0.909
0.893
0.686
0.777
0.714
0.781
0.851
VFM-4B
0.942
0.927
0.899
0.671
0.779
0.704
0.782
0.857
Appendix
Table 15: Zero-shot ROC-AUC (no finetuning, no threshold tuning). Ternary benchmarks (governance, ContractNLI, INSURE-Dial) are scored on their YES/NO subset. lm-init = verbalizer-initialized head on the raw backbone, no CriteriaBank pretraining. LMUnit-70B is run on the binary benchmarks only (see text).
Pair
Ternary agreement ( κ )
yes vs. ¬ yes ( κ )
no vs. ¬ no ( κ )
A1 – A2
.67 (.51)
.92 [.84,.96] (.82 [.67,.94])
.67 (.28)
A1 – A3
.73 (.60)
.88 [.79,.94] (.73 [.55,.88])
.77 (.41)
A2 – A3
.65 (.49)
.88 [.79,.94] (.72 [.54,.88])
.71 (.38)
A1 – Labeler
.81 (.72)
.92 [.84,.96] (.82 [.69,.94])
.81 (.54)
A2 – Labeler
.65 (.48)
.87 [.77,.93] (.69 [.51,.86])
.67 (.30)
A3 – Labeler
.69 (.54)
.88 [.79,.94] (.73 [.55,.88])
.72 (.35)
Appendix
Table 16: Round-1 pairwise agreement on the 75-item governance subset under three label projections. The yes vs. ¬ yes projection collapses the n/a / no distinction and is the one reported in Section 4.1 , so only it shows 95% CIs (Wilson on agreement, bootstrap on κ ).
Pair
Ternary agreement ( κ )
yes vs. ¬ yes ( κ )
no vs. ¬ no ( κ )
A1 – A2
.81 (.71)
.89 [.80,.94] (.77 [.60,.91])
.85 (.70)
A1 – A3
.81 (.70)
.87 [.77,.93] (.71 [.54,.86])
.87 (.73)
A2 – A3
.76 (.62)
.86 [.77,.92] (.72 [.55,.86])
.80 (.59)
A1 – Teacher
.73 (.60)
.93 [.85,.97] (.85 [.70,.97])
.76 (.52)
A2 – Teacher
.70 (.55)
.91 [.82,.95] (.80 [.64,.93])
.78 (.53)
A3 – Teacher
.71 (.56)
.88 [.79,.94] (.74 [.58,.89])
.79 (.56)
Appendix
Table 17: Pairwise agreement on the 75-item CriteriaBank subset under the same three label projections as Table 16 . CIs (Wilson 95% on agreement, bootstrap 95% on κ , B=2000 ) are shown for the yes vs. ¬ yes projection only. A2 missing on 1 item drops n for affected pairs.
Benchmark
k≥2 traces
top-1
top-3
recall@ k
all-in-top- k
INSURE-Dial
35
0.676
0.933
0.666
0.505
Governance
38
0.956
1.000
0.824
0.342
Appendix
Table 18: Multi-violation localization (VFM-4B, 3-seed mean) on the k≥2 subset (traces violating ≥2 criteria) of two per-criterion benchmarks. Top-1/top-3 ask whether the highest / top-3 ranked criteria include a true violation, recall@ k is the fraction of the k violations in the top- k , all-in-top- k requires all of them.
∣C∣ stratum
traces
% FAIL
Gemma-only CoT
Cascade top-3 CoT
Δ
≤10
220
50.0%
0.944
0.930
−1.4 pp
11 – 20
149
32.2%
0.898
0.941
+4.4 pp
21 – 40
112
42.0%
0.898
0.952
+5.4 pp
>40†
62
100.0%
0.919
0.975
+5.7 pp
all
543
49.2%
0.922
0.946
+2.4 pp
Appendix
Table 19: F1 stratified by ∣C∣ (atomic criteria per trace), 3-seed means (the KL-VFM cascade of Table 5 and the 3-seed Gemma-only baseline on the same 543 traces). “% FAIL” is the ground-truth fraction.
Escalated fraction e
Traces to judge
FAIL-F1
0.00 (VFM alone)
0
0.893±0.006
0.15
81
0.917±0.005
0.25
136
0.928±0.002
0.50
272
0.937±0.004
1.00 (full cascade)
543
0.946±0.005
Appendix
Table 20: Selective escalation on the DynaBench test set (KL-VFM-4B ranker, 3 seeds). Escalating the least-confident fraction e of traces to the Gemma-4-26B judge and deciding the rest with the VFM alone. e=0 is the VFM standalone, e=1 the full top-3 cascade; e was not selected on a separate development set.
Benchmark
VFM-4B P / R / F1
Frontier judge
Judge P / R / F1
VFM pass error
Governance (in-distribution)
83.8 / 87.4 / 85.5
GPT-5.4
82.8 / 77.9 / 80.3
5.6%
Governance (held-out regulations)
76.7 / 73.0 / 74.6
GPT-5.4
76.8 / 75.5 / 76.1
11.1%
INSURE-Dial (real calls)
82.4 / 75.1 / 78.6
GPT-5.4
60.2 / 86.1 / 70.8
19.4%
DynaBench (traces)
83.1 / 94.9 / 88.5
Gemma-4-26B
96.0 / 88.8 / 92.2
5.1%
Appendix
Table 21: Violation-centric results across governance benchmarks. P/R/F1 are the precision, recall, and F1 of detecting a violation, in percent. VFM-4B results are means over three seeds. GPT-5.4 rows are single deterministic runs (on INSURE-Dial, with the extended schema), and the Gemma-4-26B row, given the raw DynaBench rules, is the mean over three deterministic runs.
As AI agents take on increasingly open-ended and complex tasks, verifying their outputs has proven to be difficult. Rubric-graded agent environments, where a verifier judges multiple natural-language criteria against the deliverables and environment state changes produced by the agent, have emerged as a popular paradigm. We conduct the first systematic study of verifiers for rubric-graded agent environments. First, we create BankerVerifierBench (BVB), a meta-evaluation dataset of 2,464 human-judged rubric criteria across 24 investment-banking tasks. Next, we develop a methodology that derives from any rubric corpus the capabilities a verifier should possess, yielding a nine-capability taxonomy that we distill into three general design principles---reactive verification, environment alignment, and domain guidance---which we implement in Gandalf, an open-source verifier. Finally, we evaluate Gandalf against three existing verifiers on BVB. Gandalf matches or exceeds every baseline's F1 at lower cost: with Gemini 3 Flash it matches the strongest baseline's F1 (0.864 vs. 0.852) at one-fifth of its cost, and with GPT-5.4 models it improves F1 by over 0.12 at lower cost. Gandalf also generalizes to OpenClaw, a structurally different personal-productivity environment, where it again achieves the highest F1 (0.955 vs. 0.934 for the best baseline). We release BVB, Gandalf, and the code and data to reproduce our experiments.
As LLM agents undertake increasingly complex, long-horizon tasks, verifying their outputs becomes increasingly challenging. We study how verification capability can be strengthened with a fixed base model, without access to reference answers or grading rubrics at test time. Repeated sampling yields multiple rollouts that can contain complementary correct claims, but we need a reliable verification mechanism to determine which claims to trust. We first find that disagreement often exposes correct alternatives, while consensus can conceal errors. These observations motivate VeriHarness, which turns the underlying LLM a generator uses into an agentic verifier by giving it a workspace, evidence tools, and reusable verification skills. A disagreement resolver checks competing claims against environmental evidence, while a consensus challenger tests shared claims and searches for omitted requirements. Their findings guide the selection and revision of the final artifact. Across five long-horizon workspace benchmarks and two frontier models, VeriHarness achieves the highest selection scores among the evaluated baselines. Evidence-backed revision further improves average performance, bringing gains over a single rollout to 6.2 points with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. We further show that verification skills can self-improve from failure feedback, demonstrating VeriHarness as a novel and critical approach for scaling long-horizon agentic verification. We release the full pool of approximately 26,000 rollouts across all five benchmarks and both models, produced at a cost of over $100,000, to support future research on agentic verification.
Caiqi Zhang, Rujun Han, Zifeng Wang +4
University of Cambridge · Google Cloud AI Research
Agentic search requires language model agents to explore many sources and answer complex information-seeking questions. Scaling test-time compute is a promising way to improve these agents, but current approaches can fail, because correct answers are often sparse and score-based selection depends on model calibration. We propose FineVerify, a fine-grained self-verification framework that decomposes each question into checkable sub-questions, verifies sampled candidates against each sub-question, and selects the candidate with the highest aggregated score. This per-check structure turns selection into simpler local judgments and produces scores under the same explicit criteria. Across four agentic search benchmarks and two models, FineVerify consistently outperforms standard scaling baselines. With only four sampled trajectories, it improves GPT-5-mini by 8.2 accuracy points and Gemini-3-flash by 5.6% on average. With 12 samples, FineVerify enables GPT-5-mini to surpass frontier GPT-5 on BrowseComp-Plus. Beyond accuracy, FineVerify produces interpretable verification traces that help audit benchmark errors, suggesting broader applications for inspecting agentic search systems. Code and data are available at https://github.com/XuZhao0/fineverify