Checking whether a generated sentence is supported by its source document is a standard safeguard for AI-written text, and the strongest published checkers are single models. We describe GauntletScore, an evidence engine whose few rules and thresholds were set on the benchmark's development split. It puts typed questions to independent readers from different providers and records each answer as a measurement in an evidence ledger. It combines the measurements with a fixed Bayesian rule that counts a contradiction only when two readers agree. On the full LLM-AggreFact test split (29,320 claims across 11 datasets), following the leaderboard's approach, GauntletScore achieves 79.51 balanced accuracy, above every average published on the leaderboard as of 15 November 2026. That places it level with the best of them, nimble-v4-max (79.45 from its published dataset scores; the engine's 95% interval, 78.47 to 80.53, contains it). On the same claims it beats two of its three readers used alone (Readers A and B) by 1.66 and 2.17 points (95% intervals 0.80 to 2.53 and 1.29 to 3.06). Without AggreFact-CNN, described below, the leads are 0.90 and 1.32 and both intervals stay above zero. The engine also records the evidence behind every verdict, which a single model does not. Its largest margin is on full-length news articles checked against multi-sentence summaries (AggreFact-CNN), a format that resembles much regulated writing. There it scores 76.45, the best on the leaderboard, against 70.1 for the next best and 63.9 to 67.8 for the six systems with the highest averages. The test split was run once, on an engine frozen before the first test claim, under a stopping plan committed in advance. We release per-claim predictions and the scoring script.
Figures & tables
Figure 1: The engine’s stages, from a claim and its source document to a verdict with its reason.
Dataset
Claims
RAGTruth
16,371
ExpertQA
3,702
LFQA
1,911
REVEAL
1,710
FactCheck-GPT
1,566
ClaimVerify
1,088
Table 1: Claims per dataset in the LLM-AggreFact test split.
Model
Size
Avg
CNN
XSum
MediaS
MeetB
WiCE
REVEAL
ClaimVerify
FactCheck
ExpertQA
LFQA
RAGTruth
GauntletScore (this work)
–
79.51
76.45
74.95
74.74
80.38
81.94
90.63
77.52
80.89
59.98
89.33
87.84
nimble-v4-max
27B
79.5
67.6
78.6
73.3
82.4
87.1
89.0
75.9
80.2
60.4
89.0
90.5
Clef
27B
79.0
67.3
75.7
75.3
81.1
85.8
89.6
78.6
80.8
59.0
89.0
87.3
Decider v1.2
27B
78.7
65.5
77.9
72.4
80.3
87.1
90.3
75.8
78.7
60.6
88.4
89.0
Jev 1.13.0
–
78.7
67.2
75.5
74.5
83.7
83.9
90.9
76.9
77.0
59.9
89.2
87.0
d1-ultra
–
78.5
63.9
77.6
74.4
79.9
82.7
91.6
76.8
79.0
60.2
87.9
89.2
Table 2: LLM-AggreFact test split, balanced accuracy (%). Published rows read from the leaderboard ( https://llm-aggrefact.github.io/ ) on 9 November and rechecked on 15 November 2026; the leaderboard shows one decimal. The eleven published scores for nimble-v4-max average 79.45. The published rows are the six systems with the highest averages.
Dataset
Engine
Reader A alone
Reader B alone
Engine minus A
Engine minus B
Average of 11
79.51
77.85
77.34
+ 1.66
+ 2.17
AggreFact-CNN
76.45
67.12
65.77
+ 9.33
+ 10.68
LFQA
89.33
84.87
87.14
+ 4.46
+ 2.19
ClaimVerify
77.52
73.54
74.41
+ 3.98
+ 3.11
REVEAL
90.63
88.04
90.78
+ 2.59
− 0.15
TofuEval-MediaS
74.74
72.37
69.98
+ 2.37
+ 4.76
Table 3: The engine against Readers A and B alone, on the same claims, balanced accuracy (%). Full test split, 29,320 claims. Intervals are under Statistical tests below.
System
Supported claims kept (true-positive rate)
Unsupported claims caught (true-negative rate)
Balanced accuracy
Engine
82.5
77.5
80.00
Reader A alone
89.5
67.8
78.67
Reader B alone
89.0
66.6
77.81
Reader C alone
79.7
79.5
79.63
Table 4: Supported claims kept and unsupported claims caught, 2,992-claim sample (%). Means over the 11 datasets.
Document length (words)
Claims
Engine
Reader A alone
Reader B alone
Engine minus A
Engine minus B
Under 200
8,524
80.99
80.20
80.49
+ 0.8
+ 0.5
200 to 399
11,185
80.17
78.45
77.62
+ 1.7
+ 2.5
400 to 799
6,056
75.75
74.17
73.91
+ 1.6
+ 1.8
800 and over
3,555
77.44
74.71
74.30
+ 2.7
+ 3.1
Table 5: Balanced accuracy by source-document length, full test split (%): Each cell is the mean over the datasets that have both labels in that band (8 or 9 of the 11).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Supported claims kept
Unsupported claims caught
Dataset
Supported
Unsupported
Engine
A alone
B alone
Engine
A alone
B alone
AggreFact-CNN
501
57
95.0
97.4
98.2
57.9
36.8
33.3
AggreFact-XSum
285
273
64.9
76.5
84.9
85.0
75.1
62.3
ClaimVerify
789
299
85.8
91.9
90.6
69.2
55.2
58.2
ExpertQA
2,971
731
60.5
70.1
63.2
59.5
51.3
57.3
FactCheck-GPT
376
1,190
72.9
78.7
71.3
88.9
83.9
89.2
Appendix
Table 6: Supported claims kept (true-positive rate) and unsupported claims caught (true-negative rate), full test split (%). The last row gives total counts and mean rates.
Dataset
Engine
Reader C alone
Engine minus C
Average of 11
80.00
79.63
+ 0.37
AggreFact-CNN
77.09
74.99
+ 2.10
AggreFact-XSum
74.26
76.47
− 2.21
ClaimVerify
77.21
76.10
+ 1.11
ExpertQA
63.24
60.29
+ 2.95
FactCheck-GPT
79.78
78.31
+ 1.47
Appendix
Table 7: The engine and Reader C alone on the 2,992-claim sample, balanced accuracy (%). 272 claims per dataset, so single-dataset gaps of 2 to 3 points are within noise. Readers A and B on the full split are in Table 3 .
Department of Interactive Media, Hong Kong Baptist University · School of Computing and Data Science, The University of Hong Kong · AI and Social Good Lab, AI Media Centre, Hong Kong Baptist University