Rethinking the Evaluation of LLM Fact-Checkers

Ayana Niwa; Miyu Yamada; Kouta Nakayama; Masahiro Kaneko; Hisami Suzuki; Satoshi Sekine

cs.CLOct 21, 2026

Rethinking the Evaluation of LLM Fact-Checkers

Authors: Ayana Niwa, Miyu Yamada, Kouta Nakayama, Masahiro Kaneko, Hisami Suzuki, Satoshi Sekine

Organizations: MBZUAI · Science Tokyo · NII-LLMC

Abstract

Large language models (LLMs) are widely used to answer factual questions, but their answers can contain errors and must be fact-checked claim by claim. Yet the resulting score depends on three design decisions: which questions the claims answer, under what conditions each claim is judged, and which sources count as evidence. No existing benchmark specifies all three. We introduce JSFactCheckBench, a Japanese benchmark that specifies all three, with 1,913 claims from LLM answers to questions about misleading information that circulated on Japanese social media, each annotated for factuality at a reference time and for importance to the question, using evidence from preselected Web sources. Across six LLMs, all verify claims directly answering the question less accurately than supplementary claims, a gap that aggregate scores obscure. Scores also drop when models must locate the relevant content in longer or multiple sources and, for all four open models, when the surrounding question and response are added. These weaknesses are visible only because the benchmark records the conditions behind each label. A benchmark that leaves them implicit cannot reveal them.

Figures & tables

		(1) Claim collection		(2) Judgment conditions		(3) Evidence sources
Benchmark	Lang.	LLM responses	Real-world origin	Reference time	Human importance	Open web	Source control
FEVER ( 2018 )	En	–	–	–	–	–	✓
TruthfulQA ( 2021 )	En	$\triangle$	–	–	–	✓	–
CHEF ( 2022 )	Zh	–	✓	$\triangle$	–	✓	$\triangle$
FactDrill ( 2022 )	Multi	–	✓	$\triangle$	–	✓	–
FActScore ( 2023 )	En	✓	–	–	–	–	✓

Table 1: Comparison of construction. ✓: satisfied;

\triangle

: partially satisfied; – : not satisfied. LLM responses : factuality or hallucination annotations of LLM-generated responses at claim, segment, or span level (

\triangle

: answer-level labels or perturbation conditions only). Real-world origin : provenance of the source questions or claims (

\triangle

: mixed provenance). VeriTaS and JAD-AFC include truth-altering claim variants. Reference time : a recorded reference time explicitly anchoring gold factuality judgments (

\triangle

: temporal metadata or constraints only). Human importance : manually annotated claim importance that is incorporated into the benchmark’s evaluation (

\triangle

: annotated but not used in evaluation, or automatically assigned). Open web : Web-derived evidence or reference sources not restricted to a fixed reference corpus. Source control : predefined restrictions on eligible evidence sources (✓: fixed reference sources or an allowlist;

\triangle

: blocklist-based restrictions). AVeriTeC excludes only the originating fact-checking article and merely flags unreliable sites.

Figure 1: Overview of the dataset construction pipeline.

Question 1	Is it free to call an ambulance in Japan?
Response	Calling an ambulance in Japan normally incurs a fee. However, $\cdots$ to request an ambulance, call 119. $\cdots$
Claim	Calling an ambulance in Japan normally incurs a fee. ✗ Refuted Main Source ID: d61417498a , Time: 2026-02-15 Evidence span: In Japan, ambulance users currently bear no charge and can use the service free of cost. $\cdots$
Question 2	Will the ¥1.1 million gift tax exemption be abolished starting in 2024?
Response	No decision to abolish the annual gift tax exclusion of ¥1.1 million in 2024 has been announced. The tax-free threshold for gift tax is set under Japanese tax legislation. $\cdots$
Claim	No decision to abolish the ¥1.1 million gift tax exemption in 2024 has been announced. ✓ Supported Main Source ID: b3655be520 , Time: 2026-02-15 Evidence span: $\cdots$ it was decided that the annual gift tax system would remain in place. $\cdots$

Table 2: Example instances from JSFactCheckBench. The Japanese text is translated into English.

	Main	Sub	Irrel.	All
Supported	250	625	54	929 (48.6%)
Refuted	70	172	8	250 (13.1%)
NotEnoughInfo	140	345	74	559 (29.2%)
Unevaluable	23	108	44	175 0 (9.1%)
All	483	1,250	180	1,913
	(25.2%)	(65.3%)	(9.4%)

Table 3: Label distribution of factuality and importance. Percentages are relative to all 1,913 claims; the 20 claims labeled Unclear are excluded from the dataset.

Topic domain	N	Topic domain	N
Medicine and health	24	Science and technology	4
Nuclear energy	12	Justice and law	4
Economy and industry	12	Society and gender	4
Culture and history	8	Politics	3
Government services	7	International affairs	2
Media and broadcasting	5

Table 4: Distribution of question domains.

Geographic scope	N	%
Japan	40	47.1
Universal/non-geographic	24	28.2
Foreign country	14	16.5
International/global	7	8.2

Table 5: Distribution of geographic scope.

Input given to the model				Macro-F1
Claim	Time	Q & R	Evidence	Llama-3.1-Sw.	Qwen3-Sw.	LLM-jp-4	Mistral-Nemo-Ja	GPT-5.1	Sonnet 5
(a) Closed book: no evidence supplied
✓	✗	✗	None	0.444	0.463	0.485	0.490	0.549	0.600
✓	✓	✗	None	0.418	0.437	0.474	0.489	0.557	0.593
✓	✓	✓	None	0.265	0.327	0.307	0.256	0.544	0.586
(b) Source given: the cited page is supplied

Table 6: Factuality classification results. See Section 3.2 for the input settings. Best score per row in bold.

Model	Acc (all)	Accuracy by importance
Model	Acc (all)	Main	Sub	Irrel.
Llama-3.1-Sw.	0.648	0.563	0.687	0.615
Qwen3-Sw.	0.741	0.697	0.754	0.785
LLM-jp-4	0.741	0.710	0.746	0.807
Mistral-Nemo-Ja	0.699	0.648	0.725	0.644
GPT-5.1	0.741	0.707	0.747	0.807

Table 7: Accuracy by claim importance (Claim

+

Evidence Full). Acc (all) is accuracy over all claims; the importance columns are accuracy within each group. We report accuracy rather than the macro-F1 of the main results because the Irrelevant group contains only 8 Refuted claims, too few for a macro-averaged score.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Model
a	openai/gpt-35-turbo-1106
b	llm-jp/llm-jp-13b-instruct-full-ac_001_16x-dolly-ichikara_004_001_single-oasst-oasst2-v2.0
c	cyberagent/calm3-22b-chat
d	anthropic/anthropic.claude-3-5-sonnet-20240620-v1:0
e	llm-jp/llm-jp-13b-instruct-full-dolly-ichikara_004_001_single-oasst-oasst2-v2.0
f	tokyotech-llm/Llama-3-Swallow-70B-Instruct-v0.1

Appendix

Table 8: Models used to generate the answers.

Figure 2: Distribution of models used for response generation and the number of responses in the dataset.

Panel of Figure 3 — Figure 3: Prompt for search-query generation. The Japanese original is translated into English; slashes mark the spaces separating keywords.

Label	Definition	Decision criterion and examples (Q: What is the capital of Japan? )
Main	A claim that directly answers the question, or that states the core or a part of the answer. It need not be a complete or correct answer, only a direct response to the question.	Shown on its own, the claim stands as an answer to the question, whether full or partial. It is Tokyo. / I do not know.
Sub	A claim that supplies information related to the subject of the question or to the answer, but does not itself contain the specific value or name being asked for.	On its own it does not answer the question; it mentions the topic of the question but not the value asked for. It became the capital in 1869.
Irrelevant	A claim with no direct relation to the question or to the content of the answer. It contributes neither to the correctness nor to the understanding of the answer.	The claim is unrelated to the topic or purpose of the question, and fits neither Main nor Sub. Osaka is famous for its food culture.

Appendix

Table 9: Criteria for the importance axis. Examples are translated from the Japanese guidelines.

Subtype	Definition	Example
Relative evaluation only	Uses relative terms such as many , few , sufficient , or excessive without stating the standard or the object of comparison, so that the assessment depends on the reader’s values.	The Japanese government allocates a large budget to scholarship programs for international students. With a figure supplied ( a budget of JPY 22.2 billion ), the claim is verifiable and is judged on the factuality axis.
Hypothetical conditional	States a prospect or a generality conditional on an event, without asserting that the event occurred.	If such an incident really did occur, the truth will come to light.
Metalinguistic statement	States the speaker’s own position, capability, constraints, or willingness to respond.	I cannot provide detailed information about specific incidents or individuals.
Value or normative statement	Asserts what should be done or what is desirable, important, or recommended.	The government should be more transparent.

Appendix

Table 10: Subtypes of the Unevaluable label. Examples are translated from the Japanese guidelines.

	JPY	USD
Total cost	2,302,300	14,632
Cost per annotation	393	2.50
Cost per claim	1,180	7.50
Annotators	3
Claims	1,952
Individual annotations	5,856

Appendix

Table 11: Cost and effort of annotation. Cost per claim covers the three independent annotations of that claim. USD amounts use the rate of JPY 157 per USD.

Domain category	Candidates	Selected
Core	7,931	539
go.jp	2,408	205
co.jp	2,758	172
or.jp	1,160	75
lg.jp	949	54
ac.jp	656	33

Appendix

Table 12: Domain categories of candidate and selected evidence set. Numbers indicate the counts of unique links.

Figure 4: Number of claims per response.

Question stance	N	Temporal stability	N
Neutral	71	Static	46
Reported	9	Long-term dynamic	18
Leading	5	Short-term dynamic	21

Appendix

Table 13: Distribution of question stance and temporal stability.

Code	Name	Definition and inclusion/exclusion criteria
D1	Medicine and health	Diseases, treatments, pharmaceuticals, vaccines, nutrition and food components, and physiology. Excludes the management or corporate history of pharmaceutical companies (D8) and the operation of health-care systems (D4).
D2	Science and technology	Natural phenomena such as earthquakes, weather, and astronomy, and scientific, technical, or infrastructure topics such as chemistry, biology, communications, and energy facilities. Includes events, policies, and facilities in these fields, not only scientific propositions. Nuclear energy and radiation take D3 priority; questions centered on health effects take D1.
D3	Nuclear energy	Nuclear power plants, radioactive materials, treated water, and nuclear fuel. If human health effects are central, assign D1; if materials, facilities, or regulation are central, assign D3.
D4	Government services	Taxation, social security, benefits, disaster administration, government organizations, and public expenditure. Excludes legal interpretation itself (D5) and party politics (D6).
D5	Justice and law	Courts, police, and other justice or law-enforcement institutions.
D6	Politics	Political parties, politicians, elections, and policy disputes. Excludes the content of public systems themselves (D4).

Appendix

Table 14: Annotation scheme for Axis 1: Question domains.

Code	Name	Definition and inclusion/exclusion criteria
G1	Japan	Institutions, events, or people in Japan are the referent.
G2	Foreign country	Domestic affairs within a specific foreign country are the referent.
G3	International or global	The claim has geographic referents but is not restricted to one country, including international organizations, cross-border events, global distributions, and global companies or platforms.
G0	Universal or non-geographic	The proposition has no geographic referent, as in natural laws, general biomedical facts, or normative propositions. Universal validity alone does not imply G3; the distinction depends on whether the proposition refers to a particular place or institution.

Appendix

Table 15: Annotation scheme for Axis 2: Geographic scope.

Code	Name	Definition and inclusion/exclusion criteria
Q1	Neutral	Presents a proposition and asks whether it is true without marking the speaker’s prior commitment, as in ordinary yes/no questions.
Q2	Reported	Marks the proposition as information attributed to another source, using expressions such as “apparently,” “is it true that,” “I heard that,” or “there is a story that.”
Q3	Leading	Presupposes the proposition and seeks agreement, using forms equivalent to “isn’t it?,” “right?,” or “wouldn’t you say?”

Appendix

Table 16: Annotation scheme for Axis 3: Question stance.

Code	Name	Definition and inclusion/exclusion criteria
T1	Static	The conclusion does not change in any substantial way, as with past events, established findings, definitions, and norms.
T2	Long-term dynamic	The conclusion may change over a period of more than a year to a decade or so, as with laws and institutions, official standards, and scientific findings that are still accumulating.
T3	Short-term dynamic	The conclusion may change within days to a year, as with current conditions, ongoing cases, private-sector specifications, and future plans.

Appendix

Table 17: Annotation scheme for Axis 4: Temporal stability.

Panel of Figure 5 — Figure 5: Prompt template for the Claim only setting (no evidence). Angle brackets mark the slots filled per claim.

Panel of Figure 6 — Figure 6: Prompt template for the Claim + Time + Q&R + Reference setting. Angle brackets mark the slots filled per claim.

Panel of Figure 7 — Figure 7: Prompt template for the Claim + Time + Q&R + Pages setting. Angle brackets mark the slots filled per claim.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation
Sep 14, 2026Sarra Gharsallah, Adele Robaldo, Mariia Tokareva +5LLM Evaluation LLM-as-a-Judge
Data Science Department, EURECOM, Campus SophiaTech, Biot, France
No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection
Oct 4, 2026Kuan-Hua Wu Lu, Yohanes Andre SetiawanAutomated Fact-Checking Data Contamination in Language Models
Georgia Institute of Technology Atlanta, Georgia, USA
Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
May 28, 2026Yuxuan Ye, Raul Santos-Rodriguez, Edwin SimpsonLLM Grounding Claim Verification
Intelligent Systems Laboratory University of Bristol United Kingdom

Sep 14, 2026J/K move · Enter open · S save

Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation

Sarra Gharsallah, Adele Robaldo, Mariia Tokareva, Giovanni Gatti Pinheiro+4

Data Science Department, EURECOM, Campus SophiaTech, Biot, France

LLM Evaluation LLM-as-a-Judge Factual Consistency Evaluation

Open paper AskPDF arXiv

Evaluating the factual correctness of large language models (LLMs) is vital for many applications. But are our evaluation tools themselves trustworthy? Despite the rise of factuality-based metrics, their sensitivity and reliability remain underexplored. This paper introduces a meta-evaluation framework that systematically tests these metrics using controlled corruptions of gold standard answers. Our method generates ranked outputs with known degrees of degradation to probe how metrics capture nuanced changes in truthfulness. Our experiments reveal that pipeline-based methods, such as the RAGAS's factual correctness metric, better track degradation than LLM-as-judge approaches. We also propose a new variant of the factual correctness metric that provides a competitive and cost-efficient.

Abstract

LLM Evaluation

Explore similar work

Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation

No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection

Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies