Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs). To enhance trust, natural language claims from diverse sources, including human-written text, web content, and model outputs, are commonly checked for factuality by retrieving external knowledge and using an LLM to verify the faithfulness of claims to the retrieved evidence. As a result, such methods are constrained by retrieval errors and external data availability, while leaving the models intrinsic fact-verification capabilities largely unused. We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source. To study this setting, we introduce a comprehensive evaluation framework focused on generalization, testing robustness to (i) long-tail knowledge, (ii) variation in claim sources, (iii) multilinguality, and (iv) long-form generation. Across 9 datasets, 18 methods and 3 models, our experiments indicate that logit-based approaches often underperform compared to those that leverage internal model representations. Building on this finding, we introduce INTRA, a method that exploits interactions between internal representations and achieves state-of-the-art performance with strong generalization. More broadly, our work establishes fact-checking without retrieval as a promising research direction that can complement retrieval-based frameworks, improve scalability, and enable the use of such systems as reward signals during training or as components integrated into the generation process.
Figures & tables
Figure 1: The task setting of fact-checking without retrieval. Claims from any source (human or LLMs) can be verified without having access to a knowledge base.
Method
ROC-AUC
PR-AUC
Llama 3.1
Ministral
Phi-4
Avg
Llama 3.1
Ministral
Phi-4
Avg
Retrieval-Based Methods
Verb+RAG
77.7
78.2
76.5
77.5
70.1
69.6
67.1
68.9
Retrieval-Free Unsupervised Methods
SP
70.9
69.0
66.4
68.8
66.9
64.9
62.8
64.9
PPL
68.6
67.3
66.6
67.5
65.6
64.2
63.1
64.3
Table 1: Average performance of claim verification methods for considered models in the proposed retrieval-free setting, measured by PR-AUC ↑ and ROC-AUC ↑ across nine datasets. Bold values indicate the best-performing retrieval-free method for each model, the second best is underlined . Avg column reports the average score across all models, summarizing overall robustness.
Figure 2: ROC-AUC ↑ performance of individual layers in the INTRA method.
Figure 3: ROC–AUC splited by popularity and language groups.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Claim
Filtered
Verification Explanation
Moby was born Richard Melville Hall.
No
This fact explicitly mentions the stage name “Moby” and provides a unique personal detail, his birth name.
Moby is an American singer-songwriter.
No
This fact explicitly mentions the entity “Moby” and describes his profession, directly tying the fact to him.
He rose to international fame.
Yes
The statement is too vague and lacks a specific entity or individual to whom it refers.
The album Play came out in 1999.
Yes
This fact is vague as it doesn’t specify the artist of the album “Play,” which could refer to multiple albums with the same title.
Appendix
Table 2: Examples of filtered data. Claims generated by the model in response to the query: “Tell me facts about Moby, American musician and songwriter.”
Dataset
# Before Filter
# After Filter
% Filtered
AC-PopQA
6,974
5,494
21
AC-WH
46,605
36,427
21
AVeriTeC
424
349
17
UHead
2,057
921
55
Cities
1,496
1,496
0
Companies
1,200
1,200
0
Appendix
Table 3: Final validation datasets after filtering claims. Cities, Common Claims, CounterFacts and Companies were not filtered out, as the claims are fully consistent with the original task.
Method
Runtime per instance (ms)
Retrieval-Based Unsupervised Methods
Verb+RAG
950±200
Retrieval-Free Unsupervised Methods
SP
30±1
PPL
30±1
MTE
84±40
Appendix
Table 4: Runtime per instance (in milliseconds) for different hallucination detection methods. Values are reported as mean ± standard deviation, measured per input instance.
Layers
PopQA
AVeriTeC
UHEAD
Cities
CC
CMP
CF
WH
X-Fact
Avg
All
89.5
66.4
63.7
98.7
74.2
89.1
73.7
73.3
58.1
76.3
0-8
84.9
59.5
53.0
83.9
62.1
65.4
59.7
64.1
55.2
65.3
24-32
86.7
63.8
61.0
73.2
71.5
70.5
58.4
70.0
57.0
68.0
2-30
89.6
66.2
63.6
98.8
73.8
89.7
74.7
73.1
58.3
76.4
8-24
89.6
66.8
65.3
99.3
74.4
92.0
77.4
73.5
57.6
77.3
11-22
89.3
66.7
66.4
99.0
75.0
93.0
79.1
73.5
57.1
77.7
Appendix
Table 5: Impact of different layer subsets on the performance by ROC-AUC ↑ of the proposed INTRA method. The best method is in bold , the second best is underlined .
Figure 4: ROC–AUC on WH by claim position for short and medium-length claims. The dashed line shows the fraction of hallucinated claims at each position; the numbers above it indicate the number of claims at that position.
Method
long-tail
human-made
cross-model
rule-generated
long-form
multilingual
Avg
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Retrieval-Based Methods
Verb + RAG
72.8
74.4
72.7
98.4
73.3
89.0
74.3
71.0
73.3
77.7
Retrieval-Free Unsupervised Methods
SP
78.2
67.4
69.9
95.8
65.0
74.9
70.1
70.0
53.5
71.6
PPL
73.3
57.4
62.4
94.2
72.5
75.0
65.9
59.3
57.6
68.6
Appendix
Table 6: Performance of claim verification methods for Llama-3.1-8B in the proposed evidence-free setting, measured by ROC-AUC ↑ across nine datasets. Bold values indicate the best-performing method for each dataset, the second best is underlined . Avg column reports the average score across all datasets, summarizing overall robustness.
long-tail
human-made
cross-modal
rule-generated
long-form
multilingual
Method
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Avg
Retrieval-Based Methods
Verb + RAG
86.0
82.5
45.5
97.1
67.0
83.6
66.2
36.0
67.0
70.1
Retrieval-Free Unsupervised Methods
SP
89.4
79.6
40.4
94.4
64.1
71.5
66.5
32.0
64.4
66.9
PPL
88.3
74.3
31.5
94.4
71.7
77.3
62.4
21.3
69.4
65.6
Appendix
Table 7: Performance of claim verification methods for Llama-3.1-8B in the proposed evidence-free setting, measured by PR-AUC ↑ (multiplied by 100) across nine datasets. Bold values indicate the best-performing method for each dataset, the second best is underlined . Avg column reports the average score across all datasets, summarizing overall robustness.
Method
long-tail
human-made
cross-model
rule-generated
long-form
multilingual
Avg
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Retrieval-Based Methods
Verb + RAG
76.0
77.0
75.0
98.0
73.0
87.0
75.0
70.0
73.0
78.2
Retrieval-Free Unsupervised Methods
SP
75.3
65.0
68.8
95.9
63.9
67.6
64.7
72.2
47.8
69.0
PPL
72.7
55.3
56.7
95.1
71.4
70.3
62.3
61.0
61.3
67.3
Appendix
Table 8: Performance of claim verification methods for Ministral-8B in the proposed evidence-free setting, measured by ROC-AUC ↑ across nine datasets. Bold values indicate the best result per dataset, the second best is underlined , and Avg the mean score across all datasets.
Method
long-tail
human-made
cross-model
rule-generated
long-form
multilingual
Avg
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Retrieval-Based Methods
Verb + RAG
87.0
85.0
47.0
96.0
67.0
81.0
67.0
31.0
67.0
69.6
Retrieval-Free Unsupervised Methods
SP
87.4
78.1
39.0
93.7
63.4
63.6
60.9
33.8
64.6
64.9
PPL
87.7
71.6
28.2
95.5
70.3
71.1
58.9
22.3
72.3
64.2
Appendix
Table 9: Performance of claim verification methods for Ministral-8B in the proposed evidence-free setting, measured by PR-AUC ↑ (multiplied by 100) across nine datasets. Bold values indicate the best result per dataset, the second best is underlined , and Avg the mean score across all datasets.
Method
long-tail
human-made
cross-model
rule-generated
long-form
multilingual
Avg
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Retrieval-Based Methods
Verb + RAG
78.0
72.0
71.0
95.0
73.0
87.0
74.0
66.0
73.0
76.5
Retrieval-Free Unsupervised Methods
SP
71.5
64.0
66.2
92.2
63.8
65.0
61.6
67.4
45.8
66.4
Perplexity
72.2
55.0
60.6
91.6
70.7
71.1
61.9
61.0
55.0
66.6
Appendix
Table 10: Performance of claim verification methods for Phi-4-mini in the proposed evidence-free setting, measured by ROC-AUC ↑ across nine datasets. Bold values indicate the best result per dataset, the second best is underlined , and Avg the mean score across all datasets.
Method
long-tail
human-made
cross-model
rule-generated
long-form
multilingual
Avg
PopQA
AVeriTeC
UHead
Cities
CC
CMP
CF
WH
X-Fact
Retrieval-Based Methods
Verb + RAG
88.0
81.0
41.0
91.0
66.0
82.0
66.0
24.0
66.0
67.1
Retrieval-Free Unsupervised Methods
SP
86.0
78.2
38.6
87.8
62.4
61.4
59.0
28.6
63.0
62.8
Perplexity
87.7
73.4
31.2
90.9
68.5
69.2
58.7
21.7
66.2
63.1
Appendix
Table 11: Performance of claim verification methods for Phi-4-mini in the proposed evidence-free setting, measured by PR-AUC ↑ (multiplied by 100) across nine datasets. Bold values indicate the best result per dataset, the second best is underlined , and Avg the mean score across all datasets.