LLMs are remarkable artifacts that have revolutionized a range of knowledge-intensive tasks. A significant contributor is their factual knowledge, which, to date, remains poorly understood, and is usually analyzed from biased samples. In this paper, we provide a framework and the results of a multi-dimensional analysis of GPTKB v1.5 (Hu et al., 2025a), a recursively elicited Knowledge Base (KB) of 100 million facts (or beliefs) of a frontier LLM, namely, GPT-4.1. Given the scale of the elicited facts, we provide a multi-dimensional approach to qualitatively and quantitatively analyze these facts as opposed to the mainstream fact completion benchmarks, which are prone to availability bias. We find that the models' factual knowledge differs quite significantly from established knowledge bases, and that its accuracy is significantly lower than indicated by previous benchmarks. We also find that inconsistency, ambiguity and hallucinations are major issues, shedding light on future research opportunities in neuro-symbolic AI concerning extraction, consolidation and verification of factual LLM knowledge.
Figures & tables
Direct LLM querying
LLM-mined knowledge analysis
✗ Lacks aggregate access to internalized knowledge
✓ Accesses millions of extracted records
✗ Gives uninformative and wrong answers
✓ Provides quantitative insights
ChatGPT interface
Mined knowledge (GPTKB v1.5)
How many individual persons do you know?
I don’t “know” any individual persons [..]. [..] millions to billions, depending on [..]. → uninformative
1,200,692
Do you know more males or more females?
Table 1: Direct LLM querying, vs. large-mined knowledge analysis. ChatGPT conversation archived at https://chatgpt.com/share/68b6cd03-85e4-800d-81c8-afebc74ccc9d .
Resource
#entities
#assertions
Wikimedia-related
Wikidata Vrandecic and Krötzsch (2014)
113M
1.62B
Yago 4.5 Suchanek et al. (2024)
50M
140M
DBpedia Auer et al. (2007)
3.8M
75M
Text-extracted
NELL Carlson et al. (2010)
1.9M
12M
Table 2: Size comparison of knowledge resources and LLM factuality benchmarks.
Class
Frequency
Relation
Frequency
person
1,061,852
instanceOf
6,588,295
human
138,840
locatedIn
2,121,234
film
123,021
occupation
1,962,461
company
122,847
country
1,927,856
book
112,188
notableWork
1,500,584
song
103,824
genre
1,268,067
Table 3: Most frequent GPTKB classes and relations.
Domain
Fraction
Domain
Fraction
Science
0.2%
Business
7.0%
Technology
4.0%
Health
0.0%
Engineering
1.3%
Environment
1.9%
Mathematics
0.0%
event
0.8%
Arts
23.8%
unknown
0.0%
Humanities
60.0%
food
1.1%
Table 4: Entity proportion of first-order subbranches of the taxonomy, based on a random sample of 1000 entities.
Figure 1: Language distribution in GPTKB.
Property
Count
Property
Count
logo
37,338
mapType
2,432
coverArt
33,717
photo
2,253
signature
32,443
portrait
1,337
coatOfArms
5,833
range
581
flag
3,424
hasPoster
517
Table 5: Top-10 predicates which come with objects being image file names.
Figure 2: Female-to-male ratio (y-axis) in 50 most frequent nationalities (x-axis).
Figure 3: Female-to-male ratio (y-axis) in 50 most frequent professions (x-axis).
No female gender
No male gender
Profession
Frequency
Profession
Frequency
sports shooter
127
gptkb:actress
11248
cowboy
74
gptkb:businesswoman
1307
gptkb:drag_queen
70
noblewoman
581
crime lord
68
gptkb:suffragist
469
gptkb:hurler
60
gptkb:feminist
323
Table 7: Top 10 gender-specific professions.
Nationality
GPTKB
Global *
Global top-10
American a
31.5
4.3
India (18.2)
British b
12.4
0.8
China (17.6)
French
3.9
0.8
United States (4.3)
German
3.6
1.0
Indonesia (3.5)
Indian
3.6
18.2
Pakistan (3.9)
Canadian
3.3
0.5
Nigeria (2.9)
Table 8: Nationality distribution in percentage in GPTKB and the real-world. Nationalities in GPTKB that feature in the global top-10 are highlighted. *Real world data obtained from https://worldpopulationreview.com/countries as of 2025.