LLMpedia: A Transparent Framework to Materialize an LLM's Encyclopedic Knowledge at Scale

Muhammed Saeed; Simon Razniewski

SG	Prompt	Reas.	Prec.%	True%	False%	Unv.%
✗	base	low	90.7	84.6	8.2	7.3
✗	base	min	$94.7^{\dagger}$	89.5	4.5	6.0
✗	calib	low	92.1	85.4	7.0	7.7
✗	calib	min	94.5	89.7	4.5	5.8
✓	base	low	93.9	89.3	5.5	5.2
✓	base	min	96.2	93.0	3.6	3.4

Appendix

Table 8: Full

2^{3}

ablation on gpt-5-mini (500 articles, seed 42). SG = self-grounding.

\dagger

: selected operating point.

Model	Execution	Role
gpt-5-mini	OpenAI Batch API	generation, NER, sim
Llama-3.3-70B	4 $\times$ A100 (on-prem)	generation, NER, sim
DeepSeek-V3-0324	16 $\times$ H100 (on-prem)	generation, NER, sim
text-embedding-3-small	OpenAI Batch API	dedup embeddings
gpt-4.1-nano	OpenAI API	evaluation judge

Appendix

Table 9: Models and execution modes across all experimental conditions.

Metric	LLMpedia	Grokipedia
Factuality (Wiki / Web)
Precision (%)	98.6 $\pm$ 8.3 / 98.8 $\pm$ 6.7	97.3 $\pm$ 10.3 / 97.0 $\pm$ 9.6
True rate (%)	86.0 $\pm$ 28.9 / 76.3 $\pm$ 33.4	79.1 $\pm$ 31.8 / 73.4 $\pm$ 31.4
False rate (%)	0.8 $\pm$ 3.7 / 0.7 $\pm$ 4.3	1.8 $\pm$ 6.4 / 1.7 $\pm$ 4.7
Unverifiable (%)	13.3 $\pm$ 28.6 / 23.0 $\pm$ 33.2	19.1 $\pm$ 31.3 / 24.9 $\pm$ 31.2
Similarity to Wikipedia

Appendix

Table 10: Full retrieval-trap results on 1,000 shared titles. Mean

\pm

std macro-averaged per article. All verdicts under the evaluation prompt described in Appendix D .

Class	Tier 1 (Wiki)	Tier 2 (Web)
Supported	87.9%	93.3%
Refuted	81.8%	70.0%
Insufficient	93.9%	90.0%
Overall	87.9%	84.4%

Appendix

Table 11: Human–judge per-class agreement. Agreement on the contribution-bearing classes ( supported drives true rate; insufficient drives unverifiable rate) ranges 87.9–93.9% across both tiers, with a per-class floor of 87.9%.

Category	Blocked root domains
Circular / Wikimedia	wikipedia.org , en.wikipedia.org , en.m.wikipedia.org , wikidata.org , www.wikidata.org , wikimedia.org
Search engines	google.com , bing.com , duckduckgo.com , yahoo.com
Social / video	facebook.com , instagram.com , twitter.com , x.com , tiktok.com , reddit.com , quora.com , youtube.com , pinterest.com
E-commerce	amazon.com , ebay.com
AI answer engines	perplexity.ai

Appendix

Table 12: Root-domain exclusions. Exclusion uses root-domain matching, so subdomains of blocked roots are also excluded.

Category	Domain	Score	Domain	Score	Domain	Score
Encyclopedias	britannica.com	100	worldhistory.org	90	encyclopedia.com	85
	scholarpedia.org	88	plato.stanford.edu	95	iep.utm.edu	88
	newworldencyclopedia.org	75
Government	loc.gov	97	archives.gov	97	congress.gov	95
	usa.gov	93	cia.gov	90	state.gov	90
	whitehouse.gov	90	nasa.gov	95	nih.gov	94

Appendix

Table 13: Explicitly scored domains (Part I). ∗ scholar.google.com is scored but excluded at runtime because google.com is blocked by root-domain matching.

Category	Domain	Score	Domain	Score	Domain	Score
Museums / Libraries	smithsonianmag.com	87	nationalgeographic.com	86	metmuseum.org	90
	moma.org	87	nga.gov	88	bl.uk	90
	bnf.fr	88	dpla.org	83	europeana.eu	83
History / Biography	history.com	80	biography.com	75	oxforddnb.com	90
	anb.org	88	historytoday.com	78	historyextra.com	77
Science / Technology	scientificamerican.com	86	newscientist.com	83	livescience.com	75

Appendix

Table 14: Explicitly scored domains (Part II). Together with Table 13 , this yields 133 explicitly scored domains.

	GPT-5-mini	DeepSeek	Llama
Generated articles	1,009K	120K	120K
Raw [[links]]	70.2M	6.5M	9.8M
Pre-NER candidates	12.48M	2.00M	4.33M
After canonical dedup	2.29M	859K	1.25M
After NER	1.65M	444K	684K
After similarity	1.12M	396K	499K

Appendix

Table 15: Full funnel with per-stage survival rates across all three models. Raw

\to

pre-NER reflects article-level candidate construction and surface-form consolidation before any LLM stage runs. Canonical deduplication is dominated by previously committed entities: as the committed index grows, canonical survival falls and canonical deduplication loss increases.

Figure 6: Funnel counts per stage, grouped by model. Each cluster on the x-axis is a pipeline stage; bars within a cluster are the three models. The decay from Generated to Inserted is visible as a near-monotone right-to-left shrinking pattern for every model. GPT-5-mini operates at roughly an order of magnitude larger scale at every stage, but the shape of the funnel is remarkably similar across models.

Figure 7: Per-stage survival rates, grouped by model. Three patterns stand out: (i) GPT-5-mini has the lowest canonical survival (18.4%) because its much larger committed index makes more candidates redundant; (ii) Llama’s NER survival is the lowest at 54.8% and is dragged down by parse failures rather than legitimate “not a named entity” rejections (Appendix H.4 ); (iii) DeepSeek’s similarity survival is the highest at 87.5%, indicating its candidate set carries less semantic redundancy at the post-NER stage.

Figure 8: Pre-NER candidate count by BFS hop, one line per model. The characteristic peak-and-decay shape reflects two regimes: rapid growth as BFS reaches the wide middle hops (peaking at hop 3 for the open-weight models and hop 4 for GPT-5-mini), followed by collapse once most surfaced entities are already in the committed index and most parents are pruned at canonical dedup.

Figure 9: Inserted children per hop, one line per model. This is the shape of the resulting corpus: the bulk of GPT-5-mini’s

{\sim}

1M articles are at hops 4–5, whereas the open-weight runs terminate at hops 4–5 because 120K max-subject budget was reached. The shared asymmetry - light hops 0–2, dense hops 3–5, depleted hops beyond - is characteristic of BFS over a power-law-degree encyclopedic graph.

Figure 10: Overall survival from raw candidate to inserted queue subject, by BFS hop and model. Survival falls sharply with depth: for GPT-5-mini the rate drops from 75.3% at hop 0 to 6.6% at hop 5, and the open-weight models trace similar trajectories. The cause is not the LLM stages (NER and similarity survival are relatively flat with depth) but canonical dedup, whose survival collapses as the committed index saturates the local neighborhood of the BFS frontier.

Figure 11: GPT-5-mini five-stage funnel from 70.2M raw candidates to 1.06M queued subjects. The largest absolute loss is at pre-NER canonical deduplication (

-

82.3%), reflecting that most wikilinks in any given article are surface variants of entities already committed elsewhere in the corpus.

Figure 12: GPT-5-mini NER reject reason breakdown. Of 553K NER rejections, 99.9% (552,565) are the legitimate not_named_entity class - the filter doing its intended job. Parse failures account for just 0.1% (556 rejections), and every NER call parsed as native JSON (no fallback regex paths were triggered).

Figure 13: GPT-5-mini similarity reject reasons. Of 534K rejections, 97.6% are LLM-confirmed duplicates - the arbitration LLM agreed with the embedding threshold that the candidate was a surface variant of an already-committed entity. Within-batch duplicates (2.3%) are concurrent NER workers independently nominating the same entity in the same BFS wave (Appendix G.2 ). Operational failures (591 rejections, 0.1%) come from transient batch-output gaps and SDK exceptions; the strict-gate policy treats these as duplicates to avoid contaminating the corpus, at the cost of a small downward recall bias.

Figure 14: Per-hop survival rates for GPT-5-mini with

{\pm}1\sigma

bands across entities. Three things to read off: (i) survival is monotonically decreasing in hop; (ii) variance across entities within a hop is large (especially at deep hops, where some entities produce dozens of survivors and many produce zero - the power-law tail of wikilink fan-out); (iii) the gap between NER survival and similarity survival narrows with depth because, at the frontier, the embedding index has already absorbed most semantically nearby entities, so similarity rejections shift toward LLM-confirmed duplicates of newly committed siblings.

Figure 15: GPT-5-mini candidate fate per hop, 100% normalized stacked area. At hop 1 most candidates make it through the entire pipeline to insertion; by hop 5 the pre-NER dedup share dominates and the inserted share has shrunk to a few percent. The displacement is driven entirely by the redundancy layer, not by NER or similarity rejecting more aggressively at depth.

Figure 16: GPT-5-mini pipeline coverage by hop. Blue bars are expanded-out entities (NER ran on their outbound wikilinks); orange bars are terminal entities (article exists and is validly filtered into the corpus, but never expanded). Terminality is concentrated at hops 5–6 where the run reached its computational ceiling: 192K of the 1.06M subjects are terminal at hop 6, which is the natural BFS boundary. The track-3 factuality evaluation samples from filtered-in entities at all hops regardless of expanded status, which is why the headline coverage and true-rate numbers are reported over the full

{\sim}

1M corpus.

Figure 17: Top-30 entities by aliases absorbed in the GPT-5-mini run. These are the most “magnetic” canonical entities: each one collapsed dozens to thousands of surface-form variants ( University of Oxford alone absorbed 1,814 distinct aliases). The top of the list is dominated by government bodies, universities, and standards organizations, where the variant space (acronyms, “U.S.”/“United States” prefixes, capitalization drift) is genuinely large. Without Stage 3 disambiguation each of these would have splintered into dozens of near-duplicate articles.

Figure 18: DeepSeek five-stage funnel from 6.5M raw candidates to 396K queued subjects. The pre-NER canonical dedup stage is less severe than for GPT-5-mini (

-

57.1% vs.

-

82.3%) because the committed index is smaller; the LLM stages take a proportionally larger share of the total loss.

Figure 19: DeepSeek NER reject reasons. 99.2% of 414K rejections are not_named_entity - the filter behaving as designed. Parse failures account for 0.8% (3,177 rejections), with 99.1% of NER calls parsing as native JSON and the remainder recovered via the JSON-substring fallback.

Figure 20: DeepSeek similarity reject reasons. 93.6% LLM-confirmed duplicates and 6.4% within-batch duplicates account for all rejections; no operational failures occurred during this run.

Figure 21: DeepSeek per-hop survival rates with

{\pm}1\sigma

bands. The overall survival trajectory mirrors GPT-5-mini’s (65.5% at hop 0 down to 16.2% at hop 4) but with consistently higher similarity survival, reflecting the smaller embedding index.

Figure 22: DeepSeek candidate fate per hop, 100% normalized. The insertion fraction is meaningfully higher at every hop than for GPT-5-mini, again because of the smaller committed index. The shape of the displacement - pre-NER dedup growing with depth - is identical to the other two models.

Figure 23: DeepSeek pipeline coverage by hop. The run terminated with hop 4 only partially expanded: 10,719 of 276,033 hop 4 entities (3.9%) are expanded-out, and hop 5 is entirely terminal. and hop 5 entirely terminal. The 96.1% terminal share at hop 4 reflects the max-subject budget being reached: those 265K entities legitimately entered the corpus as filtered-in articles, but the pipeline was stopped before their outbound wikilinks could be processed.

Figure 24: Top-30 DeepSeek entities by aliases absorbed. Smaller scale than GPT-5-mini (top entity absorbs 198 aliases vs. GPT’s 1,814) but the same categorical pattern: governmental bodies, universities, and prominent historical events dominate.

Figure 25: Llama five-stage funnel from 9.8M raw candidates to 498K queued subjects. The pre-NER dedup loss (71.2%) sits between GPT-5-mini’s and DeepSeek’s, as expected for an intermediate-sized committed index.

Figure 26: Llama NER reject reasons. This breakdown is qualitatively different from the other two models. Of 563K NER rejections, only 52.0% (292,927) are legitimate not_named_entity verdicts; 48.0% (270,157) are parse/call failures - candidates the strict-gate policy treats as rejected because the NER call did not produce a parsable decision. This is the single biggest deviation in the pipeline behavior of any model. The strict-gate response is conservative (drop the candidate, do not pass through unchecked), but the underlying cause is Llama’s lower instruction-following compliance on the structured-output schema.

Figure 27: Llama similarity reject reasons. Despite the NER-stage difficulties, the similarity stage is well-behaved: 97.3% LLM-confirmed duplicates, 2.7% within-batch duplicates, and zero operational failures.

Figure 28: Llama per-hop survival rates with

{\pm}1\sigma

bands. The similarity survival band widens noticeably at deep hops: the per-entity mean is 69.0% with

\sigma{=}24.5

at hop 3 and 47.8% with

\sigma{=}37.6

at hop 4., reflecting the increased per-entity variance that the parse-failure mechanism introduces - some entities had nearly all candidates pass NER cleanly, others lost most of them to parse failures.

Figure 29: Llama candidate fate per hop, 100% normalized. The orange “NER rejected” layer is visibly thicker than in the other two models, especially at hops 3–4 where parse failures cluster.

Figure 30: Llama pipeline coverage by hop. Mirrors the DeepSeek pattern: hop 4 entities are largely terminal (99.3%) because the run reached its subject budget before they could be expanded.

Figure 31: Top-30 Llama entities by aliases absorbed. General Electric Company sits at the top with 1,542 absorbed aliases - driven by the very common “GE” acronym variant - followed by University of Oxford (1,036) and San Francisco (1,014). The long tail of media outlets and international organizations is consistent with the other two models.

Model	Total	Filt.-in	Exp.-out	Terminal
GPT-5-mini	1,063,949	1,063,930	174,100	83.6%
DeepSeek	396,091	396,091	35,441	91.1%
Llama-70B	498,226	498,226	52,557	89.5%

Appendix

Table 16: Pipeline coverage at corpus level. Terminal percentage is the share of total entities that were filtered in but not expanded out. The high terminal share reflects each run reaching its computational ceiling at the deepest hop, not pipeline failure.

Model	Hop	Total	Expanded	Terminal	Term. %
GPT-5-mini	0	1	1	0	0.0
	1	58	58	0	0.0
	2	2,344	2,344	0	0.0
	3	40,923	40,923	0	0.0
	4	321,319	89,836	231,475	72.0
	5	507,940	40,938	466,996	91.9

Appendix

Table 17: Pipeline coverage by hop. Terminality is concentrated at the deepest hop of each run, which is the natural BFS boundary. Hops

\leq 3

are essentially fully expanded in every model. 1 1 1 Total includes a small number of unprocessed entities at the deepest hops (8 at GPT-5-mini hop 4; 6 at hop 5; 5 at hop 6; 1 at DeepSeek hop 2; etc.); these are not counted as either expanded or terminal.

Model	Ents. w/ aliases	Total aliases	Aliases/ent.
GPT-5-mini	108,030	527,469	4.88
DeepSeek	14,591	35,558	2.44
Llama-70B	26,258	101,548	3.87

Appendix

Table 18: Per-model alias absorption summary. Aliases/entity is averaged only over entities that absorbed at least one alias, i.e., the mean number of duplicate surface forms each magnetic entity received.

Model	Sim-accepted	Inserted	Race loss (%)
GPT-5-mini	1,088,968	1,063,923	2.3%
DeepSeek	396,254	395,336	0.2%
Llama-70B	499,245	497,792	0.3%

Appendix

Table 19: Residual queue-insert race rate per model. All three runs stay well under 3%, with the open-weight runs near the noise floor. GPT-5-mini’s slightly higher rate reflects its larger absolute inflight load: more concurrent workers across more hops mean proportionally more canonical-key collisions per wave.

Figure 32: Subject counts with three-way intersection (7.3% overlap).

Figure 33: Pairwise subject Jaccard (0.15–0.17).

Figure 34: Entity overlap on 1,000 shared subjects (Jaccard 0.19–0.21).

Figure 35: Per-model factuality: shared (left) vs. independent (right).

Figure 36: Model-to-Wikipedia similarity. All three models below Grokipedia’s 0.455–0.489 TF-IDF Yasseri and Mohammadi (2025) .

Topic	Model	Persona	$n$	Prec	Hall	Unv	W%	Wds
A. Bab.	GPT	cons	67	99.3	0.6	15.9	67.0	765
A. Bab.	GPT	left	75	99.3	0.4	20.1	75.0	785
A. Bab.	GPT	neut	71	99.5	0.4	20.4	71.0	775
A. Bab.	DS	cons	81	98.8	1.0	20.0	81.0	834
A. Bab.	DS	left	78	97.8	1.8	20.4	78.0	908
A. Bab.	DS	neut	68	95.3	2.7	33.5	68.0	600

Appendix

Table 20: Full per-persona factuality for all 27 topic-focused conditions.

n

= Wikipedia-covered subjects. W% = Wikipedia coverage. Wds = mean word count. Hall = false rate (hallucination).

Dimension	Vocabulary (counted verbatim per 1,000 tokens)
Dutch Colonization in Southeast Asia
colonial_positive	civilizing, civilising, development, modernization, modernisation, progress, enterprise, trade, prosperity, order, governance, administration, stability, infrastructure, introduced, established, industrialization, industrialisation, reform, reforms, prospered, advancement, advancements, institution, institutions, modernized, modernised, enlightenment, cultivation, cultivated
colonial_critical	exploitation, oppression, plunder, genocide, massacre, atrocity, atrocities, extraction, looting, dispossession, enslavement, brutal, violent, racist, racism, imperialism, colonialism, occupation, resistance, uprising, rebellion, suffering, subjugation, subjugated, tyranny, tyrannical, cruelty, slavery, slaves, forced, coercion, coerced, extracted, plundered, massacred, oppressed, exploited, colonized, colonised
indigenous_agency	indigenous, native, sovereignty, self-determination, ancestral, traditional, heritage, community, resilience, revival, autonomy, javanese, sumatran, balinese, sundanese, local, locals, ancestors, custom, customs, customary, adat, kraton, sultanate
US Civil Rights Movement
civil_rights_progressive	segregation, desegregation, discrimination, jim, crow, apartheid, lynching, disenfranchisement, systemic, institutional, structural, racism, racist, white, supremacy, oppression, oppressed, marginalized, marginalised, activist, activists, activism, grassroots, organizing, organising, mobilization, mobilisation, march, marches, boycott, boycotts, sit-in, protests, protesters, freedom, liberation, emancipation, empowerment, equality, equity

Appendix

Table 21: Full framing vocabulary used by the word-list classifier on the three primary topics. Each list is matched verbatim (case-insensitive, whole-token) against the cleaned article text; multi-word phrases use substring matching over the joined token sequence. Hits are counted per 1,000 content tokens.

Topic	Category	Cons.	Left	Neut.
D. Col.	colonial_positive	3.8	2.4	2.9
	colonial_critical	2.1	3.5	2.4
	indigenous_agency	1.4	2.6	1.9
	left_progressive	1.1	2.3	1.4
	right_conservative	2.6	1.2	1.7
	military_heroic	1.8	0.9	1.2

Appendix

Table 22: Mean framing vocabulary hits per 1,000 content tokens. Each row is one framing dimension; each column is one persona. Bold marks the highest-scoring persona per row. The colonial_positive – colonial_critical gap between conservative and left-leaning on Dutch Colonization (

\Delta{=}{+}1.7

and

{+}1.4

respectively) is the largest topic-category effect in the primary data. Scientific-neutral leads the scholarly register on Babylon (

+

0.9 over both political personas), consistent with its evidence-based instruction.

Topic	Framing	Higher	Lower	$\|\Delta\|$	Note
Dutch Colon.	colonial_critical	left-leaning	conservative	5.62
Dutch Colon.	colonial_critical	left-leaning	neutral	5.24
Dutch Colon.	left_progressive	left-leaning	conservative	3.50
Dutch Colon.	left_progressive	left-leaning	neutral	3.44
Dutch Colon.	right_conservative	conservative	neutral	2.27
Dutch Colon.	indigenous_agency	left-leaning	neutral	2.17

Appendix

Table 23: Bonferroni-significant persona framing effects on the three primary topics (

|\Delta|\geq 1.0

, topic-relevant dimensions only).

|\Delta|

= mean difference in vocabulary hits per 1,000 tokens between the higher and lower persona, computed over 30 matched subject pairs per cell. † Scientific-neutral leads on both civil-rights dimensions because neutral encyclopedic writing naturally covers the full vocabulary of the domain - both systemic-racism and law-and-order registers - rather than selecting one side. This reflects comprehensive coverage, not ideological leaning.

Actor	Category	Highest persona	Spread	Hits/1k
malcolm	progressive	left-leaning	1.42	1.78
sncc	progressive	left-leaning	1.08	1.31
panthers	progressive	left-leaning	0.94	1.12
rustin	progressive	left-leaning	0.83	0.97
fannie	progressive	left-leaning	0.71	0.84
lincoln	canonical	conservative	1.61	2.04

Appendix

Table 24: Top actors by cross-persona spread (US Civil Rights topic, all three models pooled). Spread is max-min hit rate per 1,000 tokens across the three personas. Progressive actors are foregrounded by left-leaning; canonical institutional figures by conservative - corroborating the aggregate framing-category result.

Model	Topic	Pair	Outline	3-gram	Func-cos
GPT	D.Col.	C-L	0.04	0.09	0.97
GPT	D.Col.	C-N	0.06	0.11	0.97
GPT	D.Col.	L-N	0.05	0.10	0.97
GPT	A.Bab.	C-L	0.03	0.07	0.98
GPT	US CR	C-L	0.02	0.04	0.96
Llama	D.Col.	C-L	0.81	0.14	0.97

Appendix

Table 25: Structural and stylometric overlap across persona pairs (mean over 30 matched pairs per cell). Outline = Jaccard of section heading sets. 3-gram = trigram Jaccard. Func-cos = function-word cosine. C/L/N abbreviate conservative/left-leaning/scientific-neutral.

Dimension	Vocabulary (counted verbatim per 1,000 tokens)
epistemic_hedging	perhaps, may, might, possibly, probably, likely, arguably, presumably, apparently, seemingly, reportedly, allegedly, supposedly, disputed, debated, contested, uncertain, unclear, tentative, speculative, hypothesis, hypothetical, conjecture, tradition, believed, thought, considered, held, suggests, suggest, appears, appeared, plausible, plausibly
epistemic_assertive	certainly, definitely, clearly, obviously, undoubtedly, indeed, fact, factual, established, known, proven, proves, demonstrates, confirms, reveals, indisputable, undeniable, conclusive, always, never, unequivocally, truly, actually
primary_source_citation	cited, cites, records, writes, wrote, documented, attests, attested, testifies, reports, reported, describes, described, quoted, quotation, passage, chronicler, source, sources, primary, textual, documentary, testimony, manuscript, manuscripts, colophon, fragment, fragments, tablet, tablets, herodotus, tacitus, josephus, plutarch, strabo, xenophon, diodorus, livy, ctesias, thucydides, polybius, ammianus
social_history_focus	everyday, daily, ordinary, common, commoner, commoners, peasant, peasants, worker, workers, slave, slaves, servant, servants, women, woman, children, family, families, household, households, marketplace, market, craft, crafts, artisan, artisans, trade, merchant, merchants, laborer, laborers, labour, labor, subaltern, underclass, poor, poverty, diet, clothing, shelter, housing, health, disease, mortality, wage, wages, kinship, fertility, childbirth, domestic, quotidian
great_man_history	king, kings, queen, queens, emperor, emperors, empress, general, generals, commander, commanders, ruler, rulers, reign, reigned, conqueror, conquerors, dynasty, dynasties, throne, crown, coronation, battle, battles, war, wars, campaign, campaigns, victory, victories, defeat, defeats, monument, monuments, palace, palaces, triumph, triumphs, conquest, conquests, empire, empires, ascension, heir, successor
decolonial_framing	indigenous, native, local, locals, perspective, perspectives, voices, voice, silenced, erased, decolonial, decolonize, decolonise, reclaim, reclaiming, agency, autonomy, subaltern, postcolonial, eurocentric, eurocentrism, non-western, standpoint, marginalized, marginalised, centered, centred, privileging

Appendix

Table 26: Topic-agnostic framing axes used in the extended analysis. These dimensions fire on any topic regardless of domain, making them suitable for politically neutral subjects where domain-specific lists would produce near-zero counts. Each word is counted verbatim (case-insensitive, whole-token) in the cleaned article text; hyphenated entries are matched as substrings. Hit counts are normalized per 1,000 content tokens.

Topic	Model	Framing	Higher	Lower	$\|\Delta\|$	$p_{\text{Bonf}}$
One Piece	GPT	left_progressive	left	neutral	10.30	0.0001
One Piece	GPT	left_progressive	left	cons.	0 8.54	0.0003
One Piece	GPT	right_conservative	cons.	neutral	0 6.16	0.0025
One Piece	Llama	left_progressive	left	cons.	0 7.57	0.0007
One Piece	Llama	left_progressive	left	neutral	0 7.53	0.0003
Quantum	GPT	secular_framing	left	neutral	0 3.87	0.0010

Appendix

Table 27: All 6 Bonferroni-significant effects from the extended analysis (

n{=}30

matched pairs per cell).

|\Delta|

= mean difference in hits per 1,000 tokens between the higher and lower persona. “left” = left-leaning; “cons.” = conservative; “neutral” = scientific-neutral. GPT = gpt-5-mini ; Llama = Llama-3.3-70B .

Topic	Contested?	Sig. effects	Max $\|\Delta\|$
Dutch Colonization	✓	8	5.62
US Civil Rights	✓	7	5.48
Ancient Babylon	✗	6	6.03
One Piece	✗	5	10.30
Quantum Physics	✗	1	0 3.87

Appendix

Table 28: Bonferroni-significant framing effects across all five topics. The sharp drop from contested to neutral domains confirms persona-induced framing is domain-contingent. One Piece is a neutral domain that nonetheless activates ideological vocabulary through its freedom-versus-oppression narrative register, explaining its relatively high count despite being non-political.

Figure 37: Internal tool-calling traces visible in Grokipedia output.

Figure 38: Politician and basketball player conflated into one article.

Figure 39: Conflicting biographical facts from different individuals.

Figure 40: Three “Martin Schmidt” individuals blended into one article.

System	Param.	Fact. Full Art.	Serendip.	Scale
Benchmarks
MMLU Hendrycks et al. (2021)	✓	✗	✗	$10^{4}$
HLE Phan et al. (2025)	✓	✗	✗	$10^{3}$
Knowledge materialization
GPTKB Hu et al. (2025)	✓	✗	✓	$10^{6}$
Cosmopedia Allal et al. (2024)	✓	✗	(✓)	$10^{7}$

	GPT-5-mini	DeepSeek	Llama
Generated articles	1,008,947	120,139	120,100
Raw [[links]]	70,165,356	6,490,752	9,820,122
Pre-NER input	12,479,792	1,999,933	4,326,286
After canonical dedup	2,293,089	858,516	1,247,222
After NER	1,654,284	443,736	683,681
After similarity	1,120,843	396,254	499,245

	GPT-5-mini	DeepSeek	Llama-70B
Corpus (120K each)
Mean [[links]] /art	70.6	51.5	81.0
Mean sections	6.0	5.2	2.1
Mean words	885	739	845
Subject overlap (120K $\times$ 3)
Union / Intersection	280,494 / 20,417 (7.3%)

LLMpedia: A Transparent Framework to Materialize an LLM's Encyclopedic Knowledge at Scale

Abstract

Figures & tables

Appendix

Factual Knowledge in Language Models

LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

SG	Prom.	Reas.	Prec	True	False	Unv
✓	calib	min	96.3	93.7	3.4	2.9
✓	base	min	96.2	93.0	3.6	3.4
✗	calib	min	94.5	89.7	4.5	5.8
✗	base	min	$94.7^{\dagger}$	89.5	4.5	6.0

Bucket	Ref	Sampled	Found	Cov %	Prec	True	False	Unv
hop 0	Wiki	1	1	100	100.0	70.0	0.0	30.0
hop 1	Wiki	10	10	100	97.9	94.0	2.0	4.0
hop 2	Wiki	200	184	92	98.7	84.6	1.1	14.4
hop 3	Wiki	200	168	84	98.7	80.3	0.6	19.1
hop 4	Wiki	200	134	67	97.7	72.6	1.4	26.0
hop 5	Wiki	200	111	55.5	97.6	69.0	1.2	29.8

Abstract

Factual Knowledge in Language Models

Explore similar work

LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It