Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority

Aliakbar Mehdizadeh; Faezeh Khazaee; Martin Hilbert

cs.SIDec 3, 2025

Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority

Authors: Aliakbar Mehdizadeh, Faezeh Khazaee, Martin Hilbert

Organizations: Department of Communication, University of California, Davis

Abstract

For more than two decades, Wikipedia has represented a decentralized, human-curated model of encyclopedic knowledge production. The emergence of AI-generated encyclopedias, such as Grokipedia, raises a fundamental question: do human- and AI-curated knowledge systems rely on the same foundations of epistemic authority? We examine this question through a comparative analysis of 1,000 matched Wikipedia--Grokipedia article pairs, comprising nearly 600,000 citation instances. Using an eight-category classification scheme, we construct the epistemic profiles of articles to examine the institutional composition of their cited sources. Our findings reveal substantial differences in sourcing practices. Grokipedia relies proportionally less on academic and journalistic sources and more on user-generated content and NGO and think-tank publications. These differences persist across broad topic domains, although their magnitude varies by subject. We also find that Grokipedia adds citations approximately linearly as articles grow, whereas Wikipedia exhibits greater variation in citation density. Together, these findings suggest a shift in the institutional sources invoked to justify encyclopedic claims, which we characterize as epistemic substitution. For the same topics, the two encyclopedias rest on different institutional sources. Our findings underscore the importance of systematic, ongoing audits of information provenance for accountability in generative knowledge systems.

Figures & tables

Category	Definition and scope	Typical sources
Academic & Scholarly	Peer-reviewed original research and scholarly communication.	Journal articles, conference papers, academic books
Government & Official	Information, data, services, or legal documents produced by government bodies.	Census data, legislation, court records
News & Journalism	Original, factual reporting of current events under professional journalistic standards.	Newspaper and wire reports, broadcast news
NGO & Think Tank	Mission-driven research, policy papers, and advocacy reports from non-profit and civil-society organizations.	Policy briefs, NGO reports
Corporate & Commercial	Publications by for-profit firms about their own business.	Press releases, annual reports, white papers
Opinion & Advocacy	Persuasive commentary intended to promote a specific ideological position.	Op-eds, editorials, advocacy journalism

Table 1: Epistemic classification scheme. Eight mutually exclusive categories used to classify every citation.

	Stratum A ( $n=150$ )		Stratum B ( $n=50$ )
Comparison	% Agree	Coefficient [95% CI]	% Agree	Coefficient [95% CI]
Human–human agreement
Coder A vs Coder B	90	0.89 [0.82, 0.93]	30	0.23 [0.10, 0.36]
Krippendorff’s $\alpha$	–	0.89	–	0.19
Human–model agreement
Coder A vs Gemini Flash 3.6	93	0.92 [0.87, 0.97]	22	0.12 [0.00, 0.24]

Table 2: Agreement on the 200-citation validation sample. Two human coders evaluated a subset of citations. Stratum A contains citations where the models agreed (

n=150

), while Stratum B contains citations where they disagreed (

n=50

). Agreement coefficients are Cohen’s

\kappa

with 95% bootstrap confidence intervals, except where Krippendorff’s

\alpha

is reported.

Domain	Articles	Share (%)	Includes, e.g.
Culture	517	51.7	Biography, media and entertainment, music, sports, literature, philosophy and religion
Geography	296	29.6	Countries, cities, and regions of Africa, the Americas, Asia, Europe, and Oceania
History & Society	114	11.4	History, politics and government, military and warfare, business and economics, education
STEM	73	7.3	Physics, biology, medicine and health, technology, computing, space, earth and environment
Total	1,000	100.0

Table 3: Topic composition of the corpus. Each of the

N=1000

matched articles was assigned to the top-level domain of its highest-scoring label from the Wikimedia language-agnostic topic classifier; every article received a label. The last column lists representative subtopics of each domain in the Wikimedia taxonomy.

Per article	Wikipedia	Grokipedia
Word count	10,200.8 (3,837.0)	11,785.4 (4,300.1)
Citation count	336.7 (167.3)	257.5 (112.1)
Citation density (per 1,000 words)	34.0 (16.0)	21.8 (5.5)

Table 4: Article-level descriptive statistics. Summary statistics for the

N=1{,}000

matched article pairs. Values are reported as mean (standard deviation).

Figure 1: Citation volume and density across article lengths. (a) Word count against citation count; each point is one article. Lines show the selected OLS model for each platform with its 95% confidence band: quadratic for Wikipedia (adjusted $R^{2}=0.40$ ) and linear for Grokipedia (approximately 21.2 additional citations per 1,000 words; adjusted $R^{2}=0.66$ ). Dashed grey lines mark constant densities of 10, 20 and 40 citations per 1,000 words. (b) Citation density across articles: density curves, individual articles (points), interquartile ranges (bars) and medians (white dots; Wikipedia 31.8, Grokipedia 21.7 citations per 1,000 words). CV: coefficient of variation.

Figure 2: Epistemic profiles of Wikipedia and Grokipedia. Filled markers indicate the article-level mean share of each source category with 95% confidence intervals ( $N=1{,}000$ articles per platform). Open markers indicate the pooled share across all citations ( $N_{\mathrm{Wiki}}=336{,}686$ , $N_{\mathrm{Grok}}=257{,}486$ ).

Figure 3: Epistemic profiles across topic domains. Mean article-level share of citations in each source category, displayed as 100% stacked bars for Wikipedia and Grokipedia across four topical domains. Each domain shows Wikipedia above Grokipedia. Segments representing at least 6% are labeled. Grokipedia has a lower share of Academic & Scholarly citations in all four domains.

Figure 4: Topic-based epistemic metrics. (a) Jensen–Shannon distance between the citation-category distributions of matched Wikipedia and Grokipedia articles, grouped by topical domain. Density distributions are shown with interquartile ranges (bars) and medians (dots). Domains are ordered by median distance. Kruskal–Wallis $H=21.71$ , $p<0.001$ , $\eta^{2}_{H}=0.019$ . (b) Shannon entropy of article-level citation-category distributions. Wikipedia distributions appear above each horizontal line and Grokipedia distributions below, with arrows connecting the medians. Paired effect sizes ( $d_{z}$ ): Culture 1.16, Geography 1.00, History & Society 1.25, and STEM 0.88.

Figure 5: Article-level sourcing similarity between Wikipedia and Grokipedia ( $N=1{,}000$ matched article pairs). (a) Cosine similarity between the eight-category citation distributions of matched articles. (b) Jensen–Shannon distance between the corresponding epistemic profiles. Dashed lines indicate the medians.

Article	Topic	JS dist.	Cos.
(a) Most dissimilar
Warriors (novel series)	Culture $\cdot$ Literature	0.68	0.13
American Revolutionary War	Geography $\cdot$ Regions	0.68	0.18
The Holocaust	Geography $\cdot$ Regions	0.68	0.20
Ayn Rand	Culture $\cdot$ Philosophy and religion	0.68	0.16
Marilyn Monroe	Culture $\cdot$ Media	0.65	0.16

Table 5: Most dissimilar and most similar article pairs. The eight pairs with the highest and lowest Jensen–Shannon distances between Wikipedia’s and Grokipedia’s epistemic profiles, with corresponding cosine similarity (Cos.). Topics follow the Wikimedia classification at the top and second levels.

Panel of Figure 6 — Figure 6: Core epistemic co-occurrence networks. Nodes represent source categories, and edges indicate their co-occurrence within articles. Only edges with weights above the platform-specific mean are retained (Wikipedia: 782.8; Grokipedia: 854.2). Darker and thicker edges indicate stronger co-occurrence. (a) Wikipedia. (b) Grokipedia.

Figure 7: Article similarity networks (cosine similarity $>0.75$ ). Each node represents an article, colored by topic domain and sized by degree. Edges connect articles whose epistemic profiles have cosine similarity above 0.75. Within-topic edges take the corresponding topic color, while between-topic edges are gray. Each network is visualized using a force-directed layout, at most 25,000 randomly sampled edges are drawn. (a) Wikipedia. (b) Grokipedia.

	Size		Structure					Topic
Platform	Edges	Density	Comp.	Giant	Trans.	$Q$	Comm.	$r$
Threshold 0.65
Wikipedia	278,021	0.56	1	1.000	0.87	0.246	2	0.105
Grokipedia	261,025	0.52	1	1.000	0.81	0.247	2	0.195
Threshold 0.75
Wikipedia	230,423	0.46	1	1.000	0.87	0.291	3	0.153

Table 6: Article similarity network statistics across cosine similarity thresholds (

N=1{,}000

articles per platform). Density is the proportion of possible edges present; Comp. denotes connected components; Giant is the proportion of articles in the largest component; Trans. is transitivity;

Q

is Louvain modularity; Comm. is the number of Louvain communities containing more than one article; and

r

denotes topic assortativity. All assortativity coefficients are positive and statistically significant under 1,000 topic-label permutations (

p<0.001

Figure 8: Changes in Grokipedia between November 2025 and September 2026 for 69 articles observed at both dates. (a) Pooled citation composition across the eight source categories, with percentage-point changes between observations. (b) Citation counts and (c) article lengths at the two dates. Dashed lines indicate equality, faint lines indicate changes of $+25\%$ and $-20\%$ , the open black circles represent the marginal medians, and rings highlight the articles with the largest changes in each measure.

		Median		Change
Measure	$n$	Nov 2025	Sep 2026	Median %	$p$
Words	69	14,123	13,326	+0.7	0.959
Citations	69	306	290	+2.0	0.038
Citations per 1k words	69	22.3	21.9	+0.1	0.576
Source profile
Jensen–Shannon distance, Nov $\rightarrow$ Sep, median [IQR]		0.101 [0.063, 0.159]

Table 7: Longitudinal changes in Grokipedia articles. Comparison of 69 matched articles observed in November 2025 and September 2026. Median percentage changes are calculated at the article level. Reported

p

-values are from paired Wilcoxon signed-rank tests. Jensen–Shannon distance measures changes in each article’s citation-category distribution between the two dates and is reported as median [interquartile range].

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

	Wikipedia		Grokipedia
Category	$n$	%	$n$	%
Academic & Scholarly	89,763	26.66	25,574	9.93
Corporate & Commercial	23,913	7.10	31,462	12.22
Government & Official	27,453	8.15	32,013	12.43
NGO & Think Tank	14,520	4.31	30,405	11.81
News & Journalism	146,417	43.49	79,600	30.91

Appendix

Table S1: Global epistemic profile, pooled across all citations. Citation counts and corpus-level shares for Wikipedia and Grokipedia. Values give the total composition of the dataset (

N_{\mathrm{Wiki}}=336{,}686

N_{\mathrm{Grok}}=257{,}486

) rather than the per-article average.

Figure S1: Multi-Agent Workflow to Develop Coding Manual

	ChatGPT5	Claude S.4.5	Gemini2.5	Grok4
	Deep Research	Research	Pro Deep	Expert
1 News & Journalism	1	1	1	1
2 Opinion and Advocacy	1	1	1	1
3 NGO, Civil Soc. & Think Tank	1	1	1	1
4 Government & Official	1	1	1	1
5 Academic & Scholarly	1	1	1	1

Appendix

Table S2: LLM Category Identification Summary

Domain	GP	Cl	Ge	Gr
digitalspy.co.uk	1	1	1	1
estadao.com.br	1	1	1	1
wagnerreese.com	7	7	7	7
hapsc.org	3	3	3	3
electionguide.org	6	6	6	6
tasteiran.net	7	7	7	7

Appendix

Table S3: Intercoder Reliability Test Results

Category	Primary Function	Key Indicators (for the work)	Examples
Academic & Scholarly	Peer-reviewed, original research and scholarly communication.	Peer-review, university press, scholarly journal, dissertation, conference paper.	nature.com article, jstor.org article, academic book.
Government & Official	Official information, data, services, or legal documents from government bodies.	Government agency author, court records, legislation, official statistics.	NASA report, Census data, .gov publication, UN resolution.
NGO & Think Tank	Mission-driven research and advocacy by non-profits.	Non-profit status, independent research, policy papers, advocacy reports.	Brookings report, Amnesty Int’l document, Greenpeace publication.
News & Journalism	Original, factual reporting of current events.	Verifiable facts, journalistic standards, byline, reporting.	nytimes.com article, BBC report, Reuters dispatch.
Opinion & Advocacy	Persuasive commentary from an ideological perspective.	Op-ed, editorial, commentary, advocacy journalism, ideological positioning.	huffpost.com op-ed, dailywire.com article, blog post arguing a position.
Corporate & Commercial	For-profit business products, services, and marketing.	Press release, annual report, company blog, product white paper, patent.	apple.com press release, Tesla annual report.

Appendix

Table S4: Citation Category Definitions

Case	Decision	Reasoning
Wall Street Journal factual article	NEWS	Original factual reporting (Rule 1).
Wall Street Journal op-ed	OPINION	Persuasive commentary (Rule 1).
Tesla blog post about technology	CORPORATE	Company-owned marketing/communication.
Wikipedia page for "History of Japan"	REFERENCE	Tertiary summary of a general topic.
Google Books page for Harvey’s An Introduction to Buddhism	ACADEMIC	“Look-Through Principle” (Rule 2) applied.
Kotkin’s Stalin with page numbers, title linked to its Wikipedia page	ACADEMIC	Classified by the book’s bibliographic details (Rule 2).

Appendix

Table S5: Citation Boundary Cases

Platform	Article	Citation	Category
Wikipedia	Buddhism	Harvey (2013), An Introduction to Buddhism , Cambridge University Press, linked via Google Books	Academic & Scholarly
Wikipedia	Jimmy Carter	The Washington Post , opinion section ( /opinions/ )	Opinion & Advocacy
Grokipedia	2016 U.S. presidential election	U.S. Census Bureau, “Characteristics of Voters in the Presidential Election of 2016”	Government & Official
Grokipedia	2003 invasion of Iraq	Brookings Institution, “Iraq Without a Plan”	NGO & Think Tank
Grokipedia	2003 invasion of Iraq	Britannica, “Iraq War: Summary, Causes, Dates, Combatants, Casualties, & Facts”	Reference & Tertiary
Grokipedia	50 Cent	Reddit thread (r/todayilearned)	User-Generated Content

Appendix

Table S6: Examples of classified citations. Real citations from the most-revised corpus with the category assigned by the classifier.

Category	$n$	Precision	Recall	F1
Academic & Scholarly	17	0.94	0.94	0.94
Corporate & Commercial	21	0.95	0.90	0.93
Government & Official	20	0.87	1.00	0.93
NGO & Think Tank	23	0.95	0.83	0.88
News & Journalism	20	1.00	0.95	0.97
Opinion & Advocacy	12	0.85	0.92	0.88

Appendix

Table S7: Production classifier against the two-coder consensus, by category.

n

is the number of citations whose consensus label is that category; total

n=150

Classifier	Human consensus	$n$	Example citation
Reference & Tertiary	User-Generated Content	3	"Communism" . Thinkquest . Archived from the original on 9 April 2000 . Retrieved 29 January 2012 .
Opinion & Advocacy	NGO & Think Tank	2	Uniquely Bad—But Not Uniquely American - City Journal
User-Generated Content	Corporate & Commercial	1	"Floyd Estel Parton" . Archived from the original on August 2, 2025 . Retrieved August 15, 2025 .
Reference & Tertiary	Academic & Scholarly	1	"Storia del teatro: lo spazio scenico in Toscana" (in Italian) . Retrieved 28 July 2022 .
Corporate & Commercial	News & Journalism	1	"World Air Forces 2025" . Flightglobal : 33. Archived from the original on 21 January 2025 . Retrieved 5 March

Appendix

Table S8: Disagreements between the classifier and the human consensus.

	Wikipedia	Grokipedia
Linear model
Intercept	64.30 (12.95)	7.29 (9.04)
Slope (per 1,000 words)	26.70 (1.41)	21.23 (0.84)
Adjusted $R^{2}$	0.37	0.66
Quadratic model
Intercept	$-$ 73.89 (24.16)	$-$ 36.56 (18.81)

Appendix

Table S9: Citation count as a function of article length. OLS,

N=1{,}000

articles per platform.

Figure S2: Topics of the most- and least-cited articles. For the 50 articles with the most citations (solid) and the fewest (pale) on each platform, the share of each topic domain minus that domain’s share of the full corpus, in percentage points. Zero means the topic appears as often as in the corpus as a whole.

Wikipedia		Grokipedia
Article	Citations	Article	Citations
Most cited
Vladimir Lenin	1,533	History of the United States	914
Joseph Stalin	1,371	Elon Musk	832
Nelson Mandela	1,188	India	686
Muammar Gaddafi	945	Brazil	661

Appendix

Table S10: Most- and least-cited articles on each platform. Examples from the top and bottom of each platform’s ranking by number of citations.

Panel of Figure 28 — Figure S3: Sensitivity Analysis of Epistemic Backbones (Cosine Similarity). Edges represent the cosine similarity between citation categories calculated from per-article presence vectors, retaining edges above each platform’s mean similarity. Hollow dashed nodes mark categories with no edge above the cutoff.

	Wikipedia			Grokipedia
Cutoff rule	Value	Edges	Academic in core	Value	Edges	Academic in core
Mean	783	14	yes	854	14	no
Median	782	14	yes	858	14	no
60th pct.	797	11	yes	891	11	no
75th pct.	836	7	yes	927	7	no

Appendix

Table S11: Co-occurrence backbones under four cutoff rules. Value : the edge-weight cutoff implied by each rule, computed within platform. Edges : edges retained out of 28 possible pairs.

		Distribution		Effect size		Rank-based
Topic	$N$	Skew	Shapiro $p$	Mean diff.	$d_{z}$	$r_{rb}$	% higher
Culture	517	0.73	$<$ 0.001	$+$ 0.524	1.16	0.93	89.9
Geography	296	1.49	$<$ 0.001	$+$ 0.505	1.00	0.96	92.9
History & Society	114	0.86	$<$ 0.001	$+$ 0.741	1.25	0.97	93.0
STEM	73	0.84	0.019	$+$ 0.421	0.88	0.83	83.6
All	1,000	1.06	$<$ 0.001	$+$ 0.535	1.09	0.94	90.7

Appendix

Table S12: Paired comparisons of source-mix entropy across topic groups. Differences are Grokipedia

-

Wikipedia. Distribution : skewness and Shapiro–Wilk

p

of the paired differences. Effect size : mean difference (bits) and paired Cohen’s

d_{z}

. Rank-based : matched-pairs rank-biserial correlation

r_{rb}

and the share of pairs in which Grokipedia’s entropy is higher. Wilcoxon signed-rank

p<0.001

in every row.

Threshold	Wikipedia $r$	Edges	Grokipedia $r$	Edges
0.60	0.087	299,040	0.150	292,961
0.65	0.105	278,021	0.196	261,025
0.70	0.127	255,287	0.245	228,333
0.75	0.152	230,423	0.298	195,128
0.80	0.179	201,505	0.354	160,241
0.85	0.200	170,078	0.406	124,032

Appendix

Table S13: Sensitivity of the topic assortativity coefficient

r

to the cosine threshold. Edges : edges retained at each threshold;

N=1{,}000

articles per platform.

Panel of Figure 32 — Figure S4: Topic Mixing Matrices. Heatmaps display the row-normalized mixing matrix, representing the conditional probability that an article of a given topic connects to a target topic based on sourcing similarity ( $>0.75$ ). Grokipedia has higher diagonal shares for Culture (0.76 versus 0.66) and Geography (0.52 versus 0.41).

	Most-revised		Vital Articles
	Wikipedia	Grokipedia	Wikipedia	Grokipedia
Matched article pairs	1,000		200
Epistemic profile (%)
Academic & Scholarly	26.7	9.9	61.8	35.8
Government & Official	8.2	12.4	8.7	13.1
News & Journalism	43.5	30.9	11.4	8.9

Appendix

Table S14: Epistemic profile and citation density in the most-revised and Vital Articles corpora. Shares are percentages of each platform’s citations. Citation density is citations per 1,000 words.

Data Provenance in Language Models

See all 27 papers

Mar 17, 2026

Answer Bubbles: Information Exposure in AI-Mediated Search

Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these systems differ in sources, language, and fidelity to cited material. We examine responses to 11,000 real search queries across five systems---vanilla GPT, Search GPT, Perplexity Search with Grok, Google AI Overviews, and traditional Google Search---at three levels: source diversity, linguistic characterization of the generated summary, and source-summary fidelity. We find that generative search systems exhibit significant source-selection biases in their citations, favoring certain sources over others. Incorporating search also selectively attenuates epistemic markers, reducing hedging by up to 60% while preserving confidence language in the AI-generated summaries. At the same time, AI summaries further compound the citation biases: Wikipedia and longer sources are disproportionately overrepresented, whereas cited social media content and negatively framed sources are substantially underrepresented. Our findings highlight the potential for answer bubbles, in which identical queries yield structurally different information realities across systems, with implications for user trust, source visibility, and the transparency of AI-mediated information access.

Michelle Huang, Agam Goyal, Koustuv Saha +1

Siebel School of Computing and Data Science University of Illinois Urbana-Champaign, Urbana, IL, USA

Generative Retrieval Information Retrieval Algorithmic Bias Generative AI Evaluation

Ask PDF

Jul 16, 2026

Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies

Online encyclopedias shape political opinion and, through it, democratic discourse. In late 2025, Grokipedia was released, an encyclopedia written entirely by the LLM Grok. One motivation behind the project was to provide an unbiased alternative to Wikipedia, which has faced accusations of "left-wing" and "liberal" bias. But does an encyclopedia written by an LLM deliver greater neutrality, or does it simply embed a different ideology? We conduct a large-scale political bias study on Grokipedia and Wikipedia, analysing 1,394 article pairs describing members of government for neutrality along nine expert-coded ideology dimensions employing four LLM judges, Grok, Claude, Mistral, and DeepSeek. As the LLMs could themselves be biased, we also investigate patterns in their judgments. We find all LLM-judges, including Grok, to rate Grokipedia less neutral than Wikipedia. Both encyclopedias are rated as portraying politicians favourably overall, but towards different ideological groups. Grokipedia particularly favours economically right-wing politicians and penalises socially liberal ones, while Wikipedia is rated as favourably biased towards the latter.

Filippos Vlahos, Guillaume Bied, Tijl De Bie

Ghent University, BE

LLM Auditing Political Bias in Language Models Language Model Bias Evaluation

Ask PDF

May 13, 2026

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact

Google AI Overviews (AIOs) are arguably the most widely encountered deployment of generative AI, reaching over 2 billion users who may not realize the answers they see are AI-generated. Where search engines have traditionally surfaced ranked sources and left users to evaluate them, AIOs synthesize and deliver a single answer - giving Google unprecedented editorial control over what users read and know. We present a large-scale longitudinal measurement study, issuing 55,393 trending queries across 19 topical categories over a 40-day window (March 13 - April 21, 2026). We report four main findings. First, overall AIO activation is 13.7%, rising to 64.7% for question-form queries, while politically sensitive topics see markedly lower rates. Second, AIO-cited domains are more credible than co-displayed first-page results, yet nearly 30% do not appear in those results at all, indicating a source selection mechanism distinct from Google's ranking algorithm. Third, decomposing responses into 98,020 atomic claims, 11.0% are unsupported by the cited pages - with omission the dominant failure mode - and source quality and claim fidelity are largely independent. Fourth, well over half of AIO-cited pages carry display advertising, meaning publishers lose revenue when AIOs suppress the click-through, even as Google's own sponsored ads continue to appear on the same page. Together, these findings document a rapid transformation of the online information ecosystem whose consequences for epistemic security remain poorly understood.

Haofei Xu, Umar Iqbal, Jacob M. Montgomery

Washington University in St. Louis

Citation Verification Generative Retrieval Evidence-Grounded Generation IR Evaluation Generative AI Evaluation

Ask PDF

Abstract

Data Provenance in Language Models

Explore similar work

Answer Bubbles: Information Exposure in AI-Mediated Search

Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact