Answer Bubbles: Information Exposure in AI-Mediated Search

Michelle Huang; Agam Goyal; Koustuv Saha; Eshwar Chandrasekharan

cs.IRMar 17, 2026

Answer Bubbles: Information Exposure in AI-Mediated Search

Authors: Michelle Huang, Agam Goyal, Koustuv Saha, Eshwar Chandrasekharan

Organizations: Siebel School of Computing and Data Science University of Illinois Urbana-Champaign, Urbana, IL, USA

Abstract

Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these systems differ in sources, language, and fidelity to cited material. We examine responses to 11,000 real search queries across five systems---vanilla GPT, Search GPT, Perplexity Search with Grok, Google AI Overviews, and traditional Google Search---at three levels: source diversity, linguistic characterization of the generated summary, and source-summary fidelity. We find that generative search systems exhibit significant source-selection biases in their citations, favoring certain sources over others. Incorporating search also selectively attenuates epistemic markers, reducing hedging by up to 60% while preserving confidence language in the AI-generated summaries. At the same time, AI summaries further compound the citation biases: Wikipedia and longer sources are disproportionately overrepresented, whereas cited social media content and negatively framed sources are substantially underrepresented. Our findings highlight the potential for answer bubbles, in which identical queries yield structurally different information realities across systems, with implications for user trust, source visibility, and the transparency of AI-mediated information access.

Figures & tables

Figure 1: Paper Overview. Traditional search returns a ranked list of links for users to evaluate, while generative search produces an answer bubble containing AI-generated summaries synthesized from multiple sources. We study these answer bubbles along three dimensions: the sources they cite (RQ1), the linguistic and epistemic qualities of their summaries (RQ2), and how faithfully those summaries represent cited content (RQ3).

Figure 2: Top-15 cited domains by query topic for each source (% of queries citing each domain). Cell values $\geq$ 1% are shown. Domain preferences are strongly topic-dependent: IMDB dominates entertainment, ESPN dominates sports, and Spotify/Genius dominate music, but only in Google’s systems and not Search GPT.

	Mean				Cohen’s $d$
Feature	$\textbf{GPT}_{\textbf{V}}$	$\textbf{GPT}_{\textbf{S}}$	$\textbf{Google}_{\textbf{AI}}$	$\textbf{GPT}_{\textbf{V}}$ vs $\textbf{GPT}_{\textbf{S}}$	$\textbf{GPT}_{\textbf{V}}$ vs $\textbf{Google}_{\textbf{AI}}$	$\textbf{GPT}_{\textbf{S}}$ vs $\textbf{Google}_{\textbf{AI}}$
Verbosity & readability
Word Count	87	140	83	$\par\ignorespaces\pagecolor{negcolor!69!white}-0.65$ ***	$\ignorespaces\pagecolor{poscolor!5!white}0.05$ ***	$\ignorespaces\pagecolor{poscolor!68!white}0.64$ ***
Avg. Sent. Length	16.3	18.0	11.4	$\par\ignorespaces\pagecolor{negcolor!18!white}-0.17$ ***	$\ignorespaces\pagecolor{poscolor!62!white}0.58$ ***	$\ignorespaces\pagecolor{poscolor!63!white}0.59$ ***
Avg. Word Length	5.07	5.25	4.17	$\par\ignorespaces\pagecolor{negcolor!36!white}-0.34$ ***	$\ignorespaces\pagecolor{poscolor!80!white}1.06$ ***	$\ignorespaces\pagecolor{poscolor!80!white}1.31$ ***
Coleman-Liau Index	11.5	12.8	11.7	$\par\ignorespaces\pagecolor{negcolor!34!white}-0.32$ ***	$\par\ignorespaces\pagecolor{negcolor!6!white}-0.06$ **	$\ignorespaces\pagecolor{poscolor!29!white}0.27$ ***

Table 1: Psycholinguistic and lexico-semantic differences across response sources (Benjamini-Hochberg corrected). KW: Kruskal-Wallis test. Pairwise significances determined by Mann-Whitney

U

. ***

p<.001

, **

p<.01

, *

p<.05

	Search GPT		Google AIO
Topic	EC	Coverage	EC	Coverage
Business/Fin./Econ.	0.162	0.500	0.162	0.431
Education/Lit.	0.189	0.481	0.175	0.465
Entertainment	0.185	0.480	0.217	0.459
History	0.160	0.460	0.169	0.445
Music	0.236	0.449	0.177	0.407

Table 2: Equal Coverage (EC) and mean coverage probability (Coverage) by topic. EC measures how unevenly a summary draws from its cited sources ( lower = more even). Coverage indicates the fraction of source content entailed by summary ACUs.

Document Length
	Search GPT	Google AIO
	Mean Diff (pp)	Mean Diff (pp)
Short ( $<$ 200 words)	$-0.134$ ***	$-0.177$ ***
Medium (200–800)	$+0.000$	$+0.015$ **
Long ( $>$ 800 words)	$+0.039$ ***	$+0.046$ ***
Source Sentiment

Table 3: Coverage Parity (CP) analysis across attribute schemes. Higher CP values indicate more systematic bias. Significance via bootstrap resampling (n=1,000). 95% bootstrap confidence intervals for these estimates are reported in Table A4 .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Topic	# Queries	Top 5 Key Terms
Business, Finance, & Economy	1,000	company (53), India (51), United States (48), world (43), bank (43)
Education & Literature	1,000	book (107), wrote (95), school (77), name (50), summary (40)
Entertainment	1,000	play (193), movie (116), season (110), episode (90), film (56)
Geography & Travel	1,000	locate (148), map (90), world (69), state (61), river (60)
History	1,000	war (92), name (58), India (58), world (38), United States (34)
Music	1,000	song (239), sing (216), write (88), love (72), lyrics (47)

Appendix

Table A1 : Query Dataset Summary. Table representing all 11 topics and the number of queries per topic in our final dataset. We also report the top 5 most frequently appearing key terms per topic with frequencies in parentheses.

Domain category (% of cited domains)
	Search GPT	Google AIO	Perplexity	Organic
Queries w/ $\geq$ 1 citation	97.0%	57.8%	74.5%	99.8%
Mean citations/query	3.2	4.0	5.7	9.3
Median citations/query	3	4	6	10
Unique domains	7,606	14,279	22,562	26,891
Encyclopedia/Reference	27.3%	10.3%	13.1%	11.0%

Appendix

Table A2 : Source summary. Domain categories show each category’s share of all cited domains.

Figure A1: Top-15 cited domains by query topic for Perplexity (% of queries citing each domain), in the same format as Figure 2 . Cell values $\geq$ 1% are shown.

Feature	$\textbf{GPT}_{\textbf{V}}$	$\textbf{GPT}_{\textbf{S}}$	$\textbf{Google}_{\textbf{AI}}$	Perplexity
Word Count	87	140	83	190
Avg. Sent. Length	16.3	18.0	11.4	14.2
Coleman-Liau Index	11.5	12.8	11.7	12.8
Type-Token Ratio	0.73	0.64	0.75	0.68
cogproc	0.087	0.067	0.044	0.082
tentat	0.025	0.015	0.010	0.022

Appendix

Table A3 : Descriptive comparison of Perplexity against the systems in Table 1 on the subset of features computed for all four systems. Values are means; Perplexity is interpreted descriptively rather than via inferential tests.

Document Length
	Search GPT	Google AIO	Perplexity
	Mean Diff (pp) [95% CI]	Mean Diff (pp) [95% CI]	Mean Diff (pp) [95% CI]
Short ( $<$ 200 words)	$-0.134^{{}^{***}}$ $[-0.16,-0.11]$	$-0.177^{{}^{***}}$ $[-0.19,-0.16]$	$-0.144^{{}^{***}}$ $[-0.16,-0.13]$
Medium (200–800)	$+0.000$ $[-0.01,+0.02]$	$+0.014^{{}^{**}}$ $[+0.01,+0.02]$	$+0.003$ $[-0.01,+0.01]$
Long ( $>$ 800 words)	$+0.039^{{}^{***}}$ $[+0.03,+0.05]$	$+0.046^{{}^{***}}$ $[+0.04,+0.05]$	$+0.050^{{}^{***}}$ $[+0.04,+0.06]$
Source Sentiment

Appendix

Table A4 : Coverage Parity (CP) analysis across attribute schemes with 95% bootstrap confidence intervals, extending Table 3 with Perplexity and interval estimates.

	Search GPT		Google AIO		Perplexity
Topic	EC	Coverage	EC	Coverage	EC	Coverage
Business/Fin./Econ.	0.162	0.500	0.162	0.447	0.166	0.293
Education/Lit.	0.189	0.481	0.175	0.445	0.169	0.316
Entertainment	0.185	0.480	0.217	0.439	0.245	0.384
History	0.160	0.460	0.169	0.414	0.151	0.315
Music	0.236	0.449	0.177	0.472	0.213	0.448

Appendix

Table A5 : Equal Coverage (EC) and mean coverage probability (Coverage) by topic, extending Table 2 with Perplexity.

Topic	cogproc	tentat	posemo	formality	politeness
Technology	1.37	0.69	1.30	1.19	0.84
Travel/Geography	0.88	0.64	1.01	1.76	0.79
Sports	0.78	0.40	0.85	0.95	0.94
Science	0.75	0.43	0.82	1.28	0.38
Politics/News	0.56	0.37	0.76	1.21	0.02
Music	0.40	0.17	0.62	1.26	0.34

Appendix

Table A6 : Cohen’s

d

for Search GPT vs. Google AIO by topic. Positive

d

indicates higher values in Search GPT. Bold := large effect (

|d|\!\geq\!0.8

). Topics are sorted by mean

|d|

across the five features.

Generative Retrieval

Generative retrieval uses language models to produce relevant document or item identifiers directly from a query.

See all 45 papers

How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews
Apr 30, 2026Riley Grossman, Songjiang Liu, Michael K. Chen +3Generative Engine Optimization Generative Retrieval
New Jersey Institute of Technology Newark, NJ, USA · Nanyang Technological University Singapore, Singapore · Indiana University Bloomington Bloomington, IN, USA
Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact
May 13, 2026Haofei Xu, Umar Iqbal, Jacob M. MontgomeryCitation Verification Generative Retrieval
Washington University in St. Louis
Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement
Mar 9, 2026Ronald SielinskiGenerative AI Evaluation Uncertainty Quantification
IQRush

Apr 30, 2026J/K move · Enter open · S save

How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews

Riley Grossman, Songjiang Liu, Michael K. Chen, Mike Smith+2

New Jersey Institute of Technology Newark, NJ, USA · Nanyang Technological University Singapore, Singapore · Indiana University Bloomington Bloomington, IN, USA

Generative Engine Optimization Generative Retrieval IR Evaluation

Open paper AskPDF arXiv

Generative AI is being increasingly integrated into web search for the convenience it provides users. In this work, we aim to understand how generative AI disrupts web search by retrieving and presenting the information and sources differently from traditional search engines. We introduce a public benchmark dataset of 11,500 user queries to support our study and future research of generative search. We compare the search results returned by Google's search engine, the accompanying AI Overview (AIO), and Gemini Flash 2.5 for each query. We have made several key findings. First, we find that for 51.5% of representative, real-user queries, AIOs are generated, and are displayed above the organic search results. Controversial questions frequently result in an AIO. Second, we show that the retrieved sources are substantially different for each search engine (<0.2 average Jaccard similarity). Traditional Google search is significantly more likely to retrieve information from popular or institutional websites in government or education, while generative search engines are significantly more likely to retrieve Google-owned content. Third, we observe that websites that block Google's AI crawler are significantly less likely to be retrieved by AIOs, despite having access to the content. Finally, AIOs are less consistent when processing two runs of the same query, and are less robust to minor query edits. Our findings have important implications for understanding how generative search impacts website visibility, the effectiveness of generative engine optimization techniques, and the information users receive. We call for revenue frameworks to foster a sustainable and mutually beneficial ecosystem for publishers and generative search providers.

Abstract

Generative Retrieval

Explore similar work

How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact

Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement