The Role of AI in Online Reviews

Valeria Lerman; Oren Rigbi; Yaniv Dover

cs.CLAug 30, 2026

The Role of AI in Online Reviews

Authors: Valeria Lerman, Oren Rigbi, Yaniv Dover

Organizations: The Hebrew University Business School, The Hebrew University of Jerusalem, Jerusalem 9190501, Israel · Department of Cognitive and Brain Sciences, Faculty of Humanities, The Hebrew University of Jerusalem, Jerusalem 9190501, Israel · The Federmann Center for the Study of Rationality, The Hebrew University of Jerusalem, Jerusalem 9190501, Israel

Abstract

The rapid adoption of large language models (LLMs) creates new opportunities for strategic content generation on online platforms, including potentially harmful forms of manipulation that may undermine platform effectiveness and reshape platform dynamics. However, measuring such activity is difficult because AI-generated content is rarely directly observable. We introduce an empirical approach that leverages discrete LLM supply shocks - abrupt changes in model prices and capabilities, and contrasts verified with non-verified reviews to identify changes in platform activity associated with generative AI supply improvements. We apply this approach to more than 13 million reviews from Trustpilot, one of the leading online platforms for business reviews. A robust finding is that following LLM supply shocks, unverified reviews shift toward greater negativity: more 1-stars, fewer 5-stars, and lower ratings, with effects driven primarily by new model releases and concentrated among firms with the lowest and highest review volumes, suggesting that strategic AI use may reshape platform competition dynamics. We further find that LLM supply shocks trigger short, concentrated bursts of review activity. Together, these findings suggest that generative AI is already reshaping how reputation and competition operate on online platforms.

Figures & tables

Figure 1: OpenAI Model Pricing Reductions and Releases, 2023–2024 (normalized to initial price = 100%)

Figure 2: Daily Number of Reviews Over Time (2023–2024)

	(1)	(2)	(3)
	Avg. rating	Prop. 1-star	Prop. 5-star
Treatment	-1.0403***	0.2648***	-0.2268***
	(0.022)	(0.005)	(0.006)
After	0.0045	-0.0013	0.0006
	(0.005)	(0.001)	(0.002)
Treatment $\times$ After	-0.0171***	0.0034***	-0.0046***

Table 3: Difference-in-Differences Results: Seven-Day Event Window

Figure 3: Event-Study Estimates of the Effect of LLM Supply Shocks on Rating Outcomes and Review-Volume Categories

Figure 4: Event-Study Estimates for Price Reduction Events Only

Figure 5: Event-Study Estimates for New Model Release Events Only

Figure 6: Event-Study Estimates for Events Involving Both Price Reductions and New Model Releases

Quantile	Review-volume range	Number of companies
Q1 (Lowest)	50–80	5,056
Q2	80–152	5,056
Q3	152–431	5,056
Q4 (Highest)	431–240,745	5,056

Table 5: Company Size Quantiles Based on Total Review Volume

Figure S1 : OpenAI Model Pricing Reductions and Releases, 2023–2024 (raw prices)

	(1)	(2)	(3)	(4)	(5)
	Count	Log count	Length	Log length	Homogeneity
Treatment	-0.066***	0.012***	34.841***	0.670***	0.0363***
	(0.013)	(0.001)	(0.591)	(0.010)	(0.0015)
After	-0.023***	-0.008***	0.269	-0.007*	0.0006
	(0.003)	(0.000)	(0.213)	(0.004)	(0.0006)
Treatment $\times$ After	-0.004	-0.001***	0.227	0.003	-0.0009

Table S1 : Difference-in-Differences Results: Review Count, Length, and Text Homogeneity

	(1)	(2)	(3)	(4)	(5)
	Log count 1	Log count 2	Log count 3	Log count 4	Log count 5
Treatment	0.01636***	-0.00037***	-0.00181***	-0.00339***	-0.00576***
	(0.00036)	(0.00013)	(0.00018)	(0.00028)	(0.00094)
After	-0.00217***	-0.00033***	-0.00033***	-0.00079***	-0.00582***
	(0.00008)	(0.00004)	(0.00005)	(0.00007)	(0.00017)
Treatment $\times$ After	-0.00035***	0.00008**	0.00003	0.00003	-0.00038**

Table S2 : Difference-in-Differences Results: Log Review Counts by Rating Category (1–5)

Outcome	Pre-trend p-value	Post-treatment p-value	Average post coefficient
Count	0.007***	0.114	-0.004255
Log count	$<0.001$ ***	$<0.001$ ***	-0.000721
Length	0.640	0.089*	0.255823
Log length	0.599	0.122	0.003607
Homogeneity	0.270	0.285	-0.000647
Log count 1	$<0.001$ ***	$<0.001$ ***	-0.000349

Table S3 : Pre-Trend and Post-Treatment Tests

Figure S2 : Event-Study Coefficients for Total Count, Length, and Homogeneity

Figure S3 : Event-Study Coefficients for Log Review Counts by Rating Category

	(1)	(2)	(3)
	1+5-Star Rating	1+5-Star Rating	1+5-Star Rating
	Adjusted Threshold	Adjusted Threshold	Adjusted Threshold
	Low-volume: 0–17	Moderate-volume: 18–84	High-volume: 85+
Treatment	0.001243***	-0.001077***	-0.000167***
	(0.00013)	(0.00012)	(0.00004)
After	0.000224***	-0.000217***	-0.000007

Table S4 : Difference-in-Differences Results for Combined 1-Star and 5-Star Review-Volume Categories Using Adjusted Thresholds

Figure S4 : Event-Study Estimates for Combined 1-Star and 5-Star Review-Volume Categories Using Adjusted Thresholds

A Survey on Fake Review Detection: From Pre-trained Language Models to Large Language Models
Sep 12, 2026Fanji Yang, Huiyao Chen, Xi Yu +3
Guizhou University of Finance and Economics · Guizhou University of Finance and Economics, Guiyang, China · Harbin Institute of Technology +3
One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders
Jun 11, 2026Minghao Luo, Liang ChenLLM Evaluation RAG Security
The Chinese University of Hong Kong
Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
Jun 16, 2026Xi Chu, Yupeng HouGenerative Engine Optimization Language Model Bias Evaluation
Trine University · Texas A&M University

Sep 12, 2026J/K move · Enter open · S save

A Survey on Fake Review Detection: From Pre-trained Language Models to Large Language Models

Fanji Yang, Huiyao Chen, Xi Yu, Meishan Zhang+2

Guizhou University of Finance and Economics · Guizhou University of Finance and Economics, Guiyang, China · Harbin Institute of Technology+3

Open paper AskPDF arXiv

Online reviews shape consumer decisions, platform governance, and corporate reputation.Fake reviews compromise this information channel by injecting deceptive evidence into rating systems, recommendation pipelines, and public trust mechanisms.The rise of large language models, or LLMs, has changed the problem in two directions.LLMs can generate fluent and context-aware deceptive reviews, while pre-trained language models, or PLMs, and LLMs also provide stronger semantic representations for detection.This survey reviews fake review detection from an information fusion perspective, covering 211 studies published from 2018 to early 2026.We organize existing work by evidence source and fusion level, covering review text, sentiment, rating behavior, temporal metadata, user-product graphs, multimodal content, external knowledge, and LLM-generated signals.We trace the development from traditional machine learning and deep learning to PLM-based and LLM-based methods, and examine how different approaches combine textual, behavioral, structural, and multimodal evidence.We also analyze reported performance trends on widely used Amazon, Yelp, and OpSpam benchmark families, while noting the limitations caused by different label construction procedures, data splits, and evaluation protocols.Finally, we identify open problems in adversarial generation, cross-domain transfer, uncertainty-aware fusion, missing-source robustness, interpretability, and trustworthy evaluation for AI-generated deceptive content.

Abstract

Explore similar work

A Survey on Fake Review Detection: From Pre-trained Language Models to Large Language Models

One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders

Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems