cs.IRMar 30, 2026

Exploring LLM biases to manipulate AI search overview

Authors: Roman Smirnov

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern large language models (LLMs) are used in many business applications in general, and specifically in web search systems and applications that generate overviews of search results - LLM Overview systems. Such systems are using an LLM to select most relevant sources from search results and generate an answer to the user's query. It is known from many studies that LLMs have different biases, in LLM Overview application both the source selection and answer generation stages may be affected by the biases of LLMs (here we are focusing mainly on the selection stage). This research is focused on investigating the presence of the biases in LLM Overview systems and on biases exploitation to manipulate LLM Overview results. Here we train a small language model using reinforcement learning to rewrite search snippets to increase their likelihood of being preferred by an LLM Overview. Our experimental setup intentionally restricts the policy to operate only on snippets and limits reward-hacking strategies, reflecting realistic constraints of web search environments. The results prove that LLM Overview systems have biases and that reinforcement learning in most of the cases can optimize snippet's content to manipulate LLM Overview results. We also prove that LLM Overview selections are driven by comparative rather than absolute advantages among candidate sources. In addition, we examine safety aspects of LLM Overview manipulation possibilities and show that context poisoning attacks can lead to inaccurate or harmful results.

Figures & tables

Figure 1: LLM Overview schema and potential biases
URLTitle (first row) and snippetAppearances
https://www.bestbuy.comStarbucks - 10GiftCards(3−Pack)ShopStarbucks10 Gift Cards (3-Pack) Shop Starbucks 10 Gift Cards (3 Pack) products at Best Buy. Find low everyday prices and buy online for delivery or in-store pick-up.7/7
https://www.amazon.comAmazon.com: Starbucks 10GiftCards(4−Pack)Thisitemcontains4separate10 Gift Cards (4-Pack) This item contains 4 separate 10 gift cards; Starbucks Cards redeemable at most SB locations; It’s a great way to treat a friend.7/7
https://www.starbucks.comGift Cards - Starbucks # Gift cards. Carousel content with 8 slides. Carousel content with 2 slides. Carousel content with 3 slides. ## Thank You. Carousel content with 3 slides. Carousel content with 3 slides. Carousel content with 3 slides. Carousel content with 2 slides. Carousel content with 1 slides….2/7
https://www.amazon.comStarbucks Coffee: Gift Cards - Amazon.com Image of Starbucks 10GiftCards(4−Pack).Starbucks10 Gift Cards (4-Pack). Starbucks 10 Gift Cards (4-Pack). 4.7 out of 5 stars 15,726 customer reviews. $40.00.3/7
https://www.brookshires.comStarbucks 10X4Giftcard−1Each−Brookshire’s10 X4 Giftcard - 1 Each - Brookshire’s 10 x 4. You’ll always have the perfect gift, on hand. Four cards to share with four of your luckiest friends. Cards have no value until activated by cashier.3/7
https://www.walmart.comStarbucks Gift Cards in Restaurant Gift Cards - Walmart.com Shop for Starbucks Gift Cards in Restaurant Gift Cards. Buy products such as Starbucks Gift Card, Starbucks 40MP(4x40MP (4x10) Gift Card at Walmart and save.0/7
Table 1: Search results appearances for the ”starbucks gift cards 10” query
Figure 2: RL training reward without length constrains. Only LLM Overview reward.
Figure 3: RL training reward with length reward component (0.2 weight) and LLM Overview reward.
Figure 4: Trained policy evaluation results.
Figure 5: GPT 4.1 mini - randomization robustness.
Figure 6: GPT 4.1 nano - randomization robustness.
Figure 7: GPT 5 mini - randomization robustness.
More in this topic

Adversarial Attacks on LLMs

Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.

See all 355 papers

Explore similar work

CardsList
Jun 15, 2026

How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation

Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a controlled evaluation framework for measuring endorsement corruption in LLM-based web-search agents, combining a web-evidence manipulation pipeline, a five-mode attack taxonomy, and multiple output-level metrics. We evaluate 13 LLM backends on 308 cases each. Results show that vulnerability patterns vary across backends: overall attack success rate (ASR) ranges from 0.0% on Claude-Sonnet-4.6 to 31.4% on Gemini-3-Flash, the strongest attack mode differs by model family, and the same deployment scaffold could amplify or decrease ASR on different backends. An auxiliary agent-skill probe, where endorsement becomes an install command, exposes a sharp split among otherwise robust backends: Claude over-rejects while GPT over-trusts. These findings argue for treating recommendation reliability under adversarial search content as a first-class dimension of backend safety evaluation.

Yimeng Chen, Zhe Ren, Firas Laakom +3

Center of Excellence for Generative AI, KAUST · 2Jilin University · 3Zhejiang University +2

LLM Agent EvaluationAI Agent SecurityLLM SecurityLLM Agent Reliability
AskPDF
Aug 31, 2026

Authority Bias in Conversational Search Engines for Academic Paper Recommendation

Large Language Models (LLMs) are increasingly used as conversational search engines for academic literature, yet whether they judge papers on content or on authority signals has not been tested causally. We investigate authority bias: systematic preference for papers based on author prestige, venue, and citations rather than content. Holding title and abstract constant, we vary authority metadata across three counterfactual conditions (original, flipped, boosted) over eight LLMs (five open-weight and three frontier closed-weight) in an in-context, single-turn, top-1 recommendation setting. Our experiments show that authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt-level debiasing. We further document a say-do gap: debiasing instructions suppress authority mentions far faster than authority-driven flips, so surface auditing systematically underestimates behavioral bias.

Uthman Jinadu, Parsa Ghazvinian, Anjila Budathoki +3

Georgia State University · University of Tennessee, Knoxville

Language Model Bias EvaluationLarge Language Model-Based RecommendationOwnership Bias in Language Models
AskPDF
Mar 18, 2026

Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)

Current Large Language Models (LLMs) are gradually exploited in practically valuable agentic workflows such as Deep Research, E-commerce recommendation, and job recruitment. In these applications, LLMs need to select some optimal solutions from massive candidates, which we term as LLM-as-a-Recommender paradigm. However, the reliability of using LLM agents for recommendations is underexplored. In this work, we introduce a Bias Recommendation Benchmark (BiasRecBench) to highlight the critical vulnerability of such agents to biases in high-value real-world tasks. The benchmark includes three practical domains: paper review, e-commerce, and job recruitment. We construct a Bias Synthesis Pipeline with Calibrated Quality Margins that 1) synthesizes evaluation data by controlling the quality gap between optimal and sub-optimal options to provide a calibrated testbed to elicit the vulnerability to biases; 2) injects contextual biases that are logical and suitable for option contexts. Extensive experiments on both SOTA (Gemini-{2.5,3}-pro, GPT-4o, DeepSeek-R1) and small-scale LLMs reveal that agents frequently succumb to injected biases despite having sufficient reasoning capabilities to identify the ground truth. These findings expose a significant reliability bottleneck in current agentic workflows, calling for specialized alignment strategies for LLM-as-a-Recommender. The complete code and evaluation datasets will be made publicly available shortly.

Zichen Tang, Zirui Zhang, Qian Wang +3

The Hong Kong University of Science and Technology · National University of Singapore · The Hong Kong University of Science and Technology (Guangzhou)

Large Language Model-Based RecommendationLLM Agent ReliabilityLanguage Model Bias Evaluation
AskPDF