Selective Elicitation as a Commercial Influence Channel: A Reproducible Synthetic Shopping-Agent Stress Test

Jiapeng Li

cs.LGSep 29, 2026

Selective Elicitation as a Commercial Influence Channel: A Reproducible Synthetic Shopping-Agent Stress Test

Authors: Jiapeng Li

Abstract

A commercial incentive need not enter the final ranking algorithm to affect a shopping assistant's recommendation: it may instead influence which preference question the assistant asks. We make this distinction experimentally observable in a deliberately small, synthetic setting. Each task has two products, three verified numerical attributes, a price limit, and a private fixed preference vector. An honest simulated user answers one pairwise question. A separate recommender receives the products and this answer but not the sponsorship assignment. We contrast a neutral question, a soft commercial instruction, and an explicitly adversarial instruction to ask about the sponsor's advantage while omitting the rival's advantage. Across 40 held-out sponsorship-assignment cases (20 distinct catalog-preference contexts), the soft instruction changes no selections. The targeted instruction raises sponsored selection by 0.30 and reduces mean synthetic utility by 0.0547 relative to neutral questioning (95% context-bootstrap interval [-0.0828, -0.0291]) for one language-model recommender. A fixed Bayesian recommender shows a similar effect; a second model makes the same choices on all 120 frozen question-answer inputs. A terminal-answer consistency judge rates all 20 sampled targeted answers consistent, although five have synthetic regret above 0.05; a separate question-coverage dimension flags their one-sided elicitation. A robust partial-preference certificate remains valid under the stipulated synthetic utility but certifies only 16 of 40 targeted cases and is not better than asking a neutral question directly. These results establish neither typical behavior under advertising incentives nor effects on actual consumers.

Figures & tables

Question arm vs. neutral	Selector	$\overline{\Delta U}$	$\Delta P(\mathrm{sponsor})$	95% context CI for $\overline{\Delta U}$
Soft commercial	LLM	0.0000	0.00	$[0,0]$
Soft commercial	Grid Bayes	0.0000	0.00	$[0,0]$
Targeted stress test	LLM	$-0.0547$	$+0.30$	$[-0.0828,-0.0291]$
Targeted stress test	Grid Bayes	$-0.0540$	$+0.30$	$[-0.0822,-0.0281]$

Table 1: Forty paired held-out sponsorship-assignment cases. Payoffs are synthetic, not individual human outcomes. The targeted policy is explicitly directed to omit a rival strength; its effect is not a spontaneous response to mild monetization.

Preference Elicitation

Preference elicitation studies how systems learn people’s values and trade-offs from questions, choices, rankings, and feedback.

See all 69 papers

Sep 16, 2026

Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders

Large language models (LLMs) now serve as conversational shopping assistants on platforms that also sell advertising. These AI agents face a conflict of duty. They advise consumers who rely on their judgment, yet are deployed by platforms that benefit when sponsored listings are chosen. Sponsorship disclosures, designed to allow consumers to penalize paid placements, now reach the AI agent rather than the consumer, and the agent's evaluation of them is hidden from the consumer. Drawing on the fiduciary concept of conflict of duty, we argue that an agent's evaluation of a sponsored listing should not depend on which party deployed it. In controlled choice experiments, we manipulate assigned roles in the system prompt to name either a traveler or a booking platform as the agent's principal. Platform delegation significantly attenuates the penalty that agents apply to sponsored listings and weakens the skepticism that disclosure triggers in their reasoning traces. We replicate out findings across LLMs and reasoning depths. A second study decomposes the disclosure label and shows that the divergence between the two delegates widens significantly when the paid placement is attributed to the platform. Stricter terminology ("Sponsored" instead of "Promoted") lowers choice of paid listings but does not close this gap when the platform is named. The findings show that disclosure mandates designed for human consumers cannot by themselves protect consumers in AI-mediated commerce.

Davood Wadi, Yu Ma

Desautels Faculty of Management, McGill University, Montreal, Quebec, Canada

Language Model Bias Evaluation Large Language Model-Based Recommendation AI Agent Governance Agentic Commerce

Ask PDF

Apr 5, 2026

Commercial Persuasion in AI-Mediated Conversations

As Large Language Models (LLMs) become a primary interface between users and the web, companies face growing economic incentives to embed commercial influence into AI-mediated conversations. We present two preregistered experiments (N = 2,012) in which participants selected a book to receive from a large eBook catalog using either a traditional search engine or a conversational LLM agent powered by one of five frontier models. Unbeknownst to participants, a fifth of all products were randomly designated as sponsored and promoted in different ways. We find that LLM-driven persuasion nearly triples the rate at which users select sponsored products compared to traditional search placement (61.2% vs. 22.4%), while the vast majority of participants fail to detect any promotional steering. Explicit "Sponsored" labels do not significantly reduce persuasion, and instructing the model to conceal its intent makes its influence nearly invisible (detection accuracy < 10%). Altogether, our results indicate that conversational AI can covertly redirect consumer choices at scale, and that existing transparency mechanisms may be insufficient to protect users.

Francesco Salvi, Alejandro Cuevas, Manoel Horta Ribeiro

Princeton University

Persuasion in Language Models Agentic Commerce

Ask PDF

Oct 19, 2026

A Cue's Hidden Price: Auditing Seller Influence on AI Shoppers

Large language models are increasingly asked to shop on a user's behalf, which lets seller-controlled content such as badges, endorsements, and framing steer what they recommend. That poses the question at the core of generative engine optimization (GEO): not whether seller content shifts an agent's choice, but how much. The usual gauge of that influence, how often a cue wins the choice, has a blind spot: once a cue is strong enough to be picked almost every time, the choice rate can no longer show how much stronger one cue is than another. We introduce price-equivalent (PE) auditing, which removes this ceiling by pricing the cue in money. In a counterbalanced, position-controlled forced choice, we raise the cued product's price until it is chosen only half the time, and report that premium as the cue's price-equivalent: a formally defined, policy-conditioned parity threshold that makes very different cues comparable. Priced this way, cues that look equally strong at equal price differ by a wide margin, so choice-rate audits substantially under-resolve seller influence. We run the audit over 113,472 trials, seven models across families, and two product categories. To our knowledge, we also run the first price-equivalent audit of a decision-native model, Jev, which outputs a choice probability directly instead of generating text; strikingly, its profile is unlike any text model we tested, resisting most cues yet remaining susceptible to the strongest social-proof claims. We release the stimuli, logs, and code as a benchmark that reports each model's cue-price profile rather than a single score, giving developers and auditors a common, reproducible way to measure how content moves AI shopping agents.

Felix Buecheler, Gerardo Berbeglia, Mingyu Guo

Adelaide University, Adelaide, Australia · Melbourne Business School, Melbourne, Australia

AI Agent Auditing Generative Engine Optimization Persuasion in Language Models Algorithmic Auditing Agentic Commerce

Ask PDF

Abstract

Preference Elicitation

Explore similar work

Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders

Commercial Persuasion in AI-Mediated Conversations

A Cue's Hidden Price: Auditing Seller Influence on AI Shoppers