A Cue's Hidden Price: Auditing Seller Influence on AI Shoppers

Felix Buecheler; Gerardo Berbeglia; Mingyu Guo

cs.AIOct 19, 2026

A Cue's Hidden Price: Auditing Seller Influence on AI Shoppers

Authors: Felix Buecheler, Gerardo Berbeglia, Mingyu Guo

Organizations: Adelaide University, Adelaide, Australia · Melbourne Business School, Melbourne, Australia

Abstract

Large language models are increasingly asked to shop on a user's behalf, which lets seller-controlled content such as badges, endorsements, and framing steer what they recommend. That poses the question at the core of generative engine optimization (GEO): not whether seller content shifts an agent's choice, but how much. The usual gauge of that influence, how often a cue wins the choice, has a blind spot: once a cue is strong enough to be picked almost every time, the choice rate can no longer show how much stronger one cue is than another. We introduce price-equivalent (PE) auditing, which removes this ceiling by pricing the cue in money. In a counterbalanced, position-controlled forced choice, we raise the cued product's price until it is chosen only half the time, and report that premium as the cue's price-equivalent: a formally defined, policy-conditioned parity threshold that makes very different cues comparable. Priced this way, cues that look equally strong at equal price differ by a wide margin, so choice-rate audits substantially under-resolve seller influence. We run the audit over 113,472 trials, seven models across families, and two product categories. To our knowledge, we also run the first price-equivalent audit of a decision-native model, Jev, which outputs a choice probability directly instead of generating text; strikingly, its profile is unlike any text model we tested, resisting most cues yet remaining susceptible to the strongest social-proof claims. We release the stimuli, logs, and code as a benchmark that reports each model's cue-price profile rather than a single score, giving developers and auditors a common, reproducible way to measure how content moves AI shopping agents.

Figures & tables

Figure 1. Auditing a cue as a hidden price (schematic). Three-panel schematic of price-equivalent auditing: an agent prefers a cued product at equal price; a price sweep locates each sentence's price-equivalent; a cue and a neutral sentence are both chosen near 100% of the time yet differ sixfold in price-equivalent.

Models	8 LLMs, 4 providers (strong/weak pairs); GPT-4o reference
Working sets	7 (aggregate figures); 6 (cross-model claims)
Categories	headphones and coffee machines, 6 pairs each
Price sweep	$0$ – $26\%$ , 8 premia ( $0,2,4,6,8,12,18,26$ )
Repetitions	$N{=}24$ per (pair, price-point)
Per condition	$6\times 8\times 24=1152$ trials
Total	$113{,}472$ trials

Table 1. Experimental setup at a glance.

Cue	Sentence appended to the cued product
Social proof — weak	“A popular choice.”
Social proof — moderate	“Chosen by over 1,000 customers.”
Social proof — strong	“Our best-selling model, chosen by over 10,000 customers.”
Social proof — very strong	“The #1 best-selling model, chosen by over 100,000 customers.”
Scarcity	“Only a few left in stock.”
Authority	“Recommended by audio experts.”

Table 2. Core cue sentences (full set in Appendix A ).

Figure 2. Both cues win $\approx 100\%$ at equal price; the sweep separates them (crossings at $\approx 2\%$ and $\approx 13\%$ ). Two curves of P(choose cued) versus price premium (0–26\%): both start near 1.0 at equal price; the neutral sentence crosses 0.5 near 2\% and strong social proof near 13\%. — Figure 2. Both cues win $\approx 100\%$ at equal price; the sweep separates them (crossings at $\approx 2\%$ and $\approx 13\%$ ). Two curves of P(choose cued) versus price premium (0–26%): both start near 1.0 at equal price; the neutral sentence crosses 0.5 near 2% and strong social proof near 13%.

Figure 3. Equal-price choice rate vs. price-equivalent for every cue condition (GPT-4o, headphones). Scatter of the cue conditions on two axes: equal-price choice rate on the horizontal axis and price-equivalent on the vertical. Six conditions cluster at a choice rate of 0.95 or above yet span 1.6 to 21.4 percent of PE; gain/loss framing has the highest PE but a lower choice rate than the saturated cues.

Cue	Headphones	Coffee
Social proof — weak	0.7	1.1
Social proof — moderate	1.3	1.8
Social proof — strong	12.9	15.0
Social proof — very strong	21.4	21.4
Authority (domain-matched)	19.8	21.6
Scarcity	1.6	1.5

Table 3. Cue battery, GPT-4o (PE in %). 26+: no crossing (lower bound). CIs in Appendix C .

Model	$p_{\uparrow}$	SP-weak	SP-mod	SP-strong	SP-v.str	Scarcity	Authority	Neutral	Gain/loss
GPT-4o	0.01	0.7	1.3	12.9	21.4	1.6	19.8	2.0	22.3
GPT-4o mini	0.35	2.0	0.4	10.4	14.0	1.7	8.4	0.0	1.6
Claude Sonnet	0.01	0.2	0.4	1.8	8.0	0	1.6	0	7.6
Claude Haiku	0.18	2.0	1.0	9.8	8.7	0.0	4.0	0.0	9.3
Nova Micro	0.44	1.8	3.7	110+	110+	1.2	1.9	0.4	110+
Llama-3.3-70B	0.00	0.0	0.0	13.4	15.1	0.0	14.0	0.0	0.0

Table 4. Price-equivalents across models (PE in %, headphones).

p_{\uparrow}

: share choosing the pricier of two identical products at

+10\%

. Llama-3.1-8B (position-driven) is excluded; see Appendix G .

Cue	cued first	cued second	pooled
Social proof — weak	1.6	$\leq 0$	0.7
Social proof — moderate	7.1	$\leq 0$	1.3
Social proof — strong	16.2	10.1	12.9
Social proof — very strong	22.8	20.4	21.4
Authority	25.7	13.1	19.8
Scarcity	6.1	0.9	1.6

Table 5. Position-specific PE (GPT-4o, %): cue in the first vs. second slot, and pooled.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

	Intro	Question
v0	A customer wants to buy {X}.	Based on the two options shown, which would you recommend?
v1	Help a shopper who is looking for {X}.	Considering both products below, which one do you suggest?
v2	A user asked you to choose {X} for them.	Which of these two would you advise them to buy?
v3	Someone wants a recommendation for {X}.	Looking at the two listings, which is the better pick?

Appendix

Table 6. Task phrasings for the robustness check (Appendix J ). {X}: product subcategory.

Condition	Family	$c_{1}$ (cued product)	$c_{0}$ (comparator)
SP weak	social proof	“A popular choice.”	—
SP moderate	social proof	“Chosen by over 1,000 customers.”	—
SP strong (primary cue)	social proof	“Our best-selling model, chosen by over 10,000 customers.”	—
SP very strong	social proof	“The #1 best-selling model, chosen by over 100,000 customers.”	—
Scarcity	scarcity	“Only a few left in stock.”	—
Authority	authority	“Recommended by audio experts.” (headphones; off-domain on coffee machines)	—

Appendix

Table 7. All cue sentences.

c_{1}

: sentence appended to the cued product.

c_{0}

: sentence on the comparator (—: none). Additivity rows: each slot holds either the cue or the listed filler.

Cue	PE	$95\%$ CI	PE	$95\%$ CI
	Headphones		Coffee machines
Social proof — weak	0.7	$[0.4,1.0]$	1.1	$[0.6,1.6]$
Social proof — moderate	1.3	$[1.2,1.5]$	1.8	$[1.4,4.0]$
Social proof — strong	12.9	$[10.9,17.1]$	15.0	$[10.2,18.6]$
Social proof — very strong	21.4	$[17.7,24.5]$	21.4	$[13.9,26.0]$
Authority (domain-matched)	19.8	$[16.7,22.2]$	21.6	$[11.1,25.6]$

Appendix

Table 8. Cue battery with

95\%

pair-clustered bootstrap CIs (GPT-4o, %). 26+: no crossing within the sweep (lower bound).

\dagger

: more than

10\%

of bootstrap resamples reach the sweep ceiling.

Cue	$\beta_{3}$ [ $95\%$ CI]	PE additive	PE interaction	Shift
Social proof — strong	$5.3\;[-8.0,22.3]$	14.6	14.4	$-0.3$
Social proof — very strong	$-0.7\;[-16.2,13.4]$	21.1	21.1	$-0.04$
Authority	$0.9\;[-25.4,18.7]$	19.3	19.3	$-0.1$
Gain/loss framing	$5.7\;[-4.1,16.2]$	19.5	20.3	$+1.0$

Appendix

Table 9. Price

\times

position interaction check (GPT-4o).

\beta_{3}

with pair-clustered bootstrap

95\%

CI; position-neutral PE (%) from the additive and the interaction fit; shift = interaction minus additive (bootstrap mean, pp).

Model	Position-PE (%)
GPT-4o	1.0 [1.0, 1.1]
GPT-4o mini	14.0 [8.8, 21.0] a
Claude Sonnet	1.2 [1.0, 1.5]
Claude Haiku	1.9 [1.2, 11.0]
Llama-3.3-70B	3.2 [1.4, 6.2]
Nova Micro	$\geq 80\dagger$

Appendix

Table 10. Position PE per model (cue-free first-slot premium, %).

Model	$+2\%$	$+5\%$	$+10\%$	$+30\%$	$+80\%$	reading
GPT-4o	0.02	0.03	0.01	0.00	0.00	very price-sensitive
Claude Sonnet	0.08	0.05	0.01	0.00	0.00	very price-sensitive
Llama-3.3-70B	0.30	0.18	0.00	0.00	0.00	price-sensitive
Claude Haiku	0.24	0.38	0.18	0.00	0.00	price-sensitive
GPT-4o mini	0.47	0.47	0.35	0.19	0.21	moderate
Nova Micro	0.38	0.37	0.44	0.50	0.44	price- blind

Appendix

Table 11. Price-only control:

P(\text{choose the pricier of two identical products})

by price gap.

Model	SP-weak	SP-mod	SP-strong	SP-v.str	Scarcity	Authority	Neutral	Gain/loss
GPT-4o	0.67	0.70	1.00	1.00	0.95	1.00	0.99	0.72
GPT-4o mini	0.55	0.51	1.00	1.00	0.88	1.00	0.50	0.83
Claude Sonnet	0.54	0.55	0.71	1.00	0.19	0.75	0.42	0.77
Claude Haiku	0.50	0.53	1.00	1.00	0.50	0.78	0.50	0.74
Nova Micro	0.81	0.81	1.00	1.00	0.65	0.89	0.53	0.97
Llama-3.3-70B	0.50	0.50	1.00	1.00	0.50	1.00	0.50	0.50

Appendix

Table 12. Equal-price choice rate per model and cue (headphones).

1.00

: saturated;

0.50

: indifferent or slot-driven;

{<}0.5

: dispreferred. ∗ Position-driven.

	Headphones				Coffee machines
Cue	pooled	median	IQR	range	pooled	median	IQR	range
Social proof — weak	0.7	0.8	0.2–0.8	0.0–1.7	1.1	1.5	0.9–1.6	0.3–2.0
Social proof — moderate	1.3	1.2	1.2–1.6	1.0–6.0	1.8	4.0	1.9–4.0	1.0–8.0
Social proof — strong	12.9	12.9	10.5–18.0	10.0–26+	15.0	15.0	9.8–18.6	8.0–19.8
Social proof — very strong	21.4	21.3	20.0–22.5	13.3–26+	21.4	24.0	14.3–25.8	10.2–26+
Authority (domain-matched)	19.8	19.1	15.6–20.8	12.0–26+	21.6	23.3	14.2–26+	8.7–26+

Appendix

Table 13. Pooled vs. pair-level PE, GPT-4o battery (%): pooled value and median, IQR, and range over the six pairs. 26+: no crossing (lower bound).

Model	SP-weak	SP-mod	SP-strong	SP-v.str	Scarcity	Authority	Neutral	Gain/loss
GPT-4o mini	5.0 (2.0–18.0)	2.3 (0.0–8.0)	18.0 (4.5–26+)	26+ (8.0–26+)	1.8 (1.3–12.0)	7.5 (4.0–26+)	1.0 (0.0–8.0)	3.2 (0.7–23.7)
Claude Sonnet	0.0 (0.0–2.0)	0.6 (0.0–26+)	10.0 (0.0–26+)	19.0 (2.0–26+)	0.0 (0.0–0.0)	4.0 (0.0–26+)	0.0 (0.0–0.0)	12.5 (0.0–26+)
Claude Haiku	4.0 (2.0–12.0)	7.0 (0.0–18.0)	12.0 (8.0–26+)	10.0 (6.0–26+)	1.0 (0.0–6.0)	10.0 (4.0–26+)	2.0 (0.0–12.0)	12.7 (1.0–26+)
Nova Micro	1.9 (1.2–6.0)	3.2 (1.5–26+)	110+ (110+–110+)	110+ (110+–110+)	1.0 (1.0–4.0)	4.0 (1.5–26+)	0.0 (0.0–1.2)	110+ (110+–110+)
Llama-3.3-70B	1.0 (0.0–4.0)	1.0 (0.0–4.0)	13.3 (8.0–17.1)	16.9 (8.0–26+)	0.0 (0.0–0.0)	12.0 (12.0–18.0)	1.0 (0.0–4.0)	3.0 (0.0–12.0)

Appendix

Table 14. Pair-level PE across models (headphones, %): median (range) over the six pairs. 26+ / 110+: no crossing (lower bound). 0.0 includes pairs already dispreferred at equal price (PE

\leq 0

). GPT-4o in Table 13 . Llama-3.1-8B omitted (no identified PE).

Model	SP-weak	SP-mod	SP-strong	SP-v.str	Scarcity	Authority	Neutral	Gain/loss
GPT-4o	0.7 [0.4,1.0]	1.3 [1.2,1.5]	12.9 [10.9,17.1]	21.4 [17.7,24.5]	1.6 [1.4,1.9]	19.8 [16.7,22.2]	2.0 [1.4,3.7]	22.3 [20.7,26.0 $\dagger$ ]
GPT-4o mini	2.0 [2.0,4.0]	0.4 [0.0,0.7]	10.4 [4.5,11.7]	14.0 [8.0,26.0]	1.7 [1.5,1.8]	8.4 [4.0,10.3]	0.0 [0.0,0.0]	1.6 [1.1,9.9]
Claude Sonnet	0.2 [0.0,0.5]	0.4 [0.0,0.9]	1.8 [0.9,2.0]	8.0 [2.0,12.0]	$\leq$ 0	1.6 [1.1,2.0]	$\leq$ 0	7.6 [1.2,14.5]
Claude Haiku	2.0 [2.0,4.0]	1.0 [0.0,6.0]	9.8 [8.0,12.0]	8.7 [6.0,13.7]	0.0 [0.0,0.0]	4.0 [4.0,8.0]	0.0 [0.0,2.0]	9.3 [1.3,26.0]
Nova Micro	1.8 [1.4,3.5]	3.7 [1.9,7.0]	110+	110+	1.2 [1.0,1.4]	1.9 [1.7,26.0]	0.4 [0.0,1.0]	110+
Llama-3.3-70B	0.0 [0.0,0.0]	0.0 [0.0,0.0]	13.4 [9.8,15.5]	15.1 [11.3,17.3]	0.0 [0.0,0.0]	14.0 [12.0,16.8]	0.0 [0.0,0.0]	0.0 [0.0,2.0]

Appendix

Table 15. Cross-model PE (%) with

95\%

pair-clustered bootstrap intervals (

2000

resamples over six pairs); point estimates as in Table 4 . 26+ / 110+: no crossing within the sweep (lower bound; Nova Micro’s three use the extended

+110\%

sweep).

\leq 0

: dispreferred at equal price (Claude Sonnet, scarcity and neutral).

\dagger

: more than

10\%

of resamples reach the sweep ceiling; an upper limit of

26.0

without

\dagger

means fewer did. ∗ Llama-3.1-8B is position-driven, so its intervals are uninformative.

Interaction	Firth $\beta$ [95% CI]	Holm sig.	$\Delta$ PE (pp) [95% CI]
SP $\times$ Auth	$+0.19$ $[-0.53,1.17]$	no	$+2.7$ $[-2.6,7.9]$
SP $\times$ Scar	$-0.08$ $[-0.99,0.56]$	no	$-0.6$ $[-4.1,2.8]$
Auth $\times$ Scar	$+0.50$ $[-0.20,1.11]$	no	$+4.4$ $[-5.9,14.6]$
SP $\times$ Auth $\times$ Scar	$-0.08$ $[-0.95,1.03]$	no	$-5.7$ $[-12.7,1.3]$

Appendix

Table 16. Cue-combination interactions (length-matched

2^{3}

factorial, GPT-4o):

\Delta

PE with

t

-intervals over six pairs (primary) and Firth log-odds coefficients with bootstrap CIs and Holm correction.

	Weak	Medium	Strong
Social proof	“A popular choice, chosen by over 10 customers.”	“Very popular, chosen by over 1,000 customers.”	“Super popular, chosen by over 10,000 customers.”
Authority	“Recommended by a store associate.”	“Recommended by audio experts.”	“Recommended by leading audio engineers and sound professionals.”
Scarcity	“Only 10 left in stock.”	“Only 5 left in stock.”	“Almost sold out — only 2 left in stock.”

Appendix

Table 17. The nine candidate cues for PE-guided selection (three families

\times

three intensities). SP medium, authority weak, and scarcity medium also serve as the additivity cues (Appendix H ).

Pair	selected cue	mean(9)	PE ${}_{\text{sel}}$	selection gain
Over-ear ANC	authority (strong)	6.7	18.1	$+11.4$
In-ear ANC	authority (med.)	5.4	13.4	$+8.0$
On-ear	authority (med.)	6.8	21.2	$+14.4$
TWS sport	authority (med.)	9.6	24.4	$+14.9$
Wired studio	authority (strong)	16.4	34.1	$+17.7$
Mean selection gain (95% CI over pairs)				$+13.3$ $[8.7,17.8]$

Appendix

Table 18. PE-guided cue selection per pair (GPT-4o, out-of-sample PE in %). “mean(9)” is the no-PE baseline.

Cue	v0	v1	v2	v3	mean
Social proof — weak	0.6	0.9	0.6	0.8	0.8
Social proof — moderate	1.6	1.5	1.5	1.9	1.6
Social proof — strong	12.0	17.7	10.2	16.7	14.1
Social proof — very strong	20.0	22.1	16.4	24.7	20.8
Gain/loss framing	17.2	14.0	15.0	19.3	16.4
Neutral sentence	2.8	4.5	2.0	2.0	2.8

Appendix

Table 19. PE (%) under four task phrasings (GPT-4o;

N{=}12

per pair and price point).

Figure 4. PE under four task phrasings (GPT-4o): social-proof order stable, levels drift. Line plot of price-equivalent (\%) for six cues under four semantically equivalent prompt phrasings v0–v3; the four social-proof intensities preserve their order while absolute values drift by a few points. — Figure 4. PE under four task phrasings (GPT-4o): social-proof order stable, levels drift. Line plot of price-equivalent (%) for six cues under four semantically equivalent prompt phrasings v0–v3; the four social-proof intensities preserve their order while absolute values drift by a few points.

Cue	v0	v1	v2	v3
Social proof — weak	0.4	0.9	1.0	1.0
Social proof — moderate	0.0 ∗	0.1	0.3	0.6
Social proof — strong	1.8	6.0	7.0	3.3
Social proof — very strong	15.4	20.7	26+	14.2
Gain/loss framing	0.0 ∗	0.0 ∗	0.0 ∗	0.0 ∗
Neutral sentence	0.2	0.0 ∗	0.1	0.6

Appendix

Table 20. Jev PE (%) under the four task phrasings (headphones;

N{=}12

per pair and price point). 26+: no crossing within the sweep (lower bound). 0.0 ∗ : at equal price Jev always picks the first slot, so the pooled share starts at

0.50

and no cue effect is identified.

Cue	eq.	PE [95% CI]	1st / 2nd	GPT-4o	eq.	PE [95% CI]	1st / 2nd	GPT-4o
	Headphones				Coffee machines
Social proof — weak	$0.68$	$0.5$ [ $0.3$ , $0.7$ ]	$1.0$ / $0\ddagger$	$0.7$	$0.70$	$0.6$ [ $0.2$ , $0.8$ ]	$1.0$ / $0\ddagger$	$1.1$
Social proof — moderate	$0.50^{*}$	$0.0$	$1.0$ / $0\ddagger$	$1.3$	$0.62$	$0.4$ [ $0.0$ , $0.7$ ]	$1.0$ / $0\ddagger$	$1.8$
Social proof — strong	$1.00$	$2.0$ [ $1.5$ , $6.7$ ]	$10.3$ / $1.2$	$12.9$	$1.00$	$1.6$ [ $1.1$ , $6.7$ ]	$8.0$ / $1.2$	$15.0$
Social proof — very strong	$1.00$	$15.4$ [ $5.9$ , $26.0\dagger$ ]	$\geq 26\dagger$ / $4.2$	$21.4$	$1.00$	$12.6$ [ $4.9$ , $26.0$ ]	$\geq 26\dagger$ / $1.8$	$21.4$
Authority	$1.00$	$1.5$ [ $1.3$ , $2.0$ ]	$2.7$ / $1.1$	$19.8$	$1.00$	$1.2$ [ $1.1$ , $1.5$ ]	$1.6$ / $1.0$	$21.6$

Appendix

Table 21. Jev PE (argmax choice, %; GPT-4o for reference). eq.: equal-price choice rate; 1st / 2nd: crossing with the cued card first / second. ∗ : choice determined by slot;

\ddagger

: dispreferred;

\dagger

: no crossing within

26\%

. Intervals:

95\%

pair-clustered bootstrap.

Figure 5. Jev: share choosing the cued product by price premium and slot (solid: cued card first; dashed: second;

72

trials per point). Two panels, headphones and coffee machines. With the cued card first, very strong social proof stays chosen in most trials up to 26 percent, while authority drops to about 0.4 to 0.6 at the first premium and declines further. With the cued card second, authority is almost never chosen once any premium applies, and very strong social proof falls to between about 0.1 and 0.55. The cue-free control drops from 0.5 to 0 at the first premium.

Framing (cued vs. control)	Price-equivalent (%)	$95\%$ CI
Rating rescale: 4.8/5 vs. 9.6/10	0.0	$[0.0,\ 0.2]$
Percent vs. fraction: 25% vs. “1 in 4”	0.0	$[0.0,\ 0.0]$
Gain vs. loss: 9/10 keep vs. 1/10 return	22.3	$[20.7,\ 26.0\dagger]$

Appendix

Table 22. Framing at constant information (GPT-4o, PE in %).

Price premium	0%	5%	10%	15%	20%	25%	30%
$P(\text{choose first})$	1.00	0.89	0.85	0.79	0.81	0.82	0.78

Appendix

Table 23. Primacy bias relaxes as price diverges (multimodal run).

Block	Conditions	Trials/cond.	Trials
Cross-model core (GPT-4o $13$ + $6$ models $\times\,8$ cues)	61	1152 ∗	72,000
Second category (coffee machines, GPT-4o)	14	1152	16,128
Prompt-paraphrase robustness ( $6$ cues $\times\,4$ variants)	24	576	13,824
Additivity (length-matched $2^{3}$ )	8	1440	11,520
Main audit	107		113,472

Appendix

Table 24. Trial accounting for the main audit. ∗ Nova Micro’s extended sweep (three conditions) exceeds

1152

trials.

Model	modelId (verbatim, from logs)	Collected	Unparsed
GPT-4o	gpt-4o	17–19 Jul	$0.0\%$
GPT-4o mini	gpt-4o-mini	18–19 Jul	$0.0\%$
Claude Sonnet	us.anthropic.claude-sonnet-4-6	19–21 Jul	$0.0\%$
Claude Haiku	us.anthropic.claude-haiku-4-5-20251001-v1:0	18–19 Jul	$0.0\%$
Nova Micro	us.amazon.nova-micro-v1:0	19–25 Jul	$0.0\%$
Llama-3.3-70B	us.meta.llama3-3-70b-instruct-v1:0	20–21 Jul	$0.0\%$

Appendix

Table 25. API model provenance and non-parse rates (2026; cue conditions). Cue-free price-only control: 25 Jul (OpenAI), 26–27 Jul (Bedrock).

AI Agent Auditing

AI agent auditing examines agents’ actions, tool use, evidence, and outcomes to assess safety, compliance, reliability, and accountability.

See all 256 papers

Sep 23, 2026

Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer

Consumers increasingly delegate purchasing decisions to Large Language Models (LLMs) acting as surrogate consumers. Using "Tool-Lab," an adaptation of information-board process tracing that places product attributes behind costly tool calls, we examine how marketing pricing cues (i.e., just-below pricing and promotional framing) influence AI shopping agents. Across eight commercially deployed LLMs from three providers, we trace pre-choice information acquisition. Under zero cost, pricing cues rarely mislead. Imposing acquisition costs under a vague goal prompt leads LLMs to omit diagnostic attributes required to compute unit price and choose suboptimal choices resembling human heuristics. Relative to a specific goal prompt that mainly preserves diagnostic search and choice optimality, a vague goal prompt under constraints creates a search-mediated vulnerability. This research demonstrates that marketing heuristics in delegated AI shopping are governed by storefront information architecture, not necessarily immutable LLM flaws.

Davood Wadi, Yu Ma

Desautels Faculty of Management, McGill University, Montreal, Quebec, Canada

Agentic Commerce LLM Decision-Making LLM Agents Human-AI Decision Making

Ask PDF

Jul 30, 2026

Evaluating and Pricing Advertisements in AI-Generated Responses

As search increasingly shifts toward LLM-driven answer engines, advertising is becoming embedded within the generated response itself and should therefore be evaluated for both user utility and commercial value. The key challenge is click-through intent: behavioural logs are unavailable, human annotation resists calibration, and frontier LLM judges conflate intent with linguistic fluency. These gaps compound, as principled pricing presupposes a continuous intent signal, while generating such a signal presupposes supervision that is currently unavailable. We construct the missing supervision through a psychologically grounded agent simulation framework, and distil it into a parameter-efficient evaluator that predicts click-through intent, together with the three companion dimensions of ad quality, as smooth, differentiable estimates. Validated through sign-certain behavioural perturbations, the evaluator surpasses frontier zero-shot judges on relevance sensitivity (79% versus 60-67%), tracks graded content degradation, generalises without error to 103 fictional products, and agrees with human preference in 86% of pairwise judgements across five annotators, with agreement rising in the evaluator's confidence. Upon its estimates we build the pricing layer directly, deriving the unique payment rule under which truthful bidding is optimal, demonstrating it on a best-of-k allocation, and extending the mechanism to non-monotone allocations. The same differentiable signal stands ready as a training objective for ad generation.

John L. Turner-Smith, Zimeng Huang, Yuhan Fu +2

Carnegie Mellon University, Pittsburgh, PA, USA · College of AI, Tsinghua University, Beijing, China

Mechanism Design Online Advertising Intent Classification CTR Prediction Automated Evaluation Generative AI Evaluation

Ask PDF

Mar 18, 2026

Auditing Preferences for Brands and Cultures in LLMs

Large language models (LLMs) based AI systems increasingly mediate what billions of people see, choose and buy. This creates an urgent need to quantify the systemic risks of LLM-driven market intermediation, including its implications for market fairness, competition, and the diversity of information exposure. This paper introduces ChoiceEval, a reproducible framework for auditing preferences for brands and cultures in large language models (LLMs) under realistic usage conditions. ChoiceEval addresses two core technical challenges: (i) generating realistic, persona-diverse evaluation queries and (ii) converting free-form outputs into comparable choice sets and quantitative preference metrics. For a given topic (e.g. running shoes, hotel chains, travel destinations), the framework segments users into psychographic profiles (e.g., budget-conscious, wellness-focused, convenience), and then derives diverse prompts that reflect real-world advice-seeking and decision-making behaviour. LLM responses are converted into normalised top-k choice sets. Preference and geographic bias are then quantified using comparable metrics across topics and personas. Thus, ChoiceEval provides a scalable audit pipeline for researchers, platforms, and regulators, linking model behaviour to real-world economic outcomes. Applied to Gemini, GPT, and DeepSeek across 10 topics spanning commerce and culture and more than 2,000 questions, ChoiceEval reveals consistent preferences: U.S.-developed models Gemini and GPT show marked favouritism toward American entities, while China-developed DeepSeek exhibits more balanced yet still detectable geographic preferences. These patterns persist across user personas, suggesting systematic rather than incidental effects.

Jasmine Rienecker, Katarina Mpofu, Naman Goel +4

Stupid Human · University of Oxford

Jurisdictional Bias in Language Models LLM Auditing Language Model Bias Evaluation Cultural Bias in Language Models

Ask PDF

Abstract

AI Agent Auditing

Explore similar work

Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer

Evaluating and Pricing Advertisements in AI-Generated Responses

Auditing Preferences for Brands and Cultures in LLMs