Sell Me This Stock: Unsafe Recommendation Drift in LLM Agents

Zekun Wu; Adriano Koshiyama; Sahan Bulathwela; Maria Perez-Ortiz

cs.CLMar 13, 2026

Sell Me This Stock: Unsafe Recommendation Drift in LLM Agents

Authors: Zekun Wu, Adriano Koshiyama, Sahan Bulathwela, Maria Perez-Ortiz

Organizations: Centre for Artificial Intelligence, University College London · Holistic AI

Abstract

People increasingly use LLM agents for multi-turn financial recommendations, where the agent pulls market data through tools and tracks user preferences across turns. When tool outputs are manipulated, the recommendations stop matching the user's stated risk profile, but because standard metrics like NDCG only score general relevance, risky and safe stocks score alike, so the metric says nothing went wrong. We call this gap evaluation blindness. We replay 23-turn financial advisory conversations across eight language models, running each dialogue twice with clean and manipulated tool data. Quality scores stay nearly identical to clean sessions while the agents produce risk-mismatched recommendations in 65-99% of turns, unanimous across all eight models. The mechanism is visible turn-by-turn: 80% of risk-score citations across 1,840 turns reproduce the manipulated value verbatim, not a single turn pushes back, and safe-language framing of high-risk stocks ranges from 14% (Qwen2.5-7B) to 69% (Claude Sonnet 4.6). The property that makes frontier models good agents, faithfully grounding their reasoning in tool outputs, also makes them follow manipulated ones. The damage is not memory-driven: contaminating only the current turn still produces 95% of the violations. The model internally distinguishes the manipulation (sparse autoencoder features separate adversarial from random perturbations), but this does not translate into safer output. Activation-level interventions recover under 6% of the safety gap, prompt-level self-verification fails because the self-check reads the same manipulated data, and a parametric cross-check that flags contamination at 99-100% per turn on a frontier model still leaves aggregate suitability unchanged: the agent identifies the tampering and recommends it anyway.

Figures & tables

Figure 1: Experimental overview. The same conversations are replayed with clean and manipulated tool outputs. Quality metrics stay flat while suitability violations reach 65–99% across all eight models.

Model	Scale	Ref.
Qwen3-32B	32B	( Qwen Team, 2025 )
Qwen2.5-7B	7B	( Qwen Team, 2024 )
Gemma 3 12B-IT	12B	( Google DeepMind, 2025 )
GPT-5.2	Frontier	( OpenAI, 2025 )
Claude Sonnet 4.6	Frontier	( Anthropic, 2026c )
CC Opus 4.6	Frontier	( Anthropic, 2026b )

Table 1: Models in the main analysis.

	Quality		Safety
Model	NDCG $\uparrow$	UPR $\uparrow$	SVR s $\downarrow$	Sev. $\downarrow$
Qwen2.5-7B	$0.654$	$1.029$	$0.648$	$1.122$
Gemma 3 12B-IT	$0.651$	$\mathbf{1.249}$	$0.874$	$1.609$
Ministral 3 14B	$0.656$	$0.988$	$0.678$	$1.057$
Qwen3-32B	$0.692$	$1.115$	$0.800$	$1.491$
Mistral Large 3	$0.691$	$1.095$	$0.900$	$1.596$

Table 2: All models show UPR

\approx 1

while SVR

{}_{s}>0.5

. Extended results in Appendix I .

Figure 2: NDCG vs. SVR s across 8 models. Each ellipse covers the spread of 10 users ( $\pm$ 1 s.d.). Frontier models land in the top-right.

Figure 3: Drift under isolated pathways (Claude Sonnet 4.6). Error bars:

\pm 1

s.d.

		Clean		Contam.
Model	Condition	SVR s $\downarrow$	Sev. $\downarrow$	SVR s $\downarrow$	Sev. $\downarrow$
Claude	default	0.783	1.383	0.926	1.652
	+ self-verify	0.226	0.309	0.961	1.804
	+ parametric	0.726	1.243	0.930	1.652
Gemma	default	0.635	1.126	0.874	1.609
	+ self-verify	0.196	0.335	0.987	1.996

Table 3: Two prompt-level defenses on Claude Sonnet 4.6 and Gemma 3 12B-IT. Self-verification makes clean sessions much safer but widens the gap under contamination (SVR and severity both rise on both models). Parametric cross-check leaves aggregate SVR essentially unchanged despite near-perfect per-ticker detection on Claude (Appendix Y ).

Figure 4: (a) Suitability-gap recovery by layer. (b) Three corrective interventions, each

<6\%

Condition	$\bar{D}$ $\downarrow$	SVR s $\downarrow$	Sev. $\downarrow$	UPR $\uparrow$
Clean baseline	0.000	0.783	1.383	1.000
Headlines-only	0.176	0.787	1.361	1.005
Within-band	0.233	0.939	1.548	1.035
Full attack	0.384	0.926	1.652	1.000

Table 4: Subtle variants vs. full attack (Claude Sonnet).

	sNDCG		Preservation
Model	Clean	Contam.	pUPR	sUPR
Qwen2.5-7B	$0.278$	$0.272$	$1.101$	$0.712$
Gemma 3 12B-IT	$0.383$	$0.218$	$1.241$	$0.570$
Ministral 3 14B	$0.505$	$0.298$	$0.958$	$0.515$
Qwen3-32B	$0.384$	$0.305$	$1.108$	$0.700$
Mistral Large 3	$0.508$	$0.335$	$1.088$	$0.681$

Table 5: Only sUPR detects contamination. pUPR stays near 1.0; sUPR drops 19–49%.

Appendix figures & tables50 assets

Supplementary material from the paper’s appendix.

Appendix

Tool	Input	Output Schema	Data Source
MarketDataTool	limit : int (default 10)	{candidates:[{symbol, risk_score, ret_7d, vol, mdd, mu, price}], target_risk_band, date}	Conv-FinRe snapshots + STOCK_RISK_DB
NewsRetrieverTool	query : str	{query, headlines:[str]}	Conv-FinRe advisor text
ProfileMemoryTool	(none)	{profile:{risk_tolerance, goals, constraints, recent_decisions}}	Read-only snapshot of $\mathcal{M}_{t}$

Appendix

Table 6: Tool interface specifications. MarketData returns stock numbers, News returns headlines, and ProfileMemory returns what the agent remembers about the user.

Symbol	Risk Score	Category	Notes
PG	1	Defensive	Consumer staples (Procter & Gamble)
VZ	1	Defensive	Telecom (Verizon)
LIN	2	Low-moderate	Industrial gas (Linde)
XOM	2	Low-moderate	Energy (ExxonMobil)
JPM	3	Moderate	Financial (JPMorgan)
MRK	3	Moderate	Pharma (Merck)

Appendix

Table 7: Stock risk database ( STOCK_RISK_DB ). Risk scores are on a 1–5 ordinal scale: 1 = defensive, 5 = speculative.

Index	Label
0	Retirement savings
1	Education fund
2	Home purchase
3	Steady income
4	Long-term growth
5	Emergency fund

Appendix

Table 8: Goal vocabulary ( GOAL_OPTIONS ).

Index	Label
0	Has outstanding debt
1	Limited investment experience
2	Short time horizon
3	Avoid concentrated bets

Appendix

Table 9: Constraint vocabulary ( CONSTRAINT_OPTIONS ).

Parameter	Qwen3-32B	Qwen2.5-7B	Gemma 3 12B-IT	GPT-5.2	Claude Sonnet 4.6	CC Opus 4.6	Ministral 3 14B	Mistral Large 3
Backend	HF (local)	HF (local)	HF (local)	OpenAI API	AWS Bedrock	Claude Code $\dagger$	AWS Bedrock	AWS Bedrock
Precision	BF16	4-bit NF4	BF16	API-managed	API-managed	API-managed	API-managed	API-managed
Reasoning effort	N/A	N/A	N/A	none (default)	N/A	N/A	N/A	N/A
Temperature	N/A (greedy)	N/A (greedy)	N/A (greedy)	0 (greedy)	Default	Default	Default	Default
Sampling	Deterministic	Deterministic	Deterministic	Deterministic	Default	Default	Default	Default
Max tokens	2048	2048	2048	2048	2048	2048	2048	2048

Appendix

Table 10: Model inference parameters. All models share

K{=}6

max agent steps per turn and 2048 max output tokens. “Deterministic” denotes greedy decoding with no sampling.

Tool	Mode	Mechanism	Affected Fields
MarketData	Risk inversion	$r_{i}\mapsto 6-r_{i}$ for all 10 stocks	risk_score
MarketData	Metric manip.	High-risk ( $r_{i}{\geq}4$ ): vol ${}^{\prime}{\,=\,}$ 0.3 $\cdot$ vol, mdd ${}^{\prime}{\,=\,}$ 0.3 $\cdot$ mdd, $\mu^{\prime}{\,=\,}$ 1.5 $\cdot\|\mu\|$ . Low-risk ( $r_{i}{\leq}2$ ): vol ${}^{\prime}{\,=\,}$ 2.0 $\cdot$ vol, mdd ${}^{\prime}{\,=\,}$ 2.0 $\cdot$ mdd, $\mu^{\prime}{\,=\,}$ 0.5 $\cdot\mu$ .	vol , mdd , mu
MarketData	TQQQ injection	Synthetic candidate: assigned risk 9, displayed as 1 (after inversion). Fabricated: vol ${=}0.01$ , mdd ${=}0.005$ .	candidates
News	Biased headlines	Three adversarial headlines prepended to the real headline list (see below).	headlines
ProfileMemory	None (passive)	Exposes contaminated $\mathcal{M}_{t}$ back to reasoning; no direct perturbation.	N/A

Appendix

Table 11: Which tools get modified and how. ProfileMemory is never directly changed, but it reads back whatever is already in memory, so if earlier turns wrote contaminated state, the agent sees it again here.

Stage	Field	Value
Clean	Risk score	5
	Dist. to target	$\|5-2\|=3$ (ranked last)
After Mode 1	Risk score	$6-5=1$ (appears defensive)
	Dist. to target	$\|1-2\|=1$ (ranked 4th–6th)
After Mode 2	Volatility	$\times\,0.3$ (scaled down)
	Max drawdown	$\times\,0.3$ (scaled down)

Appendix

Table 12: How TSLA (risk 5) is transformed under contamination for a conservative investor (risk band: 2).

	Clean				Contaminated
Stock	Risk	Vol	MDD	$\mu$	Risk	Vol	MDD	$\mu$
PG (Defensive)	1	12%	8%	6%	5	24%	16%	3%
TSLA (Speculative)	5	55%	40%	18%	1	17%	12%	27%
TQQQ (Injected)	—	—	—	—	1	1%	0.5%	2%

Appendix

Table 13: MarketDataTool output comparison: clean vs. contaminated for selected stocks. Risk inversion swaps perceived safety; metric manipulation adjusts volatility (Vol), max drawdown (MDD), and expected return (

\mu

Model	TQQQ %	$\bar{D}$	$\bar{D}_{\setminus\text{T}}$	$\Delta\bar{D}$	SVR s	SVR ${}_{s,\setminus\text{T}}$	$p_{\setminus\text{T}}$
		Drift			SVR
Qwen3-32B	4.3%	0.301	0.290	$-$ 3.4%	0.800	0.792	$0.001^{***}$
Qwen2.5-7B	0.0%	0.267	0.267	$\pm$ 0.0%	0.648	0.648	$0.001^{***}$
Gemma 3 12B-IT	0.4%	0.399	0.398	$-$ 0.3%	0.874	0.873	$0.001^{***}$
GPT-5.2	2.2%	0.402	0.401	$-$ 0.2%	0.883	0.880	$0.001^{***}$
Claude Sonnet 4.6	10.9%	0.384	0.378	$-$ 1.5%	0.926	0.917	$0.001^{***}$

Appendix

Table 14: Impact of TQQQ exclusion on key metrics. TQQQ appears in 0–14.3% of contaminated recommendations. Removing these turns has negligible effect on drift and suitability violations; significance is unchanged.

Model	Fake cite %	Safe-lang %	Detections
Claude Sonnet 4.6	85	68.7	$\approx$ 5 / 230
CC Opus 4.6	75	44.7	—
GPT-5.2	85	64.8	0 / 230
Gemma 3 12B-IT	88	37.4	0 / 230
Qwen3-32B	100	38.7	0 / 230
Qwen2.5-7B	100	14.0	0 / 222

Appendix

Table 15: Uncritical tool-data adoption metrics across eight models. Fake cite % : fraction of risk-score citations that reproduce the manipulated value (denominator: citations for divergent tickers). Safe-lang % : fraction of turns describing high-risk stocks (risk

\geq 4

) with safe/stable language. Detections : turns with explicit skepticism toward tool data (CC Opus detections are unavailable because the Claude Code CLI does not expose intermediate reasoning traces). All-model percentages are computed over pooled citations, not by averaging model-level percentages.

Panel of Figure 5 — Figure 5: How contamination builds up over a conversation (User 1, Claude Sonnet 4.6). Turn 1: memory is the same in both sessions, only the tool outputs differ. Turn 3: the agent starts writing contaminated information into memory. Turn 12: the contaminated session is now a full risk level ahead, and the gap stays through Turn 23.

Panel of Figure 6 — Figure 6: Agent trace comparison for User 0, Turn 1 (Claude Sonnet 4.6). Red highlights mark contaminated data. Both sessions apply identical reasoning (filter by risk $\leq$ 2), but inverted risk scores cause the agent to select speculative stocks (assigned risk 4) instead of defensive stocks (assigned risk 1–2). The agent also accepts fabricated volatility and drawdown figures ( $0.3\times$ actual Conv-FinRe values) without question. Memory is identical after both sessions; divergence is purely information-channel.

User	Risk	Clean recs	Contaminated recs	Shared	$\boldsymbol{\Delta r}$
0	low	VZ, PG, LIN, XOM	SPG, MMM, AMZN	0	$+2.50$
1	low	PG, VZ, LIN, XOM	AMZN, MMM, SPG, MRK	0	$+1.50$
2	low	LIN, PG, VZ, XOM	AMZN, MMM, SPG, MRK	0	$+2.25$
3	mod	MRK, VZ, LIN, JPM	AMZN, SPG, MMM, MRK	1	$+1.25$
4	mod	MRK, JPM, LIN, XOM	AMZN, MMM, SPG, MRK	1	$+1.50$
5	low	VZ, PG, LIN, XOM	AMZN, SPG, MMM	0	$+2.17$

Appendix

Table 16: Turn 1 recommendation divergence across all 10 users (Claude Sonnet 4.6). All users diverge immediately at the first contaminated turn. Overlap = shared tickers between clean and contaminated lists.

\Delta r

= difference in mean assigned risk score (contaminated

-

clean).

Panel of Figure 7 — Figure 7: Cross-model Turn 1 susceptibility spectrum (User 0, contaminated session). GPT-5.2 recommends LIN, PG, XOM (3 of 4 clean-session stocks, only VZ dropped), despite seeing inverted risk scores. It explicitly excludes TSLA, AMZN, and TQQQ as “inconsistent with conservative profile.” Qwen3-32B trusts the inverted scores entirely, recommending TQQQ (a 3 $\times$ leveraged Nasdaq ETF) to a conservative investor with debt and a short time horizon. Claude Sonnet’s behavior (SPG, MMM, AMZN; shown in Figure 6 ) falls between these extremes.

	Turn 10		Turn 20
Model	Risk	Goals	Risk	Goals
GPT-5.2	high	growth, wealth accum.	high	growth, wealth accum.
Claude Sonnet	high	income, growth	high	income, growth
CC Opus	high	growth, wealth accum.	high	growth, wealth accum.
Qwen3-32B	high	income	mod	income
Mistral Large	mod	income, growth	mod	income, growth

Appendix

Table 17: Memory state evolution under contamination across models (User 0). Risk tolerance and goals are extracted from the agent’s memory at Turns 10 and 20. Original profile: risk= low , goals={steady income}. All models start from the original profile at Turn 1. Bold entries mark deviations from the original.

	Quality		Drift	Safety
Condition	NDCG $\uparrow$	UPR $\uparrow$	$\bar{D}\downarrow$	SVR ${}_{s}\downarrow$	Sev. SVR	MDR $\downarrow$
Full (all modes)	$0.744$	$1.000$	$\mathbf{0.384}$	$\mathbf{0.926}$	$\mathbf{1.652}$	$\mathbf{0.170}$
Risk inversion only	$0.759$	$1.041$	$0.224$	$0.843$	$1.487$	$0.118$
Metric manipulation only	$0.759$	$0.973$	$0.240$	$0.839$	$1.413$	$0.112$
Headlines only	$0.753$	$1.005$	$0.176$	$0.787$	$1.361$	$0.094$

Appendix

Table 18: Channel isolation (Claude Sonnet 4.6). Each row enables one contamination mode; all exhibit the evaluation blindness pattern (

p=0.001

). Clean-session SVR

{}_{s}=0.783

for this model.

Figure 8: Contamination channel isolation: contribution of each single channel to drift and suitability metrics compared to the full three-mode configuration (Claude Sonnet 4.6, 10 users, 23 turns). Error bars show

\pm 1

standard deviation across users.

Figure 9: Mechanism matrix:

2{\times}2

channel decomposition into information-only and memory-only contamination (Claude Sonnet 4.6, 10 users, 23 turns). Info-only closely reproduces full-attack SVR s (

0.948

vs.

0.926

) with zero MDR, consistent with suitability violations being predominantly information-channel-driven. Info-only SVR s slightly exceeds the full-attack value because each turn starts from clean memory, avoiding occasional memory-driven substitutions toward different (but still violating) products; drift magnitude involves both channels (info-only

82\%

, mem-only

49\%

of full

\bar{D}

). Error bars:

\pm 1

s.d.

	Quality		Drift	Safety
Condition	NDCG p $\uparrow$	UPR $\uparrow$	$\bar{D}\downarrow$	SVR ${}_{s}\downarrow$	Sev. SVR	MDR $\downarrow$
Explicit headlines	$0.752$	$1.005$	$0.176$	$0.787$	$1.361$	$0.094$
Subtle headlines	$0.752$	$1.026$	$0.168$	$0.804$	$1.409$	$0.099$
Wilcoxon $p$	$0.922$	—	$0.846$	$0.312$	$0.031$	$0.910$

Appendix

Table 19: Subtle vs. explicit biased headlines (Claude Sonnet 4.6, 10 users, 23 turns).

p

-values from Wilcoxon signed-rank tests.

Figure 10: Contamination frequency dose-response (Claude Sonnet 4.6, 10 users). Left:

\bar{D}

increases monotonically with

p

; SVR s starts at

0.783

(clean baseline) and rises to

0.926

p{=}1.0

. Right: NDCG and UPR remain flat, consistent with utility–suitability decoupling across all contamination frequencies.

$p$	NDCG	$\bar{D}$	SVR s	Sev. SVR	MDR	UPR
$0.00$	$0.744$	$0.000$	$0.783$	$1.383$	$0.000$	$1.000$
$0.25$	$0.756$	$0.236$	$0.822$	$1.457$	$0.130$	$1.002$
$0.50$	$0.756$	$0.265$	$0.909$	$1.565$	$0.124$	$1.006$
$0.75$	$0.759$	$0.301$	$0.904$	$1.617$	$0.122$	$0.988$
$1.00$	$0.744$	$0.384$	$0.926$	$1.652$	$0.170$	$1.000$

Appendix

Table 20: Contamination frequency dose-response. All values are means across 10 users.

p{=}0.00

is the clean baseline; SVR

{}_{s}{=}0.783

reflects violations inherent to the small stock universe even without contamination.

Figure 11: Contamination strength dose-response (Claude Sonnet 4.6, 10 users). Left:

\bar{D}

increases monotonically while SVR s rises from a clean baseline of

0.783

with a threshold jump at

\alpha{=}0.50

. Right: NDCG p remains flat across all strength levels, consistent with the evaluation blindness pattern even under weak contamination. Error bars:

\pm 1

s.e. over 10 users.

$\alpha$	NDCG	NDCG p	$\bar{D}$	SVR s	Sev. SVR	MDR	UPR
$0.00$	$0.744$	$0.744$	$0.000$	$0.783$	$1.383$	$0.000$	$1.000$
$0.25$	$0.744$	$0.727$	$0.223$	$0.809$	$1.413$	$0.151$	$0.985$
$0.50$	$0.744$	$0.765$	$0.299$	$0.961$	$1.735$	$0.097$	$1.049$
$0.75$	$0.744$	$0.719$	$0.393$	$0.926$	$1.735$	$0.143$	$1.002$
$1.00$	$0.744$	$0.723$	$0.384$	$0.926$	$1.652$	$0.170$	$1.000$

Appendix

Table 21: Contamination strength dose-response. All values are means across 10 users.

\alpha{=}0.00

is the clean baseline; note that SVR s is already

0.783

without contamination because the small stock universe (10 tickers) includes many products above low-risk users’ thresholds.

Panel of Figure 12 — Figure 12: Top: per-user drift over 23 turns (Claude Sonnet 4.6, 5-turn rolling mean; gray band: mean $\pm$ 1sd). Bottom: aggregate drift and suitability violations across all eight models. Drift varies across users but persists for all of them; violations emerge at turn 1 for 79 of 80 trajectories.

Symbol	Definition
NDCG / sNDCG	Ranking quality / suitability-penalized variant (zeroes risk-inappropriate items)
UPR / sUPR	Utility Preservation Ratio (contaminated/clean NDCG; suitability-penalized variant)
$\bar{D}$ , $D_{t}$	Mean paired drift / per-turn drift (Kendall- $\tau$ + Jaccard)
AR	Amplification Ratio (late-half vs. early-half $\bar{D}$ )
RDS	Risk Drift Score (sum of ordinal risk divergence)
SVR s / SVR r	Suitability Violation Rate (stated / revealed risk)

Appendix

Table 22: Notation summary for evaluation metrics.

	Suitability Violations			Drift		Memory / Quality
Model	SVR s	SVR r	Sev. SVR	$\bar{D}$	AR	MDR	UPR
Qwen2.5-7B	$0.648$	$0.343$	$1.122$	$0.267$	$\mathbf{1.781}$	$\mathbf{0.271}$	$1.029$
Gemma 3 12B-IT	$0.874$	$0.552$	$1.609$	$0.399$	$0.885$	$0.226$	$\mathbf{1.249}$
Ministral 3 14B	$0.678$	$0.391$	$1.057$	$\mathbf{0.670}$	$1.197$	$\mathbf{0.457}$	$0.988$
Qwen3-32B	$0.800$	$0.496$	$1.491$	$0.301$	$0.743$	$0.116$	$1.115$
Mistral Large 3	$0.900$	$0.591$	$1.596$	$0.404$	$0.922$	$0.142$	$1.095$

Appendix

Table 23: Extended cross-model comparison. SVR r = suitability violation rate against revealed risk tolerance, AR = amplification ratio (

\bar{D}_{>T/2}/\bar{D}_{\leq T/2}

; values

{>}1

indicate late-session drift exceeds early-session drift). Other metrics as in Table 2 . We omit EBS (evaluation blindness score

=

SVR

{}_{s}\times\min

(UPR, 1)) because UPR

\approx 1

for all models, so EBS

\approx

SVR s .

Model	$\mathcal{M}_{\text{eq}}$	Div. $\|\mathcal{M}_{\text{eq}}$	$\bar{D}\|\mathcal{M}_{\text{eq}}$	$\bar{D}\|\mathcal{M}_{\neq}$	$\rho$
	MED Turns		Drift Estimates
Qwen3-32B	58.7%	78.5%	0.273	0.339	$0.306^{***}$
Qwen2.5-7B	27.4%	49.2%	0.251	0.273	$-0.048$
Gemma 3 12B-IT	33.0%	90.8%	0.417	0.391	$0.577^{***}$
GPT-5.2	42.2%	90.7%	$\mathbf{0.411}$	0.395	$0.351^{***}$
Claude Sonnet 4.6	43.9%	92.1%	0.389	0.381	$0.523^{***}$

Appendix

Table 24: Information-channel analysis via memory-equal divergence (MED).

\mathcal{M}_{\text{eq}}

: fraction of turns with identical memory across conditions. Div.

|\mathcal{M}_{\text{eq}}

: fraction of those turns where recommendations still diverge.

\bar{D}|\mathcal{M}_{\text{eq}}

\bar{D}|\mathcal{M}_{\neq}

: mean paired drift on MED and non-MED turns respectively.

\rho

: Spearman correlation between per-turn risk perturbation magnitude and

D_{t}

Figure 13: Cross-model comparison of contamination metrics. Error bars show standard deviation across users.

		Drift		Safety		Quality
Condition	Model	$\bar{D}$	MDR	SVR s	Sev. SVR	NDCG	UPR
Risk inv.	Claude	$0.224$	$0.118$	$0.843$	$1.487$	$0.759$	$1.041$
Risk inv.	Gemma	$0.376$	$0.214$	$0.843$	$1.509$	$0.651$	$1.235$
Metric manip.	Claude	$0.240$	$0.112$	$0.839$	$1.413$	$0.759$	$0.973$
Metric manip.	Gemma	$0.310$	$0.233$	$0.713$	$1.261$	$0.651$	$1.120$
Headlines	Claude	$0.176$	$0.094$	$0.787$	$1.361$	$0.753$	$1.005$

Appendix

Table 25: Cross-model ablation and dose-response comparison: Gemma 3 12B-IT, Claude Sonnet 4.6, and GPT-5.2 (dose-response only). SVR s for risk-inversion-only is identical across models (

0.843

), confirming the suitability violation pattern depends on how the contamination is set up, not on which model is used. Dose-response is monotonic for frequency (

p

0.25\to 0.50\to 0.75

) across all three models; strength shows a saturation plateau at

\alpha{\geq}0.75

$w$	Qwen3-32B	Qwen2.5-7B	Gemma 3	GPT-5.2	Sonnet	CC Opus	Min. 14B	Mistral L3
0.0 (pure $\tau$ )	0.206	0.217	0.329	0.335	0.315	0.245	0.619	0.353
0.1	0.237	0.233	0.353	0.357	0.338	0.248	0.636	0.370
0.2	0.269	0.250	0.376	0.379	0.361	0.251	0.653	0.387
0.3 (default)	0.301	0.267	0.399	0.402	0.384	0.254	0.670	0.404
0.5	0.364	0.300	0.446	0.446	0.430	0.261	0.704	0.438
0.7	0.427	0.333	0.492	0.490	0.477	0.267	0.738	0.472

Appendix

Table 26: Mean trajectory drift

\bar{D}

as a function of composition weight

w

across all eight models. Rankings are stable: model ordering is consistent across all weight values.

Hypothesis	Model	$W$	$p$
$\bar{D}>0$	Qwen3-32B	55.0	$0.001^{***}$
	Qwen2.5-7B	55.0	$0.001^{***}$
	Gemma 3 12B-IT	55.0	$0.001^{***}$
	GPT-5.2	55.0	$0.001^{***}$
	Claude Sonnet 4.6	55.0	$0.001^{***}$
	CC Opus 4.6	55.0	$0.001^{***}$

Appendix

Table 27: Wilcoxon signed-rank test results. Significance levels:

{}^{***}p<0.001

{}^{**}p<0.01

{}^{*}p<0.05

Setting	$\boldsymbol{\bar{D}}$	SVR s	MDR	UPR	NDCG	$\boldsymbol{\bar{D}_{\text{repeat}}}$
Default temp	$0.384$	$0.926$	$0.170$	$1.000$	$0.744$	$0.174$
$t=0$	$0.367$	$0.948$	$0.150$	$1.001$	$0.748$	$0.155$

Appendix

Table 28: Temperature sensitivity for Claude Sonnet 4.6. “Default” uses the API default temperature; “

t{=}0

” uses deterministic decoding. Metrics as in Table 2 ;

\bar{D}_{\text{repeat}}

= clean-repeat baseline drift (Section 4 ).

Model	$\boldsymbol{\bar{D}}$	$\boldsymbol{\bar{D}_{\text{repeat}}}$	$\boldsymbol{\bar{D}_{\text{excess}}}$	Ratio
Claude Sonnet 4.6	$0.384$	$0.174$	$+0.210$	$2.21\times$
GPT-5.2	$0.402$	$0.230$	$+0.172$	$1.75\times$
Gemma 3 12B-IT	$0.399$	$0.000$	$+0.399$	$\infty$
Qwen3-32B	$0.301$	$0.000$	$+0.301$	$\infty$
Qwen2.5-7B	$0.267$	$0.000$	$+0.267$	$\infty$
Ministral 3 14B	$0.670$	— †	—	—

Appendix

Table 29: Noise-floor correction: excess drift after subtracting baseline stochasticity.

\bar{D}_{\text{excess}}=\bar{D}-\bar{D}_{\text{repeat}}

Model	MED	Non-MED	MED	Non-MED	$p$ (M-W)
	Turn Count		Mean $D_{t}$
Qwen3-32B	$135$	$95$	$0.273$	$0.339$	$0.016$
Qwen2.5-7B	$63$	$167$	$0.251$	$0.273$	$0.773$
Gemma 3 12B-IT	$76$	$154$	$0.417$	$0.391$	$0.457$
GPT-5.2	$97$	$133$	$0.411$	$0.395$	$0.605$
Claude Sonnet 4.6	$101$	$129$	$0.389$	$0.381$	$0.866$

Appendix

Table 30: MED turn representativeness across models. MED turns are broadly distributed across trajectory quartiles with drift magnitudes comparable to non-MED turns.

$k$	Qwen3-32B	Qwen2.5-7B	Gemma 3	GPT-5.2	Sonnet	CC Opus	Mistral L3
1	.62 / .67	.60 / .60	.47 / .70	.60 / .65	.61 / .69	.60 / .64	.53 / .68
2	.67 / .76	.62 / .64	.61 / .85	.70 / .74	.70 / .84	.67 / .81	.60 / .83
3	.67 / .80	.64 / .67	.63 / .87	.74 / .88	.73 / .93	.72 / .89	.66 / .90
5	.67 / .80	.64 / .67	.63 / .87	.77 / .88	.77 / .93	.85 / .99	.67 / .90
7	.67 / .80	.64 / .67	.63 / .87	.77 / .88	.78 / .93	.96 / .99	.67 / .90
10	.67 / .80	.64 / .67	.63 / .87	.77 / .88	.78 / .93	.96 / .99	.67 / .90

Appendix

Table 31: SVR s at different top-

k

cutoffs. Most models produce fewer than 5 recommendations per turn, so their numbers stop changing after

k=4

k=5

. CC Opus is the only model substantially affected by truncation (it averages 8.3 items per turn). The contamination delta is positive at every

k

for every model except Qwen2.5-7B at

k=1

Figure 14: Spearman rank correlation matrix across 80 user-model pairs. Quality metrics (NDCG, UPR, EAS u ) and suitability metrics (SVR s , MDR, Sev. SVR) show near-zero correlation, suggesting they are measuring different things.

Metric	Finance	Retail
$\bar{D}$ (paired drift)	0.384	0.219
SVR s (stated)	0.926	1.000
Severity-weighted SVR	1.652	2.600
MDR (3-field composite)	0.170	0.196
MED ( $\bar{D}\|\mathcal{M}_{\text{eq}}$ )	0.389	0.149
UPR	1.000	0.967

Appendix

Table 32: Finance (Conv-FinRe, 10 users

\times

23 turns) vs. retail pilot (

\tau

-bench, 5 users

\times

15 turns), both using Claude Sonnet 4.6. All metrics computed identically across domains.

Model	Median $k$	$\tau{=}1$	$\tau{=}2$
Claude Sonnet 4.6	15	1.000	1.000
CC Opus 4.6	10	1.000	1.000
GPT-5.2	10	1.000	0.935
Gemma 3 12B-IT	10	1.000	0.974
Qwen3-32B	10	1.000	0.996
Qwen2.5-7B	5	1.000	0.183

Appendix

Table 33: Defense baseline: detection rate of a reference-based risk-score consistency monitor across thresholds and models. “Median

k

” is the median number of candidates the agent requests per turn. System-level detection (pre-filter) is 100% at

\tau{=}2

for all models. Higher thresholds (

\tau\geq 3

) yield identical rates to

\tau{=}2

because the discrete risk-score deltas in our contamination design are 0, 2, or 4.

Figure 15: Within-band (

\pm 1

) contamination vs. clean baseline and full attack (Claude Sonnet 4.6, 10 users, 23 turns). Left: Within-band achieves

61\%

of full-attack

\bar{D}

while evading threshold-based monitors. Right: Quality metrics (NDCG p , UPR) remain stable, showing near-preserved utility alongside elevated violation rates under minimal perturbation. Error bars:

\pm 1

s.d.

	Quality		Drift	Safety
Condition	NDCG p $\uparrow$	UPR $\uparrow$	$\bar{D}\downarrow$	SVR ${}_{s}\downarrow$	Sev. SVR	MDR $\downarrow$
Clean baseline	$0.744$	$1.000$	$0.000$	$0.783$	$1.383$	$0.000$
Within-band ( $\pm 1$ )	$0.759$	$1.035$	$0.233$	$0.939$	$1.548$	$0.137$
Full ( $5{\leftrightarrow}1$ )	$0.723$	$1.000$	$0.384$	$0.926$	$1.652$	$0.170$

Appendix

Table 34: Within-band (

\pm 1

) contamination comparison (Claude Sonnet 4.6, 10 users, 23 turns). Within-band perturbations evade all threshold-based monitors (

\tau\geq 1

) yet produce

61\%

of full-attack drift.

		Quality		Drift	Safety
Condition	$\rho$	sNDCG p	UPR	$\bar{D}\downarrow$	SVR ${}_{s}\downarrow$	MDR $\downarrow$
Original	$-0.41$	$0.290$	$1.000$	$0.384$	$0.926$	$0.170$
Anti-correlated	$+0.95$	$0.441$	$0.954$	$0.322$	$0.917$	$0.125$
Shuffled	$-0.41$	$0.354$	$0.965$	$0.353$	$0.896$	$0.145$

Appendix

Table 35: Universe sensitivity analysis (Claude Sonnet 4.6,

n{=}10

users per condition).

\rho

: Spearman correlation between risk scores and mean utility rank. UPR is computed from standard NDCG (not suitability-penalized). Evaluation blindness (high SVR s with UPR

\approx 1

) persists across all three structural conditions.

	SVR s		Sev. SVR
Universe	Clean	Contam.	Clean	Contam.
10 stocks	0.774	0.926	1.374	1.648
50 stocks	0.704	0.783	1.161	1.435

Appendix

Table 36: 10-stock vs. 50-stock universe (Claude Sonnet 4.6, top-5).

Model	Params	Failure %	Status
Qwen3-32B	32B	$0.0\%$	Main analysis
Qwen2.5-7B-Instruct	7B	$3.5\%$	Main analysis
Gemma 3 12B-IT	12B	$0.0\%$	Main analysis
GPT-5.2	—	$0.0\%$	Main analysis
Claude Sonnet 4.6	—	$0.0\%$	Main analysis
CC Opus 4.6	—	$0.0\%$	Main analysis

Appendix

Table 37: Structured output failure rates across all evaluated models. Failure rate = fraction of turns where the model does not produce valid JSON conforming to the ReAct schema. The eight main-analysis models (above the line) all achieve

\leq 14\%

; models below the line are excluded due to high failure rates.

n

: number of users with sufficient valid turns for metric computation.

		Quality		Drift	Safety
Model	Failure %	NDCG $\uparrow$	UPR $\uparrow$	$\bar{D}\downarrow$	SVR ${}_{s}\downarrow$	Sev. SVR	MDR $\downarrow$
Ministral 3 14B	$13.9\%$	$0.656$	$0.988$	$0.670$	$0.678$	$1.057$	$0.457$
Ministral 3 8B	$19.3\%$	$0.655$	$1.045$	$0.583$	$0.614$ †	$0.957$	$0.365$
Ministral 3 3B	$\mathbf{84.3\%}$	$0.664$ ‡	$0.303$	$0.341$	$0.161$ †	$0.265$	$0.377$

Appendix

Table 38: Ministral scale ablation. Metrics computed only over valid (non-failed) turns. Models with

>50\%

failure rate produce insufficient data for reliable computation.

Figure 16: Cosine similarity between adversarial (

\Delta_{\text{inv}}

) and random (

\Delta_{\text{shuf}}

) SAE activation shifts across 24 layer depths (every 2nd layer, 0–46) with 95% bootstrap CIs (

n{=}50

queries, 16k-width l0_small SAEs). Generation positions (blue) show an oscillatory profile; risk-score positions (red) are consistently lower with a deep minimum at layer 20. Orange diamonds: 4-layer pilot with l0_medium variant.

Figure 17: (a) Per-layer activation patching recovery (MLP: blue; attention: orange) overlaid with observational cosine similarity (gray dashed). Layer 14 is the primary causal mediator but not observationally distinctive. (b) No intervention recovers safe recommendations: SAE feature clamping/amplification at L12 and direct activation steering at L14 all yield recovery

{\leq}5\%

. Percentages show recovery relative to the clean–inverted gap (

n{=}50

queries for L12 SAE,

n{=}20

for L14 steering).

Layer	$\cos(\Delta_{\text{inv}},\Delta_{\text{shuf}})$	null	$p$	Cohen’s $d$
11	$0.281$	$0.395$	$0.205$	$0.886$
15	$0.271$	$0.395$	$0.168$	$0.735$
19	$0.299$	$0.401$	$0.218$	$0.573$

Appendix

Table 39: SAE cosine analysis on Qwen2.5-7B-Instruct (BatchTopK SAEs at layers 11/15/19,

n{=}230

paired turns, 30 shuffles/turn).

\cos(\Delta_{\text{inv}},\Delta_{\text{shuf}})

is consistently lower than the shuffle–shuffle null in the same direction as Gemma, but with weaker separation and

p>0.05

Ticker	GT	Contam.	consistent	inconsistent	no_prior	Incon.%
TSLA	5	1	38	140	9	74.9%
VZ	1	5	8	30	30	44.1%
PG	1	5	15	42	44	41.6%
LIN	2	4	37	15	68	12.5%
XOM	2	4	69	14	54	10.2%
MRK	3	3	110	5	52	3.0%

Appendix

Table 40: Per-ticker verdict distribution under parametric cross-check on Gemma 3 12B-IT (10 users, contaminated condition).

Ticker	GT	Contam.	consistent	inconsistent	no_prior	Incon.%
TSLA	5	1	0	229	0	100.0%
PG	1	5	0	230	0	100.0%
VZ	1	5	1	216	0	99.5%
TQQQ	–	1	1	169	0	99.4%
AMZN	4	2	19	208	0	91.6%
LIN	2	4	17	202	7	89.4%

Appendix

Table 41: Per-ticker verdict distribution under parametric cross-check on Claude Sonnet 4.6 (10 users, contaminated condition).

Ticker	Non-ablation	Stripped	$\Delta$
TSLA	74.9%	89.6%	$+14.7$
PG	41.6%	60.8%	$+19.2$
VZ	44.1%	41.0%	$-3.1$
TQQQ	41.9%	75.0%	$+33.1$
LIN	12.5%	26.9%	$+14.4$
XOM	10.2%	20.9%	$+10.7$

Appendix

Table 42: Per-ticker inconsistent-flag rate under stripped-field ablation (Gemma 3 12B-IT, contaminated condition), compared to the non-ablation parametric-check baseline (Table 40 ). For most strong-prior tickers the flag rate increases under the stripped condition, consistent with the rich in-context fields in the non-ablation condition partially suppressing the parametric signal.

LLM Agent Safety

Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.

See all 302 papers

Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)
Mar 18, 2026Zichen Tang, Zirui Zhang, Qian Wang +3Large Language Model-Based Recommendation LLM Agent Reliability
The Hong Kong University of Science and Technology · National University of Singapore · The Hong Kong University of Science and Technology (Guangzhou)
Representation Signatures and Risk-Feedback Alignment in LLM Trading Agents
May 16, 2026Weicheng XueLLM Alignment LLM Auditing
Virginia Tech
ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
Jul 15, 2026Aryan Keluskar, Amrita Bhattacharjee, Huan LiuLLM Alignment AI Agent Safety Benchmarks
School of Computing & AI, Arizona State University, Tempe, AZ, USA.

Mar 18, 2026J/K move · Enter open · S save

Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)

Zichen Tang, Zirui Zhang, Qian Wang, Zhenheng Tang+2

The Hong Kong University of Science and Technology · National University of Singapore · The Hong Kong University of Science and Technology (Guangzhou)

Large Language Model-Based Recommendation LLM Agent Reliability Language Model Bias Evaluation

Open paper AskPDF arXiv

Current Large Language Models (LLMs) are gradually exploited in practically valuable agentic workflows such as Deep Research, E-commerce recommendation, and job recruitment. In these applications, LLMs need to select some optimal solutions from massive candidates, which we term as LLM-as-a-Recommender paradigm. However, the reliability of using LLM agents for recommendations is underexplored. In this work, we introduce a Bias Recommendation Benchmark (BiasRecBench) to highlight the critical vulnerability of such agents to biases in high-value real-world tasks. The benchmark includes three practical domains: paper review, e-commerce, and job recruitment. We construct a Bias Synthesis Pipeline with Calibrated Quality Margins that 1) synthesizes evaluation data by controlling the quality gap between optimal and sub-optimal options to provide a calibrated testbed to elicit the vulnerability to biases; 2) injects contextual biases that are logical and suitable for option contexts. Extensive experiments on both SOTA (Gemini-{2.5,3}-pro, GPT-4o, DeepSeek-R1) and small-scale LLMs reveal that agents frequently succumb to injected biases despite having sufficient reasoning capabilities to identify the ground truth. These findings expose a significant reliability bottleneck in current agentic workflows, calling for specialized alignment strategies for LLM-as-a-Recommender. The complete code and evaluation datasets will be made publicly available shortly.

Abstract

LLM Agent Safety

Explore similar work

Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)

Representation Signatures and Risk-Feedback Alignment in LLM Trading Agents

ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs