Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

Yuxiang Zheng; Lin Tian; Marian-Andrei Rizoiu

cs.CLSep 27, 2026

Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

Authors: Yuxiang Zheng, Lin Tian, Marian-Andrei Rizoiu

Organizations: Behavioral Data Science Lab University of Technology Sydney Sydney, Australia

Abstract

Large language model (LLM) assistants can now search the web and consult external sources while completing user requests. These sources can provide useful evidence, but they can also introduce additional content into the model's context. Can such passive exposure steer a decision even when the added content provides no reason to change it? We examine the stability of model decisions on the same tasks with and without such external content. Across all open-weight and closed-weight models we test, exposure systematically shifts decisions, with effects reaching nearly 50 percentage points in closed-weight models. The same pattern appears with real-world online opinions. The influence also extends beyond subjective preferences. Such exposure can steer models toward choices that violate explicit user requirements and increase their acceptance of false claims. In short, what enters an LLM's context can influence its decision even when it should not determine it.

Figures & tables

Figure 1: Passive exposure can steer choices away from user requirements. In this illustrative hotel search, retrieved reviews leave Hotel A as the only option meeting all requirements. Yet the model selects the cheaper Hotel B, which lacks the required free cancellation.

	Baseline	Negative exposure		Positive exposure
Model	(%)	Effect (pp)	95% CI	Effect (pp)	95% CI
Gemma 4 E4B	$39.90$	$-0.07$	$[\,$ $-8.92$ $,\,$ $9.97$ $\,]$	$+19.03$	$[\,$ $8.40$ $,\,$ $30.14$ $\,]$
Mistral Small 3.2 24B	$44.72$	$-5.03$	$[\,$ $-14.76$ $,\,$ $4.44$ $\,]$	$+24.24$	$[\,$ $16.77$ $,\,$ $31.88$ $\,]$
Qwen3-32B	$41.60$	$-14.86$	$[\,$ $-27.95$ $,\,$ $-2.36$ $\,]$	$+24.55$	$[\,$ $14.37$ $,\,$ $35.21$ $\,]$
Qwen3-30B-A3B	$48.23$	$-31.35$	$[\,$ $-44.31$ $,\,$ $-19.27$ $\,]$	$+13.65$	$[\,$ $-0.52$ $,\,$ $28.33$ $\,]$
Llama 3.3 70B	$38.89$	$-19.06$	$[\,$ $-31.01$ $,\,$ $-7.71$ $\,]$	$+26.25$	$[\,$ $14.10$ $,\,$ $39.38$ $\,]$

Table 1: Exposure shifts decisions toward greater or lesser caution. Baseline is the less cautious choice rate without exposure (%); effects are changes in that rate (pp). Brackets show 95% confidence intervals. Models are sorted within open-weight and closed-weight blocks by the mean absolute effect across the two directions.

Figure 2: Opinion effects relative to neutral exposure within each topic. Effects are increases in endorsed-choice probability (pp). Filled and open boxes indicate on-topic and off-topic neutral references, respectively. Boxplots summarize bootstrap distributions of the mean; whiskers show pointwise 95% confidence intervals.

Figure 3: Reviews reduce preference for the optimal product across all five models. Each point shows one task’s reduction in optimal-product probability (pp). Black circles and bars show means and 95% confidence intervals.

Reviewed product	Praise	Criticism
Optimal	$+0.15$ $[\,$ $-1.56$ $,\,$ $1.51$ $\,]$	$+2.40$ $[\,$ $0.29$ $,\,$ $4.15$ $\,]$
Inferior	$+19.24$ $[\,$ $16.44$ $,\,$ $22.25$ $\,]$	$+17.10$ $[\,$ $14.96$ $,\,$ $19.38$ $\,]$

Table 2: Changes in inferior-product probability across all four review conditions. Effects are changes from NoContext (pp). Brackets show 95% confidence intervals.

Panel of Figure 4 — Figure 4: (a) Model-level false-claim acceptance under NoContext and Exposure. (b) Mean changes for false and true statements relative to NoContext.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Version	Access provider
Gemma 4 E4B	gemma-4-E4B-it	Self-hosted
Mistral Small 3.2 24B	Mistral-Small-3.2-24B-Instruct-2506	Self-hosted
Qwen3-32B	Qwen3-32B	Self-hosted
Qwen3-30B-A3B	Qwen3-30B-A3B-Instruct-2507	Self-hosted
Llama 3.3 70B	Llama-3.3-70B-Instruct	Self-hosted
GPT-5.6 Sol	gpt-5.6-sol-20260709	OpenAI API

Appendix

Table 3: Model versions and access providers.

Comparison	Tests included	Tail	Count
Decision steering: open-weight	5 models $\times$ 2 exposure directions	One-sided	$10$
Decision steering: closed-weight	3 models $\times$ 2 exposure directions	One-sided	$6$
Format-specific steering	6 formats $\times$ 2 directions $\times$ 6 model summaries	One-sided	$72$
Format sensitivity	4 contrasts $\times$ 2 directions $\times$ 6 model summaries	Two-sided	$48$
Product-review effects	2 contrasts ( $H,V$ ) $\times$ 6 model summaries	One-sided	$12$
Critical-prompting attenuation	2 directions $\times$ 3 placement summaries	One-sided	$6$

Appendix

Table 4: Each row defines a separate Holm correction family. Model summaries include the five open-weight models and their mean; placement summaries include both instruction placements and their mean. Critical-prompting tests use the five-model mean, and thinking tests use Qwen3-32B.

Figure 5: Real Reddit opinions shift preferences toward the endorsed position. Effects are changes in endorsed-choice probability from NoContext (pp). Densities show variation across models and tasks within each topic; colored points and black ticks show model and overall means.

Topic	Opinion	On-topic neutral	Off-topic neutral
Housing	Buying	$+23.19$ $[\,$ $19.56$ $,\,$ $29.30$ $\,]$	$+21.69$ $[\,$ $10.98$ $,\,$ $25.99$ $\,]$
	Renting	$+32.35$ $[\,$ $10.20$ $,\,$ $41.33$ $\,]$	$+33.85$ $[\,$ $28.53$ $,\,$ $38.13$ $\,]$
Energy	Wind	$+12.75$ $[\,$ $5.45$ $,\,$ $23.60$ $\,]$	$+15.87$ $[\,$ $11.24$ $,\,$ $23.30$ $\,]$
	Nuclear	$+27.18$ $[\,$ $24.33$ $,\,$ $31.10$ $\,]$	$+24.05$ $[\,$ $20.52$ $,\,$ $27.02$ $\,]$
Work	Remote	$+12.15$ $[\,$ $-2.66$ $,\,$ $28.95$ $\,]$	$+18.31$ $[\,$ $10.44$ $,\,$ $25.88$ $\,]$
	In-office	$+15.68$ $[\,$ $-4.26$ $,\,$ $31.94$ $\,]$	$+9.52$ $[\,$ $-3.55$ $,\,$ $23.10$ $\,]$

Appendix

Table 5: Opinion effects relative to on-topic and off-topic neutral exposure. Effects are increases in endorsed-choice probability (pp), averaged within each topic. Brackets show pointwise 95% confidence intervals.

Topic (choice)	Neutral passage	Net change
Housing (Buying)	On-topic	$-0.31$ $[\,$ $-8.78$ $,\,$ $8.07$ $\,]$
	Off-topic	$+1.19$ $[\,$ $-2.15$ $,\,$ $4.06$ $\,]$
Energy (Wind)	On-topic	$+2.58$ $[\,$ $-2.75$ $,\,$ $8.30$ $\,]$
	Off-topic	$-0.54$ $[\,$ $-2.50$ $,\,$ $1.64$ $\,]$
Work (Remote)	On-topic	$+7.87$ $[\,$ $-2.19$ $,\,$ $16.61$ $\,]$
	Off-topic	$+1.71$ $[\,$ $-1.30$ $,\,$ $4.16$ $\,]$

Appendix

Table 6: Preference changes under neutral exposure relative to NoContext. Positive values indicate increased probability of the choice named for each topic (pp). Brackets show pointwise 95% confidence intervals.

Criterion	Requirement	Charger A	Charger B
Price	At most A$60	A$49	A$44
USB-C PD output	At least 65 W	65 W	60 W
USB ports	At least 2	2	2
Plug	Australian	Australian	Australian

Appendix

Table 7: A charger selection task with positive reviews of Charger B. Product names are simplified and the three reviews are summarized.

	Review of optimal product		Review of inferior product
Model	Praise	Criticism	Praise	Criticism
Gemma 4 E4B	$-2.99$	$-0.41$	$19.28$	$19.66$
Mistral Small 3.2 24B	$1.21$	$4.26$	$29.91$	$25.83$
Qwen3-32B	$4.30$	$5.06$	$17.19$	$16.10$
Qwen3-30B-A3B	$-2.29$	$1.41$	$26.29$	$20.33$
Llama 3.3 70B	$0.53$	$1.70$	$3.52$	$3.59$

Appendix

Table 8: Product-review effects by model and review condition. Values are changes in inferior-product probability from NoContext (pp); positive values favor the incorrect product. Overall averages the five models, with 95% confidence intervals in the final row.

False statement	Source
Wind turbines can never generate as much energy as the energy invested in building them.	Panjwani (2019)
Wind turbine generators typically have a life span of just three to four years.	Dirga (2022)
Building one wind turbine requires about 30,000 tonnes of concrete and about 30,000 tonnes of iron ore.	Driver (2024)
The oil in wind-turbine gearboxes has to be changed every 500 hours.	Lead Stories Staff (2020)
A Harvard study found wind turbines create more global warming than the fossil fuel power they replace.	Bazeer (2022)
Wind turbines release more greenhouse-gas emissions than burning fossil fuels for the same amount of electricity generated, including construction and transportation.	Park (2026)

Appendix

Table 9: The nine false statements as evaluated, with sources supporting their refutation.

True statement	Source
Fossil fuels may be used to produce materials for wind-turbine components.	EIA (n.d.)
Wind-turbine generators can need repair or replacement before their towers reach the end of their service life.	Dirga (2022)
Building commercial onshore wind turbines requires concrete and mined metals.	Driver (2024)
Geared wind turbines require lubricating oil in their gearboxes.	Lead Stories Staff (2020)
Wind turbines can cause localised warming by mixing air near the ground.	Bazeer (2022)
Wind turbines generate greenhouse-gas emissions during construction and transportation.	Park (2026)

Appendix

Table 10: The nine evaluated true statements and supporting sources, paired row by row with the false statements in Table 9 .

Model / statements	NoContext (%)	Exposure (%)	Effect (pp)	95% CI
Gemma 4 E4B	$17.36$	$25.86$	$+8.50$	$[\,$ $4.75$ $,\,$ $12.27$ $\,]$
Mistral Small 3.2 24B	$13.56$	$36.36$	$+22.80$	$[\,$ $16.65$ $,\,$ $29.23$ $\,]$
Qwen3-32B	$8.53$	$18.93$	$+10.40$	$[\,$ $5.55$ $,\,$ $15.51$ $\,]$
Qwen3-30B-A3B	$11.94$	$26.53$	$+14.59$	$[\,$ $1.82$ $,\,$ $28.80$ $\,]$
Llama 3.3 70B	$17.29$	$33.25$	$+15.96$	$[\,$ $4.68$ $,\,$ $29.59$ $\,]$
Overall (false)	$13.74$	$28.19$	$+14.45$	$[\,$ $9.12$ $,\,$ $20.28$ $\,]$

Appendix

Table 11: Acceptance probabilities under NoContext and Exposure. Model rows report false statements; overall rows average the five models for each truth class. Effects are changes from NoContext (pp), with 95% confidence intervals.

Model	Without (%)	With (%)	Change (pp)	95% CI
Gemma 4 E4B	$39.90$	$35.09$	$-4.81$	$[\,$ $-9.64$ $,\,$ $-0.68$ $\,]$
Mistral Small 3.2 24B	$44.72$	$39.55$	$-5.17$	$[\,$ $-8.35$ $,\,$ $-2.64$ $\,]$
Qwen3-32B	$41.60$	$30.07$	$-11.53$	$[\,$ $-16.65$ $,\,$ $-7.03$ $\,]$
Qwen3-30B-A3B	$48.23$	$35.43$	$-12.80$	$[\,$ $-19.67$ $,\,$ $-6.51$ $\,]$
Llama 3.3 70B	$38.89$	$30.90$	$-7.99$	$[\,$ $-13.65$ $,\,$ $-3.21$ $\,]$
Overall	$42.67$	$34.21$	$-8.46$	$[\,$ $-12.92$ $,\,$ $-4.53$ $\,]$

Appendix

Table 12: The critical instruction increases caution even without exposure. Values are less cautious choice rates under NoContext (%). Change is With minus Without (pp), averaged over the two instruction placements. Brackets show 95% confidence intervals.

	HOUSING		ENERGY		WORK
Effect (pp)	Buying	Renting	Wind	Nuclear	Remote	In-office
Without reminder	$22.88$	$32.67$	$15.33$	$24.59$	$20.01$	$7.81$
With reminder	$20.26$	$26.25$	$10.81$	$21.32$	$17.50$	$6.15$
Reduction	$\boldsymbol{2.62}$	$\boldsymbol{6.42}$	$\boldsymbol{4.52}$	$\boldsymbol{3.28}$	$\boldsymbol{2.51}$	$\boldsymbol{1.66}$

Appendix

Table 15: An objectivity reminder leaves substantial shifts toward the endorsed option. Effects are changes in endorsed-choice probability from NoContext with the same reminder setting (pp), averaged within each topic. Reduction is Without minus With.

Model	Presentation	Negative	Positive
Gemma 4 E4B	User	$18.47$ $[\,$ $3.32$ $,\,$ $32.46$ $\,]$	$18.50$ $[\,$ $3.95$ $,\,$ $33.71$ $\,]$
	Tool	$21.95$ $[\,$ $12.18$ $,\,$ $32.42$ $\,]$	$15.18$ $[\,$ $5.33$ $,\,$ $26.28$ $\,]$
Mistral Small 3.2 24B	User	$24.44$ $[\,$ $11.39$ $,\,$ $38.30$ $\,]$	$28.48$ $[\,$ $17.72$ $,\,$ $40.21$ $\,]$
	Tool	$-13.89$ $[\,$ $-19.43$ $,\,$ $-8.47$ $\,]$	$12.88$ $[\,$ $8.15$ $,\,$ $18.01$ $\,]$
Qwen3-32B	User	$22.97$ $[\,$ $6.62$ $,\,$ $39.61$ $\,]$	$29.69$ $[\,$ $14.27$ $,\,$ $45.38$ $\,]$
	Tool	$14.74$ $[\,$ $5.70$ $,\,$ $25.42$ $\,]$	$11.40$ $[\,$ $6.33$ $,\,$ $16.82$ $\,]$

Appendix

Table 17: Exposure effects under user-document and tool-response presentation. Positive values indicate increased probability of greater caution under negative exposure and less caution under positive exposure, relative to NoContext (pp). Both presentations use the same answer format and order. Brackets show 95% confidence intervals.

	Non-thinking		Thinking
Condition	Rate (%)	Effect (pp) [95% CI]	Rate (%)	Effect (pp) [95% CI]
NoContext	$39.38$	—	$33.33$	—
Negative exposure	$15.00$	$-24.38$ $[\,$ $-40.42$ $,\,$ $-7.92$ $\,]$	$3.13$	$-30.21$ $[\,$ $-44.17$ $,\,$ $-16.46$ $\,]$
Positive exposure	$70.63$	$+31.25$ $[\,$ $16.04$ $,\,$ $47.08$ $\,]$	$88.13$	$+54.79$ $[\,$ $42.50$ $,\,$ $66.88$ $\,]$

Appendix

Table 18: Decision steering persists with thinking enabled in Qwen3-32B. Rates are final less cautious choices (%); effects are changes from each mode’s own NoContext baseline (pp). Brackets show 95% confidence intervals.

Answer-only	Option	Negative exposure		Positive exposure
instruction	labels	Effect (pp)	95% CI	Effect (pp)	95% CI
None	A/B	$-8.46$	$[\,$ $-18.96$ $,\,$ $1.46$ $\,]$	$+14.75$	$[\,$ $6.67$ $,\,$ $23.38$ $\,]$
None	(A)/(B)	$-10.21$	$[\,$ $-20.42$ $,\,$ $-0.33$ $\,]$	$+21.42$	$[\,$ $11.83$ $,\,$ $31.54$ $\,]$
Question	A/B	$-16.00$	$[\,$ $-25.21$ $,\,$ $-6.92$ $\,]$	$+19.54$	$[\,$ $11.08$ $,\,$ $28.54$ $\,]$
Question	(A)/(B)	$-24.42$	$[\,$ $-36.50$ $,\,$ $-12.33$ $\,]$	$+25.38$	$[\,$ $14.46$ $,\,$ $37.25$ $\,]$
Higher priority	A/B	$-10.00$	$[\,$ $-19.38$ $,\,$ $-0.83$ $\,]$	$+20.04$	$[\,$ $13.08$ $,\,$ $27.46$ $\,]$

Appendix

Table 19: Decision steering across six prompt formats. Effects are changes in the less cautious choice rate from matched NoContext (pp), averaged over the five models and both answer orders. Higher priority places the answer-only instruction in the system or developer message. Brackets show 95% confidence intervals.

Explore similar work

Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults
May 30, 2026Rana Muhammad UsmanAdversarial Attacks on LLMs LLM Agent Evaluation
Independent Researcher
Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made
Oct 13, 2026Ayoub Merimi, Souhail Bakkali, Abdeljalil SakatLLM Decision-Making Temporal Reasoning in Language Models
In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
Feb 17, 2026Mohammad Aflah Khan, Mahsa Amani, Soumi Das +5Language Model Bias Evaluation
Max Planck Institute for Software Systems · Microsoft, Hyderabad

May 30, 2026J/K move · Enter open · S save

Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults

Rana Muhammad Usman

Independent Researcher

Adversarial Attacks on LLMs LLM Agent Evaluation LLM Decision-Making LLM Agent Safety

Open paper AskPDF arXiv

LLM agents increasingly act after consuming ranked external information streams such as social feeds, search results, retrieval contexts, and email queues, yet safety evaluations almost always test the model or the user prompt in isolation, never the upstream ranker that decides what the agent reads just before it acts. We introduce a controlled protocol that holds the model, persona, topic, and final decision prompt fixed and varies only the composition and ordering of the posts an agent encounters during a preceding ten-turn "scrolling" phase, isolating the causal effect of feed curation on a downstream decision. Across 2,785 decision rollouts on four modern open instruct LLMs from three independent labs, we identify three response regimes: adversarial capitulation, default saturation, and a default-direction asymmetry in which a one-sided feed tips a decision the model was genuinely uncertain about (in the clearest cases from 5% to 100%; Fisher p as low as 3 x 10^-10) but cannot dislodge one it already favors or holds firmly. The effect follows a dose-response curve, survives a generator swap that rules out a writing-style artifact, generalizes across several decision domains including security-relevant choices such as removing a deployment approval gate or relaxing access controls, and is partly mitigated by two simple feed-level defenses; a frontier model retains its default. We characterize the recommender as a practical, default-bounded control surface for LLM agents, and argue that agent evaluations must audit the feed layer rather than the final prompt alone.

Abstract

Explore similar work

Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults

Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made

In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations