Et Tu, Brute? Economic Misalignment in Personal AI Agents

Aman Priyanshu; Supriti Vijay; Brian Jabarian; Niloofar Mireshghallah

cs.AISep 21, 2026

Et Tu, Brute? Economic Misalignment in Personal AI Agents

Authors: Aman Priyanshu, Supriti Vijay, Brian Jabarian, Niloofar Mireshghallah

Organizations: Foundation AI, Cisco · Carnegie Mellon University

Abstract

Personal AI agents make recommendations and take actions on people's behalf in high-stakes economic contexts, e.g., buying a flight, choosing health insurance, or selecting a graduate program. The agent is given access to the user's personal context, e.g., their email inbox and a structured profile of personal attributes, with the intention of making an optimal, personalized decision for the user. We show that by simply providing this personal context, the agent steers recommendations based on inferred wealth, without being explicitly instructed to do so. In a suite of 325K experiments on 13 agents across three types of economic decisions (flights, health insurance, and graduate programs), we find that 8 models systematically choose more expensive options for wealthier users when requests are identical. This steering continues even when it directly goes against the user's stated objective: when explicitly instructed to find the cheapest option, some agents still act on the wealth profile they have inferred. It also occurs when wealth is inferred from ambient data, such as emails unrelated to the task. And it persists under privacy controls that block specific attributes: blocking financial attributes largely removes the disparity, but blocking other attributes leaves it unchanged and can increase it by up to 40% for insurance, as agents rely on the remaining signals to infer wealth. Larger and more capable models are no better; Claude Opus 4.8 shows the largest effect. We term this misalignment "adversarial delegation", in which the very conditions that make a personal AI agent useful - access to personal information - enable it to act against the user's interests.

Figures & tables

Figure 1: Example of adversarial delegation in a single evaluation. The user queries for the most affordable airfare to Chicago. The agent returns a $91 economy class ticket if it is uncalibrated—that is, has no knowledge of the user (left). However, when the agent has access to user data (right), it finds three emails about finance, from which it infers that the user would be willing to pay more for a seat, and recommends a $601 business-class ticket, despite the existence of the $91 ticket. The agent was not informed of the user’s net worth nor instructed to take it into account. We show that this phenomenon is systematic and not just anecdotal across our experiments (Table 1 ). — Figure 1: Example of adversarial delegation in a single evaluation. The user queries for the most affordable airfare to Chicago. The agent returns a $91 economy class ticket if it is uncalibrated—that is, has no knowledge of the user (left). However, when the agent has access to user data (right), it finds three emails about finance, from which it infers that the user would be willing to pay more for a seat, and recommends a$ 601 business-class ticket, despite the existence of the $91 ticket. The agent was not informed of the user’s net worth nor instructed to take it into account. We show that this phenomenon is systematic and not just anecdotal across our experiments (Table 1 ).

Figure 2: Blocking financial information most strongly changes the discrimination gap, while blocking other attributes generally leaves it intact or amplifies it. Each row blocks a single attribute (returning access_restricted ) while the other four remain available. Results are shown for GPT-5, Claude Opus 4.8, and Gemini 2.5 Flash across flights, insurance, and graduate programs. Each cell reports the discrimination gap in the blocked condition, with the percentage change in its magnitude relative to the corresponding tool-full condition shown in parentheses. Color encodes this relative change: blue indicates a reduction in gap magnitude, red indicates an increase, and zero indicates no change. Negative gap values indicate that the direction of the disparity has reversed. Across most model–domain pairs, blocking financial information sharply reduces the gap, whereas blocking employment, health, life events, or demographics typically preserves or increases it.

Figure 3: Limited email access can produce larger discrimination gaps than full-inbox access. Discrimination gaps on flights for GPT-5, Claude Opus 4.8, and Gemini 2.5 Flash across the access spectrum, from inferential email access to direct profile access. With no email bodies available, gaps are near zero across all three models. For Opus 4.8 and Gemini 2.5 Flash, the gap is largest after only two emails (

248 and

175, respectively) and decreases as more of the inbox becomes available, reaching

59 and

91 with full-inbox access. GPT-5 shows the same qualitative pattern at smaller magnitude.

Figure 4: Explicit preferences do not necessarily remove wealth-conditioned recommendations. Recommended flight prices under four user intents for Gemini 2.5 Flash, GPT-5, and Claude Opus 4.8. Bars show the low-income recommendation and the additional wealth gap between the wealthy and the low-income. Even when users explicitly request the cheapest option, Gemini 2.5 Flash still has a

208 gap, while the corresponding gaps for GPT-5 and Opus 4.8 are

21 and $20, respectively.

Family	Model	Flights ($)	Insurance ($/mo)	Grad schools ($/yr)	Mean $d$
GPT	GPT-5.5	+92	+122	+763	0.26
	GPT-5	+107	+191	+1,061	0.33
	GPT-5-mini	+74	+124	+2,647	0.34
	GPT-5-nano	+13	+56	+1,622	0.16
Claude	Claude Opus 4.8	+198	+284	+3,467	0.85
	Claude Sonnet 5	+176	+151	+2,406	0.52

Table 1: Discrimination gap by model and domain (tool-full condition); larger is not safer. Models are grouped by family. The disparity increases with model size within the same family (e.g. GPT-5 nano to base with +

13

\to

+

\to

107; Appendix B.5 examines the gap between retrieval and recommendation steering), whereas the disparity with GPT-5.5 (+

92) is lower than GPT-5, implying capabilities and discriminatory behavior are not monotonically increasing with model generations. Claude Opus 4.8 produces the largest effect across all models (Cohen’s

d=0.85

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Inventory	Price range	Context
Flights	200 flights	$91–$ 883	Denver (DEN) to Chicago (ORD), Jan 10–24
Health insurance	200 plans	$85–$ 1,350/mo	Colorado, single zip code
Graduate programs	200 CS PhD programs	$-$ $20K to +$ 61K net/yr	Same undergraduate background

Appendix

Table 2: Domain inventories and price ranges.

Intent	Flights	Insurance	Graduate programs
Neutral	“Find me a flight”	“Find me a plan”	“Help me find programs”
Cheap	“Find the cheapest”	“Cheapest covering basics”	“Funded, highest acceptance”
Quality	“Most comfortable”	“Best coverage regardless”	“Highest-ranked regardless”
Ceiling	“Under $200”	“Under $220/month”	“Full funding”

Appendix

Table 3: User intents by domain.

Condition	Count	Persona access	Role in the design
Control	1	No persona information	Provides the stateless baseline.
In-context	1	All five axes of persona information	Provides the full profile in the system prompt without any retrieval.
Tool-full	1	All five axes through get_attributes(axis)	Enables us to measure which profile information the agent chooses to access.
Tool-blocked	5	Four axes available and one restricted	Measures how access to each axis changes the recommendation gap relative to tool-full.
Email-full	1	All ten bodies through read_email(id)	Measures recommendations when the agent can infer the full persona from email content.
Email-capped	5	Subjects plus 0, 2, 4, 6, or 8 bodies	Measures how reading more emails changes recommendations; the zero-read condition tests subject lines alone (giving us the subject-baseline to control for bias).

Appendix

Table 4: Information-access conditions. The count gives the number of conditions in each group.

Subject	High-level body	Low-level body
“Your Annual Tax Filing”	Capital gains $847K, K-1 forms	W-2 income $31,400, EITC
“Health Insurance Update”	Platinum PPO renewal, concierge care	Medicaid redetermination notice
“Neighborhood Newsletter”	HOA meeting, $2.1M median home value	Section 8 recertification
“Employment Update”	Board meeting, stock vesting schedule	Shift schedule for next week
“Life Update”	Closing on second property	Eviction prevention resources

Appendix

Table 5: Matched email examples for the five persona axes. Subjects are held fixed within each high–low pair. The objective here is to both test information access-priority and ability to infer user-profile

Condition	GPT-5	GPT-5- mini	GPT-5- nano	GPT-5.5	Claude Opus 4.8	Claude Sonnet 5	Claude Haiku 4.5	Gemini 2.5 Flash	Gemini 3.1 Flash Lite	Gemini 3 Flash	Qwen3.5- 2B	Qwen3.5- 9B	Qwen3.5- 35B-A3B
in_context	+153	+119	+64	+134	+136	+124	+103	+249	+98	+181	+12	+154	+148
tool_full	+107	+74	+13	+92	+198	+176	+141	+177	+112	+145	+14	+138	+141
tool_blocked_financial	-11	+9	-2	-1	+4	+1	-2	+17	+8	-4	+6	-5	+10
tool_blocked_employment	+117	+69	+25	+123	+224	+220	+159	+197	+162	+192	+21	+153	+170
tool_blocked_health	+92	+62	+38	+88	+176	+173	+138	+155	+119	+150	+12	+135	+118
tool_blocked_life_events	+113	+69	+46	+96	+214	+204	+151	+178	+99	+175	+10	+136	+144

Appendix

Table 6: Flight discrimination gap ($) by information condition, averaged over intents and motivations.

Condition	GPT-5	GPT-5- mini	GPT-5- nano	GPT-5.5	Claude Opus 4.8	Claude Sonnet 5	Claude Haiku 4.5	Gemini 2.5 Flash	Gemini 3.1 Flash Lite	Gemini 3 Flash	Qwen3.5- 2B	Qwen3.5- 9B	Qwen3.5- 35B-A3B
in_context	+240	+192	+128	+189	+346	+186	+201	+247	+60	+181	+26	+203	+174
tool_full	+191	+124	+56	+122	+284	+151	+158	+217	+179	+332	+17	+195	+133
tool_blocked_financial	-1	+1	-7	-14	-9	+1	-13	-3	-43	+17	-14	-1	+0
tool_blocked_employment	+210	+105	+77	+171	+317	+195	+194	+246	+183	+370	+20	+236	+151
tool_blocked_health	+220	+129	+53	+126	+323	+171	+169	+219	+326	+508	+29	+179	+104
tool_blocked_life_events	+201	+109	+69	+124	+287	+157	+166	+232	+250	+330	+63	+214	+150

Appendix

Table 7: Insurance discrimination gap ($/mo) by information condition, averaged over intents and motivations.

Condition	GPT-5	GPT-5- mini	GPT-5- nano	GPT-5.5	Claude Opus 4.8	Claude Sonnet 5	Claude Haiku 4.5	Gemini 2.5 Flash	Gemini 3.1 Flash Lite	Gemini 3 Flash	Qwen3.5- 2B	Qwen3.5- 9B	Qwen3.5- 35B-A3B
in_context	+1,599	+2,859	+2,078	+1,644	+2,576	+2,623	+2,898	+3,938	+8,214	+9,337	+267	+3,070	+3,605
tool_full	+1,061	+2,647	+1,622	+763	+3,467	+2,406	+2,322	+607	+5,591	+11,081	-2,056	+2,872	+3,827
tool_blocked_financial	+117	+4	+152	-114	-58	+50	+27	-1,001	-648	+656	+1,792	-471	-403
tool_blocked_employment	+1,678	+2,501	+2,558	+664	+4,486	+3,570	+3,105	+682	+6,384	+11,787	+2,911	+3,609	+4,470
tool_blocked_health	+1,733	+2,279	+2,341	+635	+3,803	+2,740	+2,962	+3,926	+5,193	+15,226	-627	+3,299	+3,290
tool_blocked_life_events	+2,542	+2,887	+2,636	+1,228	+3,993	+3,550	+3,128	+2,427	+6,226	+10,513	-59	+2,694	+4,908

Appendix

Table 8: Graduate school discrimination gap ($/yr) by information condition, averaged over intents and motivations.

Blocked axis	GPT-5	Gemini 2.5 Flash	Gemini 3 Flash	Qwen3.5-9B	Qwen3.5- 35B-A3B
None (tool-full)	+107	+177	+145	+138	+141
Financial	$-$ 11	+17	$-$ 4	$-$ 5	+10
Employment	+117	+197	+192	+153	+170
Health	+92	+155	+150	+135	+118
Life Events	+113	+178	+175	+136	+144
Demographics	+119	+178	+172	+150	+146

Appendix

Table 9: Flight discrimination gap ($) by blocked axis, with all intents pooled. Appendix B.2 reports all models and domains.

Model	Financial Retrieval (%)	Flights $d$	Insurance $d$	Flights Gap ($)
GPT-5	100.0	0.41	0.42	+107
GPT-5-mini	97.9	0.29	0.29	+74
GPT-5-nano	87.1	0.07	0.15	+13
GPT-5.5	100.0	0.39	0.28	+92
Claude Opus 4.8	100.0	1.01	0.73	+198
Claude Sonnet 5	100.0	0.83	0.33	+176

Appendix

Table 10: Financial-axis retrieval and recommendation effects for all 13 models in the tool-full condition.

Personal AI Agents

Personal AI agents use persistent user context and digital tools to carry out personalized, proactive tasks across apps, devices, and services.

See all 64 papers

Strategic Advice in the Age of Personal AI
Mar 2, 2026Yueyang Liu, Wichinpong Park SinchaisriTrust in AI Human-AI Decision Making
1Jones Graduate School of Business, Rice University · 2Haas School of Business, University of California, Berkeley
When Agents Shop for You: Role Coherence in AI-Mediated Markets
Apr 29, 2026Soogand Alavi, Salar NozariPrivacy Leakage in Language Models Agentic Commerce
Tippie College of Business, University of Iowa
Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making
Sep 21, 2026Hasibur Rahman, Benjamin R. Cowan, Smit DesaiAI-Assisted Decision Making Human-AI Decision Making
Northeastern University, Boston, Massachusetts, USA · University College Dublin, Dublin, Ireland

Mar 2, 2026J/K move · Enter open · S save

Strategic Advice in the Age of Personal AI

Yueyang Liu, Wichinpong Park Sinchaisri

1Jones Graduate School of Business, Rice University · 2Haas School of Business, University of California, Berkeley

Trust in AI Human-AI Decision Making Personal AI Agents

Open paper AskPDF arXiv

Personal AI assistants are changing how individuals use advice. We study how an advisor should design its recommendation in anticipation of stochastic consultation with personal AI whose recommendation is predictable. Personal AI enters through two dimensions: consultation probability and relative trust, which captures the relative influence personal AI receives when consulted. In the baseline model, the advisor optimally counteracts the personal AI signal. Counteraction increases with consultation probability but is hump-shaped in relative trust. The advisor's minimized loss is hump-shaped in consultation probability, vanishing when personal AI is never or always consulted. Greater relative trust in personal AI increases the irreducible loss arising from stochastic consultation. We extend the analysis to partial predictability and costly recommendation adjustment, characterizing their effects on optimal recommendations and minimized loss. The framework also accommodates richer information structures, including settings in which personal AI is perceived as having access to private information relevant to the task. We introduce an online forecasting experiment that examines how participants obtain personal AI advice and combine it with an advisor's recommendation and their initial judgments. Participants place weight on all three inputs. When access requires an additional action, some participants do not seek personal AI advice, while some others attempt to obtain it without success. Together, these findings highlight two distinct aspects of personal AI use: whether advice is obtained and how much weight it receives when available.

Abstract

Personal AI Agents

Explore similar work

Strategic Advice in the Age of Personal AI

When Agents Shop for You: Role Coherence in AI-Mediated Markets

Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making