Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

Haoran Bu; Yiting Shen; Litian Zhang; Ruihan Hu; Xi Zhang

cs.CROct 28, 2026

Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

Authors: Haoran Bu, Yiting Shen, Litian Zhang, Ruihan Hu, Xi Zhang

Organizations: Beijing University of Posts and Telecommunications, Beijing, China

Abstract

LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit. We identify Role Grounding Misattribution, in which an attacker-designed interaction causes the victim agent to incorrectly record a preference from the attacker as a preference of the victim's user. Specifically, we are the first to formalize Role Grounding Attack (RGA), an attack model that exploits this vulnerability through nested conversation structures. In the memory formation stage, an experience-sharing post quoting a conversation between the attacker agent and its user is published, which explicitly identifies its publisher. The inner conversation expresses preferences from the user of the attacker agent. When the victim agent remembers preferences from this nested structure, its memory loses the context binding and grounds the attacker's preference to the victim's user instead. In the memory retrieval stage, the misattributed preference persists and influences the victim agent's decisions when it encounters relevant scenarios. We instantiate RGA as tool selection manipulation and privacy leakage. Tool selection manipulation causes a tool-call preference to be misattributed to the victim's user, steering the victim toward an attacker-controlled tool. Privacy leakage extends this by misattributing both a tool-call preference and a task-requirement preference to the victim's user. The victim selects the attacker's tool, then includes sensitive information in tool arguments to satisfy the task requirement. We evaluate RGA across five mainstream memory systems and six language models. Tool manipulation and privacy leakage both achieve high attack success rates. These results call for stronger role grounding.

Figures & tables

Figure 1. Threat scenario. Left: agents acquire benign experiences and preferences. Right: agents misattribute attacker preferences as their own user.

Figure 2. Framework overview. Top: attack construction separates preference from publisher, then packages it in nested dialogue. Bottom-left: memory update with misattribution. Bottom-right: downstream tasks trigger manipulation.

Memory	Model	Tool Manipulation $(\mathrm{ASR}_{\mathrm{sel}},\%)$				Privacy Leakage $(\mathrm{ASR}_{\mathrm{piv}},\%)$
Memory	Model	MPMA	AS	HIM	Ours	MPMA	AS	HIM	Ours
Hindsight	DS-V4-Flash	$28.40\,\pm\,2.01$	$20.80\,\pm\,3.56$	$52.40\,\pm\,6.92$	$\mathbf{79.60}\,\boldsymbol{\pm}\,\mathbf{10.49}$	$1.60\,\pm\,0.70$	$4.02\,\pm\,1.36$	$26.40\,\pm\,9.74$	$\mathbf{68.40}\,\boldsymbol{\pm}\,\mathbf{5.93}$
	DS-V4-Pro	$31.60\,\pm\,5.15$	$20.00\,\pm\,6.72$	$76.80\,\pm\,2.47$	$\mathbf{79.60}\,\boldsymbol{\pm}\,\mathbf{12.34}$	$2.80\,\pm\,2.74$	$3.20\,\pm\,1.84$	$50.80\,\pm\,9.31$	$\mathbf{68.80}\,\boldsymbol{\pm}\,\mathbf{11.32}$
	Qwen3.8-flash	$74.00\,\pm\,10.24$	$\mathbf{91.60}\,\boldsymbol{\pm}\,\mathbf{2.11}$	$67.20\,\pm\,4.76$	$82.40\,\pm\,13.73$	$10.40\,\pm\,4.27$	$12.00\,\pm\,7.36$	$52.00\,\pm\,4.88$	$\mathbf{70.40}\,\boldsymbol{\pm}\,\mathbf{12.08}$
	Qwen3.8-Max	$32.00\,\pm\,7.52$	$83.20\,\pm\,11.67$	$71.20\,\pm\,6.15$	$\mathbf{84.00}\,\boldsymbol{\pm}\,\mathbf{6.58}$	$5.60\,\pm\,1.83$	$21.20\,\pm\,8.63$	$51.60\,\pm\,1.93$	$\mathbf{74.00}\,\boldsymbol{\pm}\,\mathbf{8.47}$
	GPT-5.6-Luna	$84.00\,\pm\,2.91$	$\mathbf{94.00}\,\boldsymbol{\pm}\,\mathbf{3.15}$	$57.20\,\pm\,1.62$	$68.00\,\pm\,11.57$	$0.00\,\pm\,0.00$	$0.00\,\pm\,0.00$	$37.20\,\pm\,3.82$	$\mathbf{50.00}\,\boldsymbol{\pm}\,\mathbf{8.48}$

Table 1. Attack success rates (ASRs, %) under different attack methods. AS means Adversarial Search; HIM means Hidden in Memory. Boldface indicates the highest ASR in the same configuration.

\pm

means standard deviation.

Model	A-MEM					LangMem					Hindsight					MemOS
Model	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours
DS-V4-Flash	0.91	0.00	0.00	5.45	$\mathbf{92.73}$	0.91	0.91	0.00	0.00	$\mathbf{93.64}$	0.00	1.82	2.73	13.64	$\mathbf{70.91}$	0.00	0.00	0.00	2.73	$\mathbf{96.36}$
DS-V4-Pro	0.00	0.00	0.00	6.36	$\mathbf{93.64}$	0.00	0.00	0.00	1.82	$\mathbf{96.36}$	2.73	1.82	0.00	20.00	$\mathbf{72.73}$	0.00	0.00	0.00	1.82	$\mathbf{98.18}$
GPT-5.6-Luna	0.00	2.73	0.00	27.27	$\mathbf{67.27}$	0.00	0.91	0.00	12.73	$\mathbf{79.09}$	0.91	0.91	0.00	30.00	$\mathbf{55.45}$	0.00	0.91	0.00	17.27	$\mathbf{79.09}$
GPT-5.6-Terra	0.00	0.00	0.00	$\mathbf{58.18}$	41.82	0.00	0.00	0.00	$\mathbf{59.09}$	40.91	0.91	1.82	0.00	46.36	$\mathbf{47.27}$	0.00	0.00	0.00	36.36	$\mathbf{63.64}$
Qwen3.8-Flash	0.00	0.91	1.82	10.91	$\mathbf{83.64}$	0.00	0.91	0.00	2.73	$\mathbf{85.45}$	1.82	2.73	3.64	10.91	$\mathbf{74.55}$	0.00	1.82	0.00	1.82	$\mathbf{95.45}$

Table 2. Tool Manipulation Attack Success Rates (

\mathrm{ASR}_{\mathrm{sel}}

, %) across memory and agent LLM backends in environments where multiple attack methods are deployed concurrently. The compared methods include MPMA Best Name (BN), MPMA Best Description (BD), MPMA GAPMA (GAP), and Adversarial Search (AS).

Model	Info.	A-MEM	LangMem	Mem0	Hindsight	MemOS
DS-V4-Flash	E-mail	96.36	98.18	100.00	78.18	96.36
	Phone	96.36	99.09	100.00	80.00	99.09
	ID	94.55	99.09	100.00	84.55	99.09
	API	79.00	89.90	95.34	79.57	90.23
GPT-5.6-Luna	E-mail	57.27	94.55	90.00	48.18	65.45
	Phone	52.73	75.45	70.00	48.18	43.64

Table 3.

\mathrm{ASR}_{\mathrm{piv}}

(%) for four sensitive fields across five memory systems and three models.

Mem0
Stage	Attribution Category
Stage	Direct Misattribution	Ambiguous Attribution	Correct Attribution
$\operatorname{PMR}_{\mathrm{mem}}(k)$	$78.10\pm 3.78$	$3.61\pm 4.94$	$18.30\pm 1.19$
$\operatorname{PMR}_{\mathrm{ctx}}(k)$	$100.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
$\mathrm{ASR}_{\mathrm{sel}}$	$95.01\pm 0.02$	–	–
A-MEM

Table 4. Memory Manipulation Rate and Retrieval Manipulation Rate (%) across different memory mechanisms.

Panel of Figure 3 — Figure 3. Principal attribution labeled by memory systems during memory consolidation. Each bar shows the proportion of attack-payload entries labeled as belonging to the memory system’s own user versus other users.

Panel of Figure 4 — Figure 4. Success rates of user preference injection for our method and Hidden in Memory (HIM) across (a) tool manipulation and (b) privacy leakage.

	Posts		MCP Description
Category	Ours	HIM	Ours	BN	BD	GAPMA	AS
Jobs	no(0.99)	yes(0.98)	no(0.98)	no(0.99)	no(0.98)	no(0.98)	yes(0.99)
Shopping	no(0.98)	yes(0.99)	no(0.98)	no(0.99)	no(0.99)	no(0.98)	yes(0.99)
Papers	no(0.99)	yes(0.99)	no(0.98)	no(0.98)	no(0.99)	no(0.98)	yes(0.98)
Housing	no(0.99)	yes(0.97)	no(0.98)	no(0.99)	no(0.99)	no(0.98)	yes(0.99)
Weather	no(0.99)	yes(0.99)	no(0.98)	no(0.98)	no(0.98)	no(0.98)	yes(0.98)

Table 5. Security review results across five categories. Values in parentheses denote detection confidence.

Variant	DM	AA	CA
Full	$78.10\pm 3.78$	$3.61\pm 4.94$	$18.30\pm 1.19$
w/o PPD	$4.76\pm 1.65$	$2.86\pm 1.43$	$92.38\pm 2.18$
w/o NCC	$2.86\pm 0.95$	$1.90\pm 1.65$	$95.24\pm 1.43$

Table 6. Ablation of RGA on Mem0 (

\operatorname{PMR}_{\mathrm{mem}}(k)

, %).

Panel of Figure 5 — Figure 5. The ablation task-requirement preference. TM: tool selection manipulation and PL: privacy leakage

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 517 papers

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning
May 24, 2026Xuanye Zhang, Yongsen Zheng, Zhuqin Xu +5LLM Agent Security Tool-Using Agents
Nanyang Technological University, Singapore
Memory-Induced Tool-Drift in LLM Agents
May 24, 2026Mahavir Dabas, Jihyun Jeong, Ming Jin +1AI Agent Benchmarks LLM Tool Use
Virginia Tech
Hidden in Memory: Sleeper Memory Poisoning in LLM Agents
May 14, 2026Sidharth Pulipaka, Stanislau Hlebik, Leonidas Raghav +4Persistent Memory for Language Models LLM Security
1SPAR · 2ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center · 3APTA +1

May 24, 2026J/K move · Enter open · S save

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning

Xuanye Zhang, Yongsen Zheng, Zhuqin Xu, Kaiyu Zhou+4

Nanyang Technological University, Singapore

LLM Agent Security Tool-Using Agents Agent Memory Poisoning Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

LLM-driven agents are capable of selecting external tools to complete users' tasks. However, attackers could compromise such process, steering agents toward inappropriate/wrong tools and enabling malicious actions. Most existing attacks primarily manipulate the tool metadata, which is easily detectable by auditing and may lose effectiveness as modern agents increasingly adopt memory modules to refine tool selection policies through accumulated experience. This paper proposes MemMorph, the first attack that bias tool selection by poisoning the agent's long-term memory. Rather than explicitly dictating the tool invocation decision, MemMorph injects a small number of crafted records that are disguised as technical facts, incident reports, and operational policies. These poisoned records reshape the agent's contextual perception and decision-making process, leading it to autonomously infer and select the tool preferred by the attacker. Experiments across 3 benchmarks, 10 agent backbones, and 3 memory-module implementations show that MemMorph achieves up to 85.9% attack success rate with only three injected records, outperforming the strongest baseline by up to 25% while retaining potency under 3 representative defenses. Our findings expose long-term memory as a critical and under-explored attack surface in tool-augmented agents, urging the development of memory-level integrity safeguards.

Abstract

AI Agent Security

Explore similar work

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning

Memory-Induced Tool-Drift in LLM Agents

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents