Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

Haoran Bu; Yiting Shen; Litian Zhang; Ruihan Hu; Xi Zhang

cs.CROct 28, 2026

Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

Authors: Haoran Bu, Yiting Shen, Litian Zhang, Ruihan Hu, Xi Zhang

Organizations: Beijing University of Posts and Telecommunications, Beijing, China

Abstract

LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit. We identify Role Grounding Misattribution, in which an attacker-designed interaction causes the victim agent to incorrectly record a preference from the attacker as a preference of the victim's user. Specifically, we are the first to formalize Role Grounding Attack (RGA), an attack model that exploits this vulnerability through nested conversation structures. In the memory formation stage, an experience-sharing post quoting a conversation between the attacker agent and its user is published, which explicitly identifies its publisher. The inner conversation expresses preferences from the user of the attacker agent. When the victim agent remembers preferences from this nested structure, its memory loses the context binding and grounds the attacker's preference to the victim's user instead. In the memory retrieval stage, the misattributed preference persists and influences the victim agent's decisions when it encounters relevant scenarios. We instantiate RGA as tool selection manipulation and privacy leakage. Tool selection manipulation causes a tool-call preference to be misattributed to the victim's user, steering the victim toward an attacker-controlled tool. Privacy leakage extends this by misattributing both a tool-call preference and a task-requirement preference to the victim's user. The victim selects the attacker's tool, then includes sensitive information in tool arguments to satisfy the task requirement. We evaluate RGA across five mainstream memory systems and six language models. Tool manipulation and privacy leakage both achieve high attack success rates. These results call for stronger role grounding.

Figures & tables

Figure 1. Threat scenario. Left: agents acquire benign experiences and preferences. Right: agents misattribute attacker preferences as their own user.

Figure 2. Framework overview. Top: attack construction separates preference from publisher, then packages it in nested dialogue. Bottom-left: memory update with misattribution. Bottom-right: downstream tasks trigger manipulation.

Memory	Model	Tool Manipulation $(\mathrm{ASR}_{\mathrm{sel}},\%)$				Privacy Leakage $(\mathrm{ASR}_{\mathrm{piv}},\%)$
Memory	Model	MPMA	AS	HIM	Ours	MPMA	AS	HIM	Ours
Hindsight	DS-V4-Flash	$28.40\,\pm\,2.01$	$20.80\,\pm\,3.56$	$52.40\,\pm\,6.92$	$\mathbf{79.60}\,\boldsymbol{\pm}\,\mathbf{10.49}$	$1.60\,\pm\,0.70$	$4.02\,\pm\,1.36$	$26.40\,\pm\,9.74$	$\mathbf{68.40}\,\boldsymbol{\pm}\,\mathbf{5.93}$
	DS-V4-Pro	$31.60\,\pm\,5.15$	$20.00\,\pm\,6.72$	$76.80\,\pm\,2.47$	$\mathbf{79.60}\,\boldsymbol{\pm}\,\mathbf{12.34}$	$2.80\,\pm\,2.74$	$3.20\,\pm\,1.84$	$50.80\,\pm\,9.31$	$\mathbf{68.80}\,\boldsymbol{\pm}\,\mathbf{11.32}$
	Qwen3.8-flash	$74.00\,\pm\,10.24$	$\mathbf{91.60}\,\boldsymbol{\pm}\,\mathbf{2.11}$	$67.20\,\pm\,4.76$	$82.40\,\pm\,13.73$	$10.40\,\pm\,4.27$	$12.00\,\pm\,7.36$	$52.00\,\pm\,4.88$	$\mathbf{70.40}\,\boldsymbol{\pm}\,\mathbf{12.08}$
	Qwen3.8-Max	$32.00\,\pm\,7.52$	$83.20\,\pm\,11.67$	$71.20\,\pm\,6.15$	$\mathbf{84.00}\,\boldsymbol{\pm}\,\mathbf{6.58}$	$5.60\,\pm\,1.83$	$21.20\,\pm\,8.63$	$51.60\,\pm\,1.93$	$\mathbf{74.00}\,\boldsymbol{\pm}\,\mathbf{8.47}$
	GPT-5.6-Luna	$84.00\,\pm\,2.91$	$\mathbf{94.00}\,\boldsymbol{\pm}\,\mathbf{3.15}$	$57.20\,\pm\,1.62$	$68.00\,\pm\,11.57$	$0.00\,\pm\,0.00$	$0.00\,\pm\,0.00$	$37.20\,\pm\,3.82$	$\mathbf{50.00}\,\boldsymbol{\pm}\,\mathbf{8.48}$

Table 1. Attack success rates (ASRs, %) under different attack methods. AS means Adversarial Search; HIM means Hidden in Memory. Boldface indicates the highest ASR in the same configuration.

\pm

means standard deviation.

Model	A-MEM					LangMem					Hindsight					MemOS
Model	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours	BN	BD	GAP	AS	Ours
DS-V4-Flash	0.91	0.00	0.00	5.45	$\mathbf{92.73}$	0.91	0.91	0.00	0.00	$\mathbf{93.64}$	0.00	1.82	2.73	13.64	$\mathbf{70.91}$	0.00	0.00	0.00	2.73	$\mathbf{96.36}$
DS-V4-Pro	0.00	0.00	0.00	6.36	$\mathbf{93.64}$	0.00	0.00	0.00	1.82	$\mathbf{96.36}$	2.73	1.82	0.00	20.00	$\mathbf{72.73}$	0.00	0.00	0.00	1.82	$\mathbf{98.18}$
GPT-5.6-Luna	0.00	2.73	0.00	27.27	$\mathbf{67.27}$	0.00	0.91	0.00	12.73	$\mathbf{79.09}$	0.91	0.91	0.00	30.00	$\mathbf{55.45}$	0.00	0.91	0.00	17.27	$\mathbf{79.09}$
GPT-5.6-Terra	0.00	0.00	0.00	$\mathbf{58.18}$	41.82	0.00	0.00	0.00	$\mathbf{59.09}$	40.91	0.91	1.82	0.00	46.36	$\mathbf{47.27}$	0.00	0.00	0.00	36.36	$\mathbf{63.64}$
Qwen3.8-Flash	0.00	0.91	1.82	10.91	$\mathbf{83.64}$	0.00	0.91	0.00	2.73	$\mathbf{85.45}$	1.82	2.73	3.64	10.91	$\mathbf{74.55}$	0.00	1.82	0.00	1.82	$\mathbf{95.45}$

Table 2. Tool Manipulation Attack Success Rates (

\mathrm{ASR}_{\mathrm{sel}}

, %) across memory and agent LLM backends in environments where multiple attack methods are deployed concurrently. The compared methods include MPMA Best Name (BN), MPMA Best Description (BD), MPMA GAPMA (GAP), and Adversarial Search (AS).

Model	Info.	A-MEM	LangMem	Mem0	Hindsight	MemOS
DS-V4-Flash	E-mail	96.36	98.18	100.00	78.18	96.36
	Phone	96.36	99.09	100.00	80.00	99.09
	ID	94.55	99.09	100.00	84.55	99.09
	API	79.00	89.90	95.34	79.57	90.23
GPT-5.6-Luna	E-mail	57.27	94.55	90.00	48.18	65.45
	Phone	52.73	75.45	70.00	48.18	43.64

Table 3.

\mathrm{ASR}_{\mathrm{piv}}

(%) for four sensitive fields across five memory systems and three models.

Mem0
Stage	Attribution Category
Stage	Direct Misattribution	Ambiguous Attribution	Correct Attribution
$\operatorname{PMR}_{\mathrm{mem}}(k)$	$78.10\pm 3.78$	$3.61\pm 4.94$	$18.30\pm 1.19$
$\operatorname{PMR}_{\mathrm{ctx}}(k)$	$100.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
$\mathrm{ASR}_{\mathrm{sel}}$	$95.01\pm 0.02$	–	–
A-MEM

Table 4. Memory Manipulation Rate and Retrieval Manipulation Rate (%) across different memory mechanisms.

Panel of Figure 3 — Figure 3. Principal attribution labeled by memory systems during memory consolidation. Each bar shows the proportion of attack-payload entries labeled as belonging to the memory system’s own user versus other users.

Panel of Figure 4 — Figure 4. Success rates of user preference injection for our method and Hidden in Memory (HIM) across (a) tool manipulation and (b) privacy leakage.

	Posts		MCP Description
Category	Ours	HIM	Ours	BN	BD	GAPMA	AS
Jobs	no(0.99)	yes(0.98)	no(0.98)	no(0.99)	no(0.98)	no(0.98)	yes(0.99)
Shopping	no(0.98)	yes(0.99)	no(0.98)	no(0.99)	no(0.99)	no(0.98)	yes(0.99)
Papers	no(0.99)	yes(0.99)	no(0.98)	no(0.98)	no(0.99)	no(0.98)	yes(0.98)
Housing	no(0.99)	yes(0.97)	no(0.98)	no(0.99)	no(0.99)	no(0.98)	yes(0.99)
Weather	no(0.99)	yes(0.99)	no(0.98)	no(0.98)	no(0.98)	no(0.98)	yes(0.98)

Table 5. Security review results across five categories. Values in parentheses denote detection confidence.

Variant	DM	AA	CA
Full	$78.10\pm 3.78$	$3.61\pm 4.94$	$18.30\pm 1.19$
w/o PPD	$4.76\pm 1.65$	$2.86\pm 1.43$	$92.38\pm 2.18$
w/o NCC	$2.86\pm 0.95$	$1.90\pm 1.65$	$95.24\pm 1.43$

Table 6. Ablation of RGA on Mem0 (

\operatorname{PMR}_{\mathrm{mem}}(k)

, %).

Panel of Figure 5 — Figure 5. The ablation task-requirement preference. TM: tool selection manipulation and PL: privacy leakage

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 517 papers

May 24, 2026

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning

LLM-driven agents are capable of selecting external tools to complete users' tasks. However, attackers could compromise such process, steering agents toward inappropriate/wrong tools and enabling malicious actions. Most existing attacks primarily manipulate the tool metadata, which is easily detectable by auditing and may lose effectiveness as modern agents increasingly adopt memory modules to refine tool selection policies through accumulated experience. This paper proposes MemMorph, the first attack that bias tool selection by poisoning the agent's long-term memory. Rather than explicitly dictating the tool invocation decision, MemMorph injects a small number of crafted records that are disguised as technical facts, incident reports, and operational policies. These poisoned records reshape the agent's contextual perception and decision-making process, leading it to autonomously infer and select the tool preferred by the attacker. Experiments across 3 benchmarks, 10 agent backbones, and 3 memory-module implementations show that MemMorph achieves up to 85.9% attack success rate with only three injected records, outperforming the strongest baseline by up to 25% while retaining potency under 3 representative defenses. Our findings expose long-term memory as a critical and under-explored attack surface in tool-augmented agents, urging the development of memory-level integrity safeguards.

Xuanye Zhang, Yongsen Zheng, Zhuqin Xu +5

Nanyang Technological University, Singapore

LLM Agent Security Tool-Using Agents Agent Memory Poisoning Memory-Augmented Language Model Agents

Ask PDF

May 24, 2026

Memory-Induced Tool-Drift in LLM Agents

LLM agents increasingly combine long-term memory for personalization with tool calling for taking actions. We study a failure of this combination: personality-driven preferences stored in memory (e.g., cost-consciousness, impatience, risk tolerance) silently shifting tool-call parameters in contexts where they do not apply. We call this memory-induced tool-drift and stress-test it with MEMDRIFT, an iteratively refined benchmark of 105 scenarios spanning five bias dimensions and seven professional domains. Measured as excess parameter degradation over a neutral-memory baseline, biased memories in MEMDRIFT induce drift in all eight models we evaluate, raising degradation rates to 0.38--0.83 versus 0.05--0.12 for neutral memories. The drift persists under three widely used memory frameworks. A vulnerability audit of 6,062 tools across 288 MCP servers demonstrates that real-world tools are also susceptible to this failure mode. Mechanistically, we find that biased memories act as implicit steering vectors along the same direction as explicit behavioral instructions and draw attention away from task-relevant context. Neither prompt-based defenses nor relevance filters offer a complete solution, motivating defenses at the intersection of memory management and tool calling.

Mahavir Dabas, Jihyun Jeong, Ming Jin +1

Virginia Tech

AI Agent Benchmarks LLM Tool Use Agent Memory Management LLM Agent Reliability

Ask PDF

May 14, 2026

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents

Large language models are increasingly augmented with persistent memory, allowing assistants to store user-specific information across sessions for personalization and continuity. This statefulness introduces a new security risk: adversarial content can corrupt what an assistant remembers and thereby influence future interactions. We propose and study sleeper memory poisoning, a delayed attack in which an adversary manipulates external context, such as a document, webpage, or repository, to cause the assistant to store a fabricated memory about the user. Unlike conventional prompt injection, the attack can remain dormant and re-emerge across multiple later conversations. We evaluate the full attack pipeline: whether poisoned memories are written, later retrieved, and ultimately used to steer the following conversations. Across stateful LLM assistants, poisoned memories were added up to 99.8% on GPT-5.5 and 95% on Kimi-K2.6. Crucially, among successful retrievals, poisoned memories cause attacker-intended agentic actions in 60-89% of evaluations across models. These results show that persistent memory can act as a long-term attack surface across multiple future conversations.

Sidharth Pulipaka, Stanislau Hlebik, Leonidas Raghav +4

1SPAR · 2ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center · 3APTA +1

Persistent Memory for Language Models LLM Security Agent Memory Poisoning Memory-Augmented Language Model Agents

Ask PDF

Abstract

AI Agent Security

Explore similar work

MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning

Memory-Induced Tool-Drift in LLM Agents

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents