cs.CRMay 8, 2026

Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful LLM agents

Authors: Jun Wen Leong

Abstract

Persistent memory in LLM agents creates an attack surface that production safety classifiers do not observe: the payload enters via RAG retrieval and persists across sessions via tool-mediated memory. We evaluate six defenses across four architectural layers against delayed-trigger attacks on nine open-source models (5,040 runs, N=40 per condition). Five of six defenses fail: input-level filters never see the payload (it enters via RAG, not user input); retrieval-level classifiers observe it but cannot distinguish compliance-framed injection from legitimate policy; instruction-level hardening is overridden by the stored rule's compliance framing. Only tool-gating at the memory layer (Memory Sandbox) reduces ASR to 0% for eight of nine models, with zero utility cost. A reasoning model inverts this defense via goal-directed RAG fallback, a mechanism that replicates cross-family on Bedrock. A reasoning-mode ablation reveals a double dissociation: no single sandbox implementation is safe across both reasoning and non-reasoning model classes. We resolve this with a content-layer proof-of-concept (RATG), validated on non-reasoning models. A loaded-corpus frontier evaluation (21 models, 3 providers, N=40) overturns an initial empty-corpus screen showing 0/210 exfiltrations: that was a threat-model artifact, not model safety. Under realistic conditions, Gemini 3.1 Pro Preview exfiltrates at 95% ASR, GPT-5.1 regresses to 22.5% relative to GPT-5 (5%), and Anthropic blocks at the injection layer (0-17.5% storage, 0% ASR). Nearly all OpenAI and Gemini models store the rule at 100% regardless of execution resistance, creating supply-chain risk in shared-memory deployments. Defense effectiveness is determined by architectural layer and reasoning capability, not classifier quality.

Explore similar work

CardsList
Jul 6, 2026cs.CR

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.
Neeraj Karamchandani, Piyush Nagasubramaniam, Sencun Zhu +1
Language Model Evasion AttacksAgent MemoryLLM Reasoning MethodsBenign
AskPDF
May 14, 2026cs.CR

Hidden in Memory: Sleeper Memory Poisoning in LLM Agents

Large language models are increasingly augmented with persistent memory, allowing assistants to store user-specific information across sessions for personalization and continuity. This statefulness introduces a new security risk: adversarial content can corrupt what an assistant remembers and thereby influence future interactions. We propose and study sleeper memory poisoning, a delayed attack in which an adversary manipulates external context, such as a document, webpage, or repository, to cause the assistant to store a fabricated memory about the user. Unlike conventional prompt injection, the attack can remain dormant and re-emerge across multiple later conversations. We evaluate the full attack pipeline: whether poisoned memories are written, later retrieved, and ultimately used to steer the following conversations. Across stateful LLM assistants, poisoned memories were added up to 99.8% on GPT-5.5 and 95% on Kimi-K2.6. Crucially, among successful retrievals, poisoned memories cause attacker-intended agentic actions in 60-89% of evaluations across models. These results show that persistent memory can act as a long-term attack surface across multiple future conversations.
Sidharth Pulipaka, Stanislau Hlebik, Leonidas Raghav +4
Large Language Model AgentsAttack Surface
AskPDF
Jul 31, 2026cs.CR

Memory Provenance Laundering in LLM Agents: A Non-Amplification Firewall for Persistent Memory

Long-term memory lets large language model(LLM) agents reuse prior preferences and work flows, but it also turns untrusted observations into persistent action context. We identify memory provenance laundering: during LLM-based memory consolidation, an external observation may be rewritten as apparent user history or workflow support, preserving an action trigger while erasing the low-trust source that should limit its authority. Existing prompt filters, content sanitizers, and tool guards do not enforce source-authority non-amplification after lossy memory consolidation. We formalize this boundary and instantiate it as Provenance-Preserving Memory Fire wall (PPMF), a lightweight memory middleware that preserves platform-maintained provenance and authorizes tool calls by matching action risk to the authority of action-relevant memories. In our schema-grounded evaluation with fixed risk policies, vulnerable consolidated memories reach up to 1.000 attack success rate(ASR); with intact platform-maintained provenance, confirmation, and risk labels, no evaluated unauthorized high-risk action passes the PPMF gate while confirmed benign actions and targeted low-risk memory use remain executable.
Jinghan Xu, Yiyong Xiao, Wanru Shao +2
Large Language Model AgentsModel ProvenancePersistenceProvenance
AskPDF