Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise
Abstract
The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks. By embedding malicious instructions, these exploits manipulate language models to bypass intended behavior and operational boundaries. In traditional chatbots, such attacks are confined to a single session because state memory is not preserved across conversations. However, persistent memory features in modern conversational interfaces enable cross-session attack propagation, establishing a threat vector known as memory injection. This work explores this emerging vulnerability in leading LLM-as-a-service platforms, highlighting its real-world severity and the need for countermeasures. While existing literature focuses on violations tailored to specific platform features, we present a general threat model. We demonstrate that all NIST attack goals are collectively achievable across ChatGPT Instant 5.3, Claude Sonnet 4.6, and Gemini 3 Flash under this model. We also show that memory injections can exploit multiple attack vectors, including personal memories, projects, skills, organization instructions, and Model Context Protocol (MCP)-connected resources. Our findings reveal that all evaluated services can be manipulated to persistently spread disinformation, generate harmful content, or simulate offline unresponsiveness, even in fresh sessions. In the particular case of Claude, shared-memory features allow an attacker to compromise multiple users within an organization and also exfiltrate user prompts. To address these vulnerabilities, we derive defense mechanisms from our proposed threat model and foundational cybersecurity principles. Our analysis indicates that nearly half of these mitigations remain unimplemented across the target services. Datasets and attack artifacts are available at https://github.com/PedroBarcha/Memory-injection.
Figures & tables
| Work | General threat model | Systematic mitigations | Target models | Targeted memory features | Cross-account attack | Integrity violation | General availability breakdown | Privacy compromise | General misuse enablement |
|---|---|---|---|---|---|---|---|---|---|
| SpAIware | ✗ | ✗ | - ChatGPT | ChatGPT: memories | ✗ | ✗ | ✗ | ✓ | ✗ |
| MINJA | ✗ | ✗ | - GPT-4 and 4o - DeepSeek-R1 | Memory features are not native to the model providers’ services | ✗ | ✓ | ✗ | ✓ | ✗ |
| ER-MIA | ✗ | ✗ | - Gemma3:27b - Llama3.2:3b - GPT-oss:20b | Memory features are not native to the model providers’ services | ✗ | ✓ | ✗ | ✗ | ✗ |
| This work | ✓ | ✓ | - ChatGPT Instant 5.3 - Claude Sonnet 4.6 - Gemini 3 Flash | - ChatGPT: memories, projects, MCP - Claude: memories, skills, org. instructions, projects, MCP - Gemini: memories, MCP | ✓ | ✓ | ✓ | ✓ | ✓ |
| Target Model | Misuse enablement | Availability breakdown | Integrity violation | Privacy compromise |
|---|---|---|---|---|
| ChatGPT Instant 5.3 | ✓ | ✓ | ✓ | ✗ |
| Claude Sonnet 4.6 | ✓ + | ✓ + | + | + |
| Gemini 3 Flash | ✓ | ✓ | ✓ | ✗ |
| Service | Memory disabled by default | Blocking memory creation from untrusted sources | Shared-memory restrictions | Filtering | CSRF protections | Global fail-safe mechanism | Open design | Monitoring & Response |
| ChatGPT | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ? |
| Claude | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ? |
| Gemini | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ? |
LLM Security
Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.
See all 324 papers