Memory Agent Benchmarks

Follow this topic Ask about this topic

These benchmarks test whether agents can remember facts and experiences across sessions and use them to answer questions, personalize responses, or guide actions. They compare long-context prompting and retrieval-augmented generation with external memory such as summaries, profiles, timelines, and graphs, often in interactive, streaming, or multimodal settings. Tests increasingly probe temporal updates, conflicting evidence, forgetting, privacy, and the separate stages of memory formation, retrieval, and use, while also measuring cost and latency. Results show that no single approach works reliably across tasks: simple retrieval and long-context baselines remain competitive, and accurately retrieving information does not guarantee that an agent will recognize when it is relevant, current, or safe to use.

Latest papers

164 papers42 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
Jul 29, 2026Xuanze Chen, Xukang Xie, Wentao Fu +3Agent Memory Poisoning Memory Agent Benchmarks
Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China · Binjiang Institute of Artificial Intelligence, ZJUT, Hangzhou 310056, China · College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023, China
Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data
Jul 29, 2026Lingyang Zeng, Guangze Chen, Kaichen Yu +9Personality Modeling in Language Models Memory Agent Benchmarks
East China Normal University · The Chinese University of Hong Kong
Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability
Jul 29, 2026Sizhe Zhou, Sheldon Yu, Hui Wei +8Agent Memory LLM Agent Memory
University of Illinois Urbana-Champaign · University of California, San Diego · University of California, Merced +2
Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory
Jul 27, 2026Ruizhe Li, Mingxuan Du, Benfeng Xu +1Agent Memory In-Context Retrieval
University of Science and Technology of China · Metastone Technology
Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings
Jul 24, 2026Quentin SpencerLong-Horizon Agent Evaluation Memory Agent Benchmarks
Independent Researcher
RUMBA: Russian User Memory Benchmark
Jul 23, 2026Elizaveta Shevtsova, Inna Glebkina, Mark Baushenko +2Temporal Reasoning in Language Models Temporal Reasoning
1DAIMLD
MemTools: A Unified Research Framework for Interoperable Agent Memory
Jul 23, 2026Chengfeng Zhao, Jinhui Chen, Sirui Liang +4Agent Memory Memory Agent Benchmarks
Institute of Automation, CAS · University of Chinese Academy of Sciences · Zhongguancun Institute of Artificial Intelligence +1
Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration
Jul 18, 2026Maksim Sheverev, David Finkelstein, Sergey NikolenkoLLM Agent Memory AI Agent Benchmarks
Quantellence Research · St. Petersburg Department of the Steklov Institute of Mathematics, St. Petersburg, Russia · St. Petersburg State University, St. Petersburg, Russia
RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts
Jul 18, 2026Mihir Shriniwas AryaCompositional Reasoning LLM Agent Memory
Department of Computer Science and Engineering RV College of Engineering
MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
Jul 14, 2026Xixuan Hao, Zeyu Zhang, Zehao Lin +6Conversational Memory Persistent Memory for Language Models
MemTensor (Shanghai) Technology Co., Ltd. · The Hong Kong University of Science and Technology (Guangzhou) · Renmin University of China
PM-Bench: Evaluating Prospective Memory in LLM Agents
Jul 14, 2026Genglin Liu, Saadia GabrielLLM Agent Memory LLM Agent Evaluation
University of California, Los Angeles
The Compliance Trap: Diagnosing How AI Agents Consume Conflicting Memory
Jul 12, 2026Yixiong Chen, Xinyi Bai, Alan YuilleWeb Agent Benchmarks Long-Horizon Agent Evaluation
1Johns Hopkins University · 2Cornell University
S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
Jul 2, 2026Xiaodong Wang, Xuanyi Zhao, Pedro Rodriguez +7Temporal Video Grounding Episodic Memory
1FAIR, Meta · 2Reality Labs, Meta
MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
Jul 1, 2026Zhishang Xiang, Zerui Chen, Yunbo Tang +5Agent Memory LLM Sycophancy
Xiamen University · Jilin University
MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation
Jun 29, 2026Kuan WangAgent Memory LLM Agent Evaluation
DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection
Jun 25, 2026Yujin Tang, Chenming Shang, Ruize Xu +1Multimodal Memory Long-Horizon Agent Evaluation
Dartmouth College
MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery
Jun 23, 2026Enze Ma, Yufan Zhou, Wei-Chieh Huang +7Agent Memory AI Agent Benchmarks
University of Illinois Chicago · KU Leuven · UC San Diego
DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings
Jun 22, 2026Wenya Xie, Shengming Zhou, Zelin Li +9Agent Memory LLM Agent Memory
University of Minnesota · University of North Carolina at Charlotte · Johns Hopkins University +2
RoboMME-Interference: Benchmarking Robot Memory Under Interference
Jun 21, 2026Soumil RathiAgent Memory Memory Agent Benchmarks
Independent Researcher
When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense
Jun 20, 2026Pranav SinghAgent Memory Bayesian Inference
Indian Institute of Technology Ropar
GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents
Jun 17, 2026Zhe Ren, Yibo Yang, Yimeng Chen +7LLM Agent Memory Agent Memory Governance
School of Artificial Intelligence, Jilin University · 2Shanghai Jiao Tong University · 3King Abdullah University of Science and Technology (KAUST) +2
MemTrace: Probing What Final Accuracy Misses in Long-Term Memory
Jun 15, 2026Xianxuan Long, Zhikai Chen, Shenglai Zeng +3Temporal Reasoning in Language Models LLM Agent Memory
Michigan State University · Case Western Reserve University
Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations
Jun 14, 2026Dongxu YangMemory Agent Benchmarks Memory-Augmented Agents
DeepLethe
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance
Jun 12, 2026Guanming Liu, Yuqi Ren, Hansu Gu +5Agent Memory Long-Horizon Agent Evaluation
1Fudan University · 2Amazon
M $^3$ Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
Jun 5, 2026Zhengjun Huang, Wenxuan Liu, Zhoujin Tian +6Multimodal Understanding Multimodal Grounding
The Hong Kong University of Science and Technology · Peng Cheng Laboratory · Beijing University of Chemical Technology +4
Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
Jun 5, 2026Liuyin WangLong-Context QA LLM Agent Memory
Independent Researcher
SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents
Jun 4, 2026Wenxuan Wang, Haoyu Sun, Fukuan Hou +4Persistent Memory for Language Models Long-Horizon Agent Evaluation
Harbin Institute of Technology · Shanghai AI Laboratory · Tongji University +4
LifeSide: Benchmarking Agents as Lifelong Digital Companions
Jun 3, 2026Yuqian Wu, Zhijie Deng, Wei Chen +8Long-Horizon Agent Evaluation AI Companions
Hong Kong University of Science and Technology (Guangzhou) · Tencent · Hong Kong University of Science and Technology
eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents
Jun 2, 2026A. Haroon Rasheed, Maria KabtoulLLM Agent Memory Memory Agent Benchmarks
MIND · Automatika Robotics Inria, France · CHROMA
Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations
May 30, 2026Adril Putra Merin, David Anugraha, Ayu Purwarianti +1Agent Memory Temporal Reasoning
Institut Teknologi Bandung · Stanford University · Capital One

Jul 29, 2026J/K move · Enter open · S save

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair

Xuanze Chen, Xukang Xie, Wentao Fu, Jiajun Zhou+2

Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China · Binjiang Institute of Artificial Intelligence, ZJUT, Hangzhou 310056, China · College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023, China

Agent Memory Poisoning Memory Agent Benchmarks Agent Memory Management

Open paper AskPDF arXiv

Memory systems allow agents to retain and reuse information from past interactions, but they can also let malicious content persist. A malicious instruction crafted by an attacker may be stored in long-term memory, recalled much later, and quietly shape a real action. Recent benchmarks increasingly examine agent memory security, yet few trace the same malicious semantics across persistence, downstream consequences, and selective repair under diverse memory-backend comparisons. To address this gap, we introduce MemSecBench, a task-grounded benchmark for the lifecycle security of agent memory systems. It contains 310 cases drawn from 48 realistic contexts across code and science, daily life, and office work. Each case follows a controlled Write--Execute--Forget protocol in an isolated runtime under an exact agent configuration, defined by an agent harness, a memory backend, and an LLM backend. Evidence-based adjudication combines a deterministic write check, checkpoint-specific judge-model evaluations, and programmatic gates across seven lifecycle checkpoints. The experimental design spans a 24-configuration matrix of two agent harnesses, four memory backends, and three LLM backends. Across all 24 configurations, malicious memory persists in 84.2% of all cases, and the full Write--Execute chain succeeds in 50.3%. Among successfully poisoned cases, 59.6% complete the full Execute chain, while 56.1% achieve selective repair.Compared with matched Native configurations, the largest absolute differences are 16.1 percentage points for end-to-end attack success and 41.3 percentage points for selective repair. These descriptive contrasts indicate that the evaluated memory system stacks differ in lifecycle security, both in the propagation of malicious memory and in selective repair after successful memory poisoning.