Memory Agent Benchmarks

Follow this topic Ask about this topic

These benchmarks test whether agents can remember facts and experiences across sessions and use them to answer questions, personalize responses, or guide actions. They compare long-context prompting and retrieval-augmented generation with external memory such as summaries, profiles, timelines, and graphs, often in interactive, streaming, or multimodal settings. Tests increasingly probe temporal updates, conflicting evidence, forgetting, privacy, and the separate stages of memory formation, retrieval, and use, while also measuring cost and latency. Results show that no single approach works reliably across tasks: simple retrieval and long-context baselines remain competitive, and accurately retrieving information does not guarantee that an agent will recognize when it is relevant, current, or safe to use.

Latest papers

164 papers42 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
May 29, 2026Rosario Forte, Giuseppe Lando, Antonino FurnariVLM Evaluation Streaming Video Understanding
Department of Mathematics and Computer Science University of Catania
Selective QA over Conflicting Multi-Source Personal Memory: A Diagnostic Testbed and Method Comparison
May 28, 2026Tiancheng Yang, Matthias Schonlau, Ilia SucholutskyConversational Memory Question Answering
Entity-Collision: A Stratified Protocol for Attributing Retrieval Lift in Agent Memory
May 28, 2026Youwang DengAgent Memory Text Embedding Evaluation
Independent Researcher
WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction
May 28, 2026Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu +14Multimodal Agent Evaluation Multimodal Memory
University of California, Santa Barbara · Johns Hopkins University · Carnegie Mellon University +3
MemFail: Stress-Testing Failure Modes of LLM Memory Systems
May 26, 2026Ishir Garg, Neel Kolhe, Dawn Song +1LLM Agent Memory LLM Memory
University of California, Berkeley
Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents
May 25, 2026Yeonjun In, Wonjoong Kim, Sangwu Park +2Memorization in Language Models Persistent Memory for Language Models
KAIST
MemGym: a Long-Horizon Memory Environment for LLM Agents
May 20, 2026Wujiang Xu, Yu Wang, Kai Mei +8LLM Agent Memory LLM Agent Evaluation
1Rutgers University · 2Capital One · 3Princeton University +1
MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems
May 18, 2026Hyunji Lee, Justin Chih-Yao Chen, Joykirat Singh +3Agent Memory Long-Horizon Agent Evaluation
UNC Chapel Hill · The University of Texas at Austin
EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
May 18, 2026Yuyao Wang, Zhongjian Zhang, Mo Chi +7LLM Agent Memory LLM Agent Evaluation
Hong Kong University of Science and Technology (Guangzhou) · Beijing University of Posts and Telecommunications · Beijing Institute of Technology +1
SocialMemBench: Are AI Memory Systems Ready for Social Group Settings?
May 18, 2026Olukunle OwolabiAgent Memory Conversational Memory
Independent Researcher
GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
May 14, 2026Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang +3Conversational Memory LLM Agent Evaluation
UC Santa Barbara · Microsoft
LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
May 12, 2026Di Wu, Zixiang Ji, Asmi Kawatkar +4Computer-Use Agent Benchmarks Retrieval-Augmented Generation
University of California, Los Angeles
MEME: Multi-entity & Evolving Memory Evaluation
May 12, 2026Seokwon Jung, Alexander Rubinstein, Arnas Uselis +2Agent Memory Persistent Memory for Language Models
1KAIST AI · 2Tübingen AI Center, University of Tübingen · 3NAVER AI Lab
MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare
May 12, 2026Yihao Wang, Haoran Xu, Renjie Gu +10Agent Memory Healthcare
Zhejiang University · Ant Group · Alibaba Group +1
Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval
May 11, 2026Jeffrey FlyntConversational Memory LLM Memory
Independent Researcher
When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory
May 8, 2026Jiaqi Shao, Yiyi Lu, Yunzhen Zhang +1Agent Memory AI Agent Evaluation
Hong Kong University of Science and Technology, Hong Kong · Duke Kunshan University, China
STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?
May 7, 2026Hanxiang Chao, Yihan Bai, Rui Sheng +2Agent Memory LLM Agent Evaluation
Wuhan University1, Wuhan, China1 · The Hong Kong University of Science and Technology3, Hong Kong, China23 · The Chinese University of Hong Kong2, Hong Kong, China23
Trojan Hippo Bench: A Dynamic Benchmark for Persistent Memory Attacks and Defenses in LLM Agents
May 3, 2026Debeshee Das, Julien Piet, Darya Kaviani +3LLM Agent Security AI Agent Security Benchmarks
ETH Zürich · UC Berkeley · Snyk
EngramaBench: Evaluating Long-Term Conversational Memory with Structured Graph Retrieval
Apr 23, 2026Julian AcunaLLM Evaluation Memorization in Language Models
From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
Apr 21, 2026Md Nayem Uddin, Kumar Shubham, Eduardo Blanco +2Long-Horizon Agent Evaluation AI Agent Benchmarks
Arizona State University · Genies · University of Arizona
Evaluating Memory Capability in Continuous Lifelog Scenario
Apr 13, 2026Jianjie Zheng, Zhichen Liu, Zhanyu Shen +6LLM Memory Memory Agent Benchmarks
Southern University of Science and Technology · Shenzhen University · Shanghai Jiao Tong University +1
ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks
Apr 13, 2026Samuel Sameer TanguturiMemory Agent Benchmarks Agent Benchmarks
Kenotic Labs
ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models
Apr 9, 2026Chonghan Qin, Xiachong Feng, Weitao Ma +2Memorization in Language Models Memory Agent Benchmarks
The University of Hong Kong · Harbin Institute of Technology
A-MBER: Affective Memory Benchmark for Emotion Recognition
Apr 8, 2026Deliang Wen, Ke Sun, Yu WangEmotion Recognition in Conversations Emotion Recognition
FileGram: Grounding Agent Personalization in File-System Behavioral Traces
Apr 6, 2026Shuai Liu, Shulin Tian, Kairui Hu +6Agent Memory Personal AI Agents
S-Lab, Nanyang Technological University · Synvo AI
Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
Apr 2, 2026Yanchen Wu, Tenghui Lin, Yingli Zhou +7LLM Agent Memory Agent Memory Management
CUHK-Shenzhen · CUHK · HITSZ +2
MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization
Mar 26, 2026Weizhi Zhang, Xiaokai Wei, Wei-Chieh Huang +4Memory Agent Benchmarks LLM Personalization
Roblox · University of Illinois Chicago · University of Cambridge
VehicleMemBench: An Executable Benchmark for Multi-User Long-Term Memory in In-Vehicle Agents
Mar 25, 2026Yuhao Chen, Yi Xu, Xinyun Ding +7Memory Agent Benchmarks Memory-Augmented Agents
University of Science and Technology of China · iFLYTEK Research · Xiamen University +1
PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
Mar 24, 2026Shuochen Liu, Junyi Zhu, Long Shu +11Memory Agent Benchmarks Long-Term Conversational Memory
University of Science and Technology of China · ESAT-PSI, KU Leuven · City University of Hong Kong +2
MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios
Mar 23, 2026Yihang Ding, Wanke Xia, Yiting Zhao +5LLM Memory Memory Agent Benchmarks
Tsinghua University · Renmin University of China · CASIA

May 29, 2026J/K move · Enter open · S save

EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision

Rosario Forte, Giuseppe Lando, Antonino Furnari

Department of Mathematics and Computer Science University of Catania

VLM Evaluation Streaming Video Understanding Episodic Memory Egocentric Video Understanding Memory Agent Benchmarks Agent Memory Management

Open paper AskPDF arXiv

Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce Egostream, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. \egostream organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, and offloading. Experiments within a unified Qwen3-VL backbone reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, while quantized offloading rescues ultra-long-term recall. Ultimately, all mechanisms operate well below real-time (>1s per frame), and top performing methods ceil at about 45% accuracy, exposing critical gaps in current architectures. Egostream provides the diagnostic testbed needed to close these gaps. Project website, news and updates at: https://saroo25.github.io/Egostream/