Hierarchical Memory

Follow this topic Ask about this topic

Hierarchical memory studies how AI systems can organize information across levels, from compact summaries and reusable patterns to detailed episodes and original evidence. Common approaches use trees, graphs, or tiered stores, with temporal or semantic indexing, consolidation, and query-adaptive routing to retrieve only the context a task needs. These methods support long-running conversational and multimodal agents, and are often evaluated on benchmarks such as LoCoMo and LongMemEval. A central challenge is balancing lower storage and context costs against accurate, traceable retrieval: deeper hierarchies do not always improve answers, and summaries can omit details needed for reasoning.

Latest papers

42 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents
Apr 20, 2026Shuqi Cao, Jingyi He, Fei TanPersistent Memory for Language Models Hierarchical Memory
East China Normal University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China
OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning
Apr 18, 2026Zhijia Liang, Jiaming Li, Weikai Chen +3Streaming Video Understanding Retrieval-Augmented Reasoning
Sun Yat-sen University · OPPO AI Center · Shenzhen Loop Area Institute +1
Hierarchical Memory Orchestration for Personalized Persistent Agents
Apr 2, 2026Junming Liu, Yifei Sun, Weihua Cheng +4LLM Agent Memory Persistent Memory for Language Models
Shanghai Artificial Intelligence Laboratory, Shanghai, China · The City University of New York, New York, USA
Multi-Layered Memory Architectures for LLM Agents: An Experimental Evaluation of Long-Term Context Retention
Mar 31, 2026Sunil Tiwari, Payal FofadiyaPersistent Memory for Language Models Long-Term Conversational Memory
Fulloop
Toward a Theory of Hierarchical Memory for Language Agents
Mar 23, 2026Yashar Talebirad, Ali Parsaee, Csongor Y. Szepesvari +2LLM Agent Memory Hierarchical Memory
University of Alberta · York University
Dynamic Sparse Attention: Access Patterns and Architecture
Mar 13, 2026Noam LevyLLM Inference Acceleration Sparse Attention
Intel Corporation.
Multi-Agent Memory from a Computer Architecture Perspective: Visions and Challenges Ahead
Mar 9, 2026Zhongming Yu, Naicheng Yu, Hejia Zhang +5Shared-Memory Multi-Agent Systems Hierarchical Memory
University of California, San Diego San Diego, California, USA · Georgia Institute of Technology Atlanta, Georgia, USA
Enhancing Web Agents with a Hierarchical Memory Tree
Mar 7, 2026Yunteng Tan, Zhi Gao, Xinxiao WuWeb Agents LLM Agent Memory
Beijing Institute of Technology, Beijing 100081, China
HTM-EAR: Importance-Preserving Tiered Memory with Hybrid Routing under Saturation
Feb 27, 2026Shubham Kumar SinghAgent Memory Management Hierarchical Memory
From Lossy to Verified: A Provenance-Aware Tiered Memory for Agents
Feb 20, 2026Qiming Zhu, Shunian Chen, Rui Yu +2Agent Memory Agent Memory Management
School of Data Science, The Chinese University of Hong Kong, Shenzhen.
Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory
Feb 17, 2026Zihao Tang, Xin Yu, Ziyu Xiao +9LLM Memory Long-Term Conversational Memory
Microsoft
Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity
Feb 3, 2026Menglin Xia, Xuchao Zhang, Shantanu Dixit +6Agent Memory Retrieval-Augmented Generation
M365 Research, Microsoft.

Apr 20, 2026J/K move · Enter open · S save

HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

Shuqi Cao, Jingyi He, Fei Tan

East China Normal University, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China

Persistent Memory for Language Models Hierarchical Memory Long-Term Conversational Memory Agent Memory Management

Open paper AskPDF arXiv

Long-term conversational large language model (LLM) agents require memory systems that can recover relevant evidence from historical interactions without overwhelming the answer stage with irrelevant context. However, existing memory systems, including hierarchical ones, still often rely solely on vector similarity for retrieval. It tends to produce bloated evidence sets: adding many superficially similar dialogue turns yields little additional recall, but lowers retrieval precision, increases answer-stage context cost, and makes retrieved memories harder to inspect and manage. To address this, we propose HiGMem (Hierarchical and LLM-Guided Memory System), a two-level event-turn memory system that allows LLMs to use event summaries as semantic anchors to predict which related turns are worth reading. This allows the model to inspect high-level event summaries first and then focus on a smaller set of potentially useful turns, providing a concise and reliable evidence set through reasoning, while avoiding the retrieval overhead that would be excessively high compared to vector retrieval. On the LoCoMo10 benchmark, HiGMem achieves the best F1 on four of five question categories and improves adversarial F1 from 0.54 to 0.78 over A-Mem, while retrieving an order of magnitude fewer turns. Code is publicly available at https://github.com/ZeroLoss-Lab/HiGMem.