Twists and Turns: A Graph-Based Benchmark and Framework for Long-Term Memory in Exploration Tasks
Organizations: Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Monash University · Hefei University of Technology
Abstract
Long-term problem-solving processes, such as scientific research and software development, proceed along an evolving trial-and-error trajectory toward a central goal. We term these exploration tasks, which require models to remember not only facts but also how solutions are tested against intermediate outcomes and early attempts are continually revised or revived. However, existing long-term memory research focuses on factual recall and updates, overlooking the interconnected nature of exploration. We observe that exploration processes naturally form graphs, with branching approaches linked by revisions and reconsiderations. Motivated by this, we propose LExMem, a graph-guided benchmark designed to evaluate Memory in Long-term Exploration tasks. We model exploration processes as graphs and introduce an evolutionary mechanism to increase their structural complexity, yielding 224 dialogues and 2,018 question-answer instances derived from graph structures. Building on the same insight, we propose LExGraph, a graph-based memory framework that incrementally organizes dialogues into interconnected memory nodes and retrieves evidence through hybrid retrieval and neighbor expansion. Extensive experiments demonstrate that LExMem poses a significant challenge to existing retrieval- and memory-based systems, whereas LExGraph achieves superior performance and generalizes effectively to external benchmarks. Codes and datasets are available at https://github.com/mbzuai-nlp/LExMem.
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Level | Criterion | Avg. | |
| Graph | Coherence | 4.10 | 0.58 |
| Exploration Fidelity | 4.21 | 0.52 | |
| Domain Plausibility | 4.42 | 0.61 | |
| Block | Faithfulness | 4.60 | 0.53 |
| Naturalness | 4.46 | 0.51 | |
| Coherence | 4.27 | 0.58 |
Memory Agent Benchmarks
Memory agent benchmarks evaluate how AI agents store, retrieve, update, and use information across interactions and changing tasks.
See all 183 papers