TreeAgent: Structure-Aware Memory Agent for Processing Long Hierarchical Data
Authors: Wei Gu, Weiwei Qi, Jiajun Tang, Tianhang Zheng, Bo Chen, Zhibo Zhang, Jinfei Liu, Kui Ren
Organizations: The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Alibaba Group · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security
Practical Web applications increasingly require agents to interpret long conversations or documents. For inputs exceeding the context windows of large language models (LLMs), memory agents process segments sequentially and retain past information in bounded memory. However, they typically divide inputs into flat token segments without preserving structure or hierarchy, causing three limitations for structured data such as API responses or HTML documents. First, fixed-length segmentation can split fields of the same record, breaking their relationships. Second, segments lack hierarchical information, obscuring their structural context and positions. Third, successive memory updates may discard details of earlier sibling records needed for reasoning on later segments. To address these limitations, we propose TreeAgent, a structure-aware agent that uses the input hierarchy to guide both segment-wise context processing and access to past information. Specifically, TreeAgent constructs segments from complete subtrees and groups of adjacent sibling subtrees to preserve local structural relationships. We provide theoretical guarantees on segment counts to guide token budget selection toward matching the segment count of fixed-length segmentation. We further attach structural paths to the segments for the agent to identify their ancestors and hierarchical positions. We also introduce residual memory links from the segments of low-level children to the segments of high-level parents, enabling direct information flows beyond successive memory updates. We evaluate TreeAgent with four representative memory agents on eight retrieval and aggregation tasks. The inputs are derived from real-world application data in JSON, HTML, YAML, and JavaScript and range from 8K to 1M tokens. TreeAgent improves the average score across the four datasets for all four agents, with 3.61% performance gain on average.
Figures & tables
Figure 1. Overview of TreeAgent, showing how an observation is constructed and used in a segment-wise memory update. Each observation ot combines the current structural section, its path, and spans relevant to the query retrieved from historic memory states through residual memory links. The memory-agent block represents the agent’s original workflow, including its internal actions and stopping behavior. After processing each section, TreeAgent saves the updated memory. Parent–child relations determine which saved states are linked and accessed by later updates. Diagram of structural sectioning, paths, residual retrieval, and recurrent memory updates.
Memory agent
Method
JSON
HTML
YAML
JavaScript
AVG.
MemAgent-7B
Flat
35.48
34.96
40.68
50.44
40.39
Tree
36.75
35.10
40.85
49.30
40.50
Tree+Path
36.94
35.22
40.62
49.20
40.50
TreeAgent
37.28
35.29
42.00
50.38
41.24
InfMem-4B
Flat
57.67
38.65
65.01
63.51
56.21
Tree
57.75
42.95
68.10
67.25
59.01
Table 1. Mean answer scores (%) for four memory agents, averaged over eight tasks and eight context lengths (8K–1M). Tree, Tree+Path, and TreeAgent successively add structural sectioning, structural paths, and residual memory links with span retrieval to Flat. AVG. averages the four datasets; the final group averages the four agents. Bold marks the best score within each group and column.
Figure 2. Mean answer scores (%) of Flat and TreeAgent as context length increases from 8K to 1M tokens. Each panel shows one memory agent with fixed model parameters. At each length, scores are averaged over eight tasks with 50 examples per task, and then averaged equally across the four datasets. Four panels compare Flat and TreeAgent scores across context lengths for the four memory agents.
Figure 3. Answer score gains of TreeAgent over Flat across eight tasks on the JSON evaluation set (TreeAgent − Flat, %). Gains are averaged equally over eight context lengths from 8K to 1M tokens, with 50 test examples per task and context length. Panels use different y-axis ranges. Four panels show TreeAgent minus Flat answer scores for eight JSON tasks. Colors distinguish retrieval of one, two, or four fields, retrieval from two or four records, conditional counting, frequent values, and numerical extrema.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Memory agent
Memory
Answer
Controller
MemAgent-7B
1,024
1,024
n/a
InfMem-4B
1,024
1,024
1,024
MemReread-4B
1,024
1,024
8,192
ReMemR1-7B
2,048
2,048
1,024
Appendix
Table 2. Maximum generated tokens for agent calls. MemAgent does not use a separate controller limit.
Memory agent
Model calls
Input tokens
MemAgent-7B
1.074
1.038
InfMem-4B
1.037
0.945
MemReread-4B
1.068
0.943
ReMemR1-7B
1.076
1.040
Appendix
Table 3. Realized computation on the JSON dataset: ratios of cumulative TreeAgent to Flat quantities. Input tokens include complete model inputs over all calls.
Memory agent
Difference
95% CI
MemAgent-7B
1.80
[0.62, 2.98]
InfMem-4B
7.45
[5.98, 8.91]
MemReread-4B
15.50
[13.77, 17.23]
ReMemR1-7B
1.64
[0.39, 2.89]
Appendix
Table 4. Paired score differences on the JSON dataset (%) and 95% confidence intervals. Absolute scores appear in Table 1 .
Context
Fixed
Trained
Diff.
8K
74.688
74.979
0.291
16K
68.979
70.125
1.146
32K
73.063
72.479
-0.584
64K
71.813
72.583
0.770
128K
65.625
66.583
0.958
256K
61.896
64.021
2.125
Appendix
Table 5. Mean answer scores (%) on the JSON test set for InfMem-4B with TreeAgent. Fixed and Trained denote the checkpoints before and after additional training, respectively. Diff. denotes Trained minus Fixed.
Long-horizon conversational agents need to interact with users through evolving events, tasks, and goals. Such histories are naturally temporal, yet many existing memory systems organize information primarily by topical similarity and may ignore the order in which events occur. We introduce Segment Tree Memory, or SegTreeMem, a memory architecture that represents conversation history as a temporally ordered Segment Tree over utterances. SegTreeMem incrementally inserts new utterances through an online rightmost-frontier update rule, preserving chronological order while forming hierarchical memory segments. For retrieval, SegTreeMem propagates relevance scores through the tree to combine local semantic matching with hierarchical temporal context. Across three long-horizon memory benchmarks and two LLM backbones, SegTreeMem improves answer quality over flat retrieval, graph-structured memory, and tree-structured memory baselines. Additional temporal-order permutation analysis shows that the performance gain depends on preserving temporal order during memory construction, supporting the claim that temporal order is a key structure for agentic memory.
Yifan Simon Liu, Liam Gallagher, Faeze Moradi Kalarde +3
University of Toronto · 2Vector Institute for Artificial Intelligence
Long-horizon LLM agents must preserve information from past interactions to support future tasks. Existing memory systems typically rely on eager consolidation, invoking LLMs after each interaction to extract, summarize, or update memories. This design makes memory construction increasingly costly as conversations grow. Coarse summarization can reduce construction cost but risks discarding fine-grained contextual evidence, whereas larger retrieval contexts or multi-hop LLM reasoning shift the overhead to query time. We present LycheeMemory V2, an efficient long-term memory framework that replaces turn-level consolidation with semantic segment-level consolidation. Instead of consolidating every interaction, LycheeMemory batches multiple exchanges into segments and encodes each finalized segment into context-independent typed memory records. Segment-level batching lowers LLM encoding frequency, while semantic boundary detection helps preserve coherent event-level and temporal evidence compared with fixed-window batching. The resulting records are organized with lightweight structured indexes for query-planned evidence retrieval. Experiments using GPT-4.1-Mini show that LycheeMemory achieves state-of-the-art performance, reaching 89.22% on LoCoMo and 92.20% on LongMemEval-S. Compared with A-Mem, it reduces construction tokens by 86.0% on LoCoMo and 75.9% on LongMemEval-S without increasing query-time token usage. More broadly, our results suggest that the accuracy--cost trade-off of long-term agent memory depends not only on what information is retained, but also on the granularity at which it is consolidated.
Large language model (LLM) agents struggle with long-horizon tasks due to their inherent statelessness, requiring all task-relevant information to be encoded in growing input contexts. The resulting degraded reasoning quality, increased inference cost, and higher latency necessitate efficient working memory mechanisms. However, existing approaches either rely on lossy compression or similarity-based retrieval, which often fail to capture temporal structure and causal dependencies required for multi-step agentic tasks. In this work, we present HORMA, a Hierarchical Organize-and-Retrieve Memory Agent that organizes experience into a file-system-like hierarchical structure, where summarized entities are linked to the corresponding raw trajectories, enabling efficient access without losing detailed information. HORMA decomposes working memory into two stages: structured memory construction and navigation-based retrieval. The construction module iteratively refines how experiences are structured by distinguishing between failures caused by missing information and those caused by misleading or overloaded context. The navigation module retrieves task-relevant context by traversing the hierarchy using a lightweight agent trained with reinforcement learning to select minimal yet sufficient context, thereby reducing latency along the critical execution path. Across ALFWorld, LoCoMo, and LongMemEval, HORMA improves task performance under constrained context budgets while requiring at most 22.17% of the baseline token usage in long conversation tasks. Compared to existing methods, it consistently achieves better efficiency-performance trade-offs and generalizes effectively to unseen tasks.