Deep research agents (DRAs) integrate LLM reasoning with external tools. Memory systems enable DRAs to leverage historical experiences, which are essential for efficient reasoning and autonomous evolution. Existing methods rely on retrieving similar trajectories from memory to aid reasoning, while suffering from key limitations of ineffective memory evolution and increasing storage and retrieval costs. To address these problems, we propose a novel Memory Intelligence Agent (MIA) framework, consisting of a Manager-Planner-Executor architecture. Memory Manager is a non-parametric memory system that can store compressed historical search trajectories. Planner is a parametric memory agent that can produce search plans for questions. Executor is another agent that can search and analyze information guided by the search plan. To build the MIA framework, we first adopt an alternating reinforcement learning paradigm to enhance cooperation between the Planner and the Executor. Furthermore, we enable the Planner to continuously evolve during test-time learning, with updates performed on-the-fly alongside inference without interrupting the reasoning process. Additionally, we establish a bidirectional conversion loop between parametric and non-parametric memories to achieve efficient memory evolution. Finally, we incorporate a reflection and an unsupervised judgment mechanisms to boost reasoning and self-evolution in the open world. Extensive experiments across eleven benchmarks demonstrate the superiority of MIA.
Figures & tables
Figure 1: (a). Comparisons between frontier LLMs and their MIA-enhanced counterparts on the LiveVQA (multimodal) dataset. (b). Comparisons between frontier LLMs and their MIA-enhanced counterparts on the HotpotQA (text-only) dataset. (c). Comparisons between MIA based on Qwen2.5-VL-7B Executor with larger LLMs (in non-tool-calling settings) across seven diverse datasets. (d). Comparisons between MIA and SOTA memory frameworks based on Qwen-2.5-VL-7B Executor across seven diverse datasets.
Figure 2: A deep research process of MIA to tackle a complex and multi-hop question.
Figure 3: Reasoning process of MIA consists of three parts: Inputs & Retrieval is for retrieving memory context similar to the inputs; Research Process is for driving Planner-Executor collaboration via a planning-execution-reflection loop; Outputs & Storage is for compressing search trajectories into structured memory.
Figure 4: Prompt for Memory Manager to extract image caption and workflow.
Step
Action / Description
Step 1
Provide <question> to Planner to obtain the initial plan. ▹ Step 2 .
Step 2
Input <question>, <image>, and initial plan to the policy model. ▹ Step 3 .
Step 3
Generate thought(<think>) and action(<tool_call> or <answer>) based on current state. If a tool call is needed ▹ Step 4 ; otherwise, generate candidate answer ▹ Step 5 .
Step 4
Execute the corresponding tool and append the returned observation. ▹ Step 3 .
Step 5
LLM Judger evaluates the candidate answer. If correct ▹ Step 7 ; otherwise ▹ Step 6 .
Step 6
Provide interaction history to Planner to get a revised plan. ▹ Step 3 . (Note: This action triggers at most once; if already triggered ▹ Step 7 ).
Table 1: Executor Training Rollout Process.
Step
Action / Description
Step 1
Input the memory context, <question>, and prompt template to the policy model. ▹ Step 2 .
Step 2
Perform rollout to generate initial plan through CoT. ▹ Step 3 .
Step 3
Executor interacts with the environment using <question>, <image>, tools, and initial plan, yielding the candidate trajectory and result. ▹ Step 4 .
Step 4
Analyze candidate trajectory. If interaction should terminate ▹ Step 7 ; otherwise ▹ Step 5 .
Step 5
Perform rollout to generate CoT reflection and revised plan. ▹ Step 6 .
Step 6
Executor continues interaction based on candidate trajectory and revised plan, yielding final trajectory and result. ▹ Step 7 .
Table 2: Planner Training Rollout Process.
Figure 5: The Executor is activated during the first-stage RL training and frozen in the test-time RL process, while the Planner is activated during both the second-stage RL training and the test-time RL process. The memory framework of MIA during exploration: (1) generating multiple plan rollouts; (2) executing the inference pipeline , where a router selects the optimal plan based on prior experience to interact with the environment, strictly ensuring no label leakage; (3) receiving the final feedback from the environment; and (4) completing the training pipeline by calculating rewards and advantages for all rollouts. These evaluations are then used to update both the parametric memory (updating the Planner’s parameters via GRPO) and the non-parametric memory (extracting workflows into the Memory Manager).
Figure 6: A novel unsupervised evaluation framework that mimics the rigorous peer-review process of scientific venues.
Model
In-Domain
Out-of-Domain
FVQA-test
InfoSeek
SimpleVQA
LiveVQA
MMSearch
In-house 1
In-house 2
Direct Answer
GPT-5.4
50.8
43.6
55.5
21.5
44.4
45.1
23.0
Gemini-3-Flash
69.3
69.0
73.7
26.0
69.0
52.5
25.5
GPT-4o
41.7
42.7
46.6
26.9
22.2
25.6
17.2
Gemini-2.5-Pro
37.2
37.0
53.4
27.7
26.9
30.8
19.6
Table 3: Overall evaluation results on multimodal datasets for Deep Research Agent. Bold denotes the highest score in each column. Underline indicates the second-highest score in each column.
Model
Out-of-Domain
SimpleQA
2Wiki
HotpotQA
GAIA
No Memory
40.7
61.2
51.0
11.7
RAG
38.3
56.3
47.5
14.6
Mem0
38.1
54.9
49.0
16.5
A-Mem
38.8
56.2
47.5
12.6
ReasoningBank
42.4
61.0
52.7
14.6
Table 4: Overall evaluation results on text-only datasets for Deep Research Agent. Bold denotes the highest score in each column. Underline indicates the second-highest score in each column.
Figure 7: Left: Reward curves in each batch of Planner and Executor during the training stage. Middle: Response length in each batch of Planner and Executor during the training stage. Right: Response length in each batch of Planner during the TTL stage.
Figure 8: Comparisons between SOTA LLMs with MIA (the green area: MIA) and without MIA (the blue area: ReAct).
Figure 9: Tool call distribution analysis. Each dot represents an individual task execution (gray indicates failures; colored indicates successful runs). The scatter and half-violin plots illustrate the frequency of tool calls among successful executions.
Model
In-Domain
Out-of-Domain
FVQA-test
InfoSeek
SimpleVQA
LiveVQA
MMSearch
In-house 1
In-house 2
Base
61.4
56.8
63.0
33.0
55.6
15.9
26.9
Only Memory
62.8
56.8
61.2
37.8
56.1
12.2
28.5
Only Plan
64.9
58.6
62.6
35.4
56.7
21.0
31.3
Memory for Planner
67.9
60.7
61.8
36.0
59.0
17.0
34.7
+ Reflect
66.2
60.1
63.0
37.9
58.5
23.1
31.3
Table 5: Ablation study results on multimodal datasets. Bold denotes the highest score in each column.
Model
Out-of-Domain
SimpleQA
2Wiki
HotpotQA
GAIA
Base
40.7
61.2
51.0
11.7
Only Memory
37.7
61.3
50.3
12.6
Only Plan
42.1
62.8
54.9
18.5
Memory for Planner
42.4
64.6
54.8
19.4
+ Reflect
43.9
66.6
57.6
26.2
Table 6: Ablation study results on text-only datasets. Bold denotes the highest score in each column.
Model
Multimodal
Text-only
FVQA-test
LiveVQA
2Wiki
HotpotQA
Base
61.4
33.0
61.2
51.0
Plan and Reflect (no memory)
59.6
36.5
64.2
56.4
Unsupervised Memory for Planner
57.6
28.5
66.9
56.4
Unsupervised MIA (epoch-1)
65.1
40.1
71.6
61.7
Unsupervised MIA (epoch-2)
66.4
41.4
73.4
63.1
Table 7: MIA’s self-evolution results in an unsupervised setting. The Planner consistently uses Qwen3-8B.
Large Language Model (LLM) agents increasingly rely on external memory to support long-horizon reasoning and decision making. Existing memory systems typically retrieve historical trajectories or summaries as independent context fragments, overlooking the procedural dependencies underlying multi-step execution. As memory scales, such flat retrieval introduces context fragmentation and cross-task interference, leading to structurally inconsistent reasoning trajectories. We propose ActiveMem, a hierarchical memory framework that recursively organizes agent experiences into dependency-aware latent execution trees. ActiveMem abstracts trajectories into reusable subtask nodes while explicitly preserving execution transitions, enabling coherent reasoning-path retrieval conditioned on the current execution state. To support continual adaptation, ActiveMem further learns dynamic memory expansion, retrieval, and pruning policies through reinforcement learning. Experiments across various agent benchmarks demonstrate that ActiveMem consistently improves task completion, reasoning stability, and memory efficiency over existing memory-based agents. Moreover, ActiveMem enables compact open-weight models to achieve competitive performance with substantially larger proprietary systems.
Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-evolution baselines.
Mingfei Lu, Mengjia Wu, Runsong Jia +2
Australian Artificial Intelligence Institute (AAII) University of Technology Sydney
Despite recent progress, LLM agents still struggle with reasoning over long interaction histories. While current memory-augmented agents rely on a static retrieve-then-reason paradigm, this rigid pipeline design prevents them from dynamically adapting memory access to intermediate evidence discovered during inference. To bridge this gap, we propose MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism. We represent memory as a Cue-Tag-Content graph, where associative tags serve as semantic bridges connecting fine-grained cues to memory contents. Operating on this structure, our active reconstruction mechanism integrates LLM reasoning directly into memory access, allowing the agent to iteratively explore and prune retrieval paths based on accumulated evidence. This ensures that memory retrieval is dynamically adapted to the reasoning context while avoiding combinatorial explosion caused by unconstrained expansion. Experiments on the LoCoMo benchmark and LongMemEval benchmark demonstrate significant improvements over strong baselines (up to 23%), while substantially reducing token and runtime cost, highlighting the effectiveness of active and associative reconstruction for long-horizon memory reasoning.