M$^2$: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight Retrieval

Dawei Yan; Haokui Zhang; Guangda Huzhang; Yang Li; Yibo Wang; Qing-Guo Chen; Zhao Xu; Weihua Luo

cs.CVFeb 28, 2026

M $^2$ : Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight Retrieval

Authors: Dawei Yan, Haokui Zhang, Guangda Huzhang, Yang Li, Yibo Wang, Qing-Guo Chen, Zhao Xu, Weihua Luo,

Organizations: Northwestern Polytechnical University, Xi’an, China · AI Business, Alibaba Group, Hangzhou, China · Xi’an University of Architecture and Technology, Xi’an, China · Zhejiang University, Hangzhou, China

Abstract

Multimodal Large Language Models (MLLMs) based agents have demonstrated remarkable potential in autonomous web navigation. However, handling long-horizon tasks remains a critical bottleneck. Prevailing strategies often rely heavily on extensive data collection and model training, yet still struggle with high computational costs and insufficient reasoning capabilities when facing complex, long-horizon scenarios. To address this, we propose M $^2$ , a training-free, memory-augmented framework designed to optimize context efficiency and decision-making robustness. Our approach incorporates a dual-tier memory mechanism that synergizes Dynamic Trajectory Summarization (Internal Memory) to compress verbose interaction history into concise state updates, and Insight Retrieval Augmentation (External Memory) to guide the agent with actionable guidelines retrieved from an offline insight bank. Extensive evaluations across WebVoyager and OnlineMind2Web demonstrate that M $^2$ consistently surpasses baselines, yielding up to a 19.6% success rate increase and 58.7% token reduction for Qwen3-VL-32B, while proprietary models like Claude achieve accuracy gains up to 12.5% alongside significantly lower computational overhead.

Figures & tables

Figure 2: Overview of the proposed framework. (a) The baseline agent operates with raw context containing redundant visual history and verbose interaction text. This creates high computational overhead and introduces noise that may impair decision-making. Our method M 2 incorporates two key mechanisms: In-Mem and Ex-Mem. (b) The In-Mem module prompts the agent to self-summarize historical steps into a concise context. Simultaneously, (c) the Ex-Mem module retrieves actionable insights from historically successful trajectories based on query similarity. (d) This dual-memory approach provides explicit guidance for the current execution step, significantly reducing errors and yielding tangible performance gains.

Figure 3: Comparison between Full Context and In Mem interaction paradigms. The Full Context (left) retains the entire sequence of raw screenshots and exhaustive historical text, leading to high redundancy. In contrast, the In Mem approach (right) employs a specific prompt to synthesize historical steps into a concise textual summary.

Figure 4: Overview of Insight Retrieval Augmentation pipeline. The workflow begins with a Trajectory Insight Database populated by successful trajectories from diverse models. For a given user query , the system filters insights based on semantic similarity, retrieving the Top- $i$ relevant insights. These insights are then injected into the System Prompt. This mechanism ensures the Web Agent generates goal-centric and error-proof actions, effectively navigating complex UI environments by leveraging cross-trajectory experiences.

Model	Config	AR	AM	AP	AX	BN	CD	CO	ES	GH	GM	HF	WA	Avg.step	Avg.token	Avg.acc.
Claude-3.7-Sonnet	Normal	71.1	65.9	69.8	74.4	78.6	83.7	83.3	77.3	75.6	63.4	44.2	76.1	14.6	121.2k	72.0
	In Mem	73.3	70.7	69.8	62.8	78.6	88.4	83.3	68.2	78.1	63.4	58.1	80.4	20.3	82.9k	72.9
	Ex Mem	82.2	78.1	76.7	76.7	83.3	86.1	85.7	68.2	90.2	80.5	67.4	82.6	15.7	164.9k	79.8
	In & Ex Mem	84.4	80.5	76.7	86.1	85.7	93.0	95.2	84.1	92.7	82.9	74.4	78.3	16.9	84.5k	84.5 ( $+$ 12.5%)
Claude-Sonnet-4	Normal	82.2	70.7	69.8	79.1	83.3	93.0	85.7	84.1	87.8	78.1	69.8	82.6	15.4	128.2k	80.5
	In Mem	84.4	75.6	79.1	81.4	78.6	88.4	85.7	79.6	82.9	70.7	67.4	82.6	16.8	64.5k	79.7

Table 2: Performance Comparison of Claude and Qwen3VL Model across Multiple WebSite on WebVoyager. (Abbreviations: AR: Allrecipes, AM: Amazon, AP: Apple, AX: ArXiv, BN: BBC News, CD: Cambridge Dict., CO: Coursera, ES: ESPN, GH: GitHub, GM: Google Map, HF: Huggingface, WA: Wolfram Alpha.)

Model

Config

Easy

Medium

Hard

Average

Step

Token

Acc. (%)

Step

Token

Acc. (%)

Step

Token

Acc (%)

Step

Token

Acc. (%)

Claude-3.7-Sonnet

Normal

21.72

219.6k

74.07

29.50

310.9k

58.27

34.51

371.6k

53.52

28.56

300.3k

61.95

In & Ex

19.89

98.0k (

-

55.4%)

79.01 (+4.9%)

27.40

144.0k (

-

53.7%)

65.47 (+7.2%)

31.03

163.9k (

-

55.9%)

61.97 (+8.5%)

26.05

135.2k (

-

55.0%)

68.82 (+6.9%)

Claude-Sonnet-4

Normal

16.23

154.7k

74.07

21.24

205.9k

66.91

21.24

292.1k

61.97

21.54

212.7k

67.70

In & Ex

18.77

95.7k (

-

38.1%)

82.72 (+8.7%)

27.00

141.2k (

-

31.4%)

73.38 (+6.5%)

37.24

207.2k (

-

29.1%)

64.79 (+2.8%)

27.21

144.6k (

-

32.0%)

73.88 (+6.2%)

Qwen3-VL-32B

Normal

25.41

220.9k

44.44

35.38

325.8k

30.22

42.11

403.7k

21.13

34.25

315.6k

31.96

Table 3: Evaluation Results on OnlineMind2Web. Green values in parentheses represent the relative reduction percentage for Tokens and absolute percentage point increase for Accuracy.

Ablation Category	Settings	Acc. (%)
Historical Window $k$	1	51.6
	3	56.0
	5 (Default)	57.8
	7	52.2
Similarity Function	TF-IDF ( Salton and Buckley, 1988 )	61.1
	Word2Vec ( Mikolov et al., 2013 )	64.4

Table 4: Ablation study of M 2 components on WebVoyager by Qwen3-VL-32B. S.T. denotes Sentence Transformer (Default).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Distribution of the 55k trajectory data across 12 distinct web domains in the Insight Bank.

Figure 6: Cumulative token consumption comparison on an Apple.com navigation task (16 steps). The agent uses Qwen3-VL-32B.

Figure 7: Comparison of agent responses between the original interaction form and the M 2 interaction form.

Figure 8: Qualitative comparison on the GitHub task. Left (Original): Agent gets trapped in a loop of irrelevant repository results due to context redundancy. Right (M 2 ): Agent navigates hierarchically to the documentation page and uses a precise search, maintaining a clear task progress via memory summarization.

Figure 9: Qualitative comparison on a geographic disambiguation task. Left (Original): The agent suffers from context hallucination, ignoring the established UK starting point and selecting a US destination (North Plymouth, MA), resulting in a routing failure. Right (M 2 ): Guided by the external insight to prioritize the first dropdown suggestion and maintaining internal context consistency, the agent correctly identifies the local UK destination.

Figure 10: Original system prompt used by the agent.

Figure 11: Update of Dynamic Trajectory Summarization compared to the original system prompt.

Figure 12: Prompt

P_{ext}

used by Abstractor to extract insight.

Figure 13: System prompts

P^{\prime}_{sys}

used in the final M 2 framework.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 429 papers

A Subgoal-driven Framework for Improving Long-Horizon LLM Agents
Mar 20, 2026Taiyi Wang, Sian Gooding, Florian Hartmann +2Long-Horizon LLM Agents Web Agents
Google DeepMind
AMBER: Training Long-Horizon Web Agents through Append-Only Memory
Oct 5, 2026Chinmay Savadikar, Zhaoyu Zhang, Mingyu Zhao +4Long-Horizon LLM Agents Agent Memory Management
North Carolina State University · Shopify
Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning
Apr 13, 2026Xiaozhe Li, Tianyi Lyu, Yizhao Yang +6Lost-in-the-Middle Effect LLM Context Management
Tongji University · CurrentsAI Research · Stanford University +1

Mar 20, 2026J/K move · Enter open · S save

A Subgoal-driven Framework for Improving Long-Horizon LLM Agents

Taiyi Wang, Sian Gooding, Florian Hartmann, Oriana Riva+1

Google DeepMind

Long-Horizon LLM Agents Web Agents RL for LLM Agents Reward Shaping

Open paper AskPDF arXiv

Large language model (LLM)-based agents have emerged as powerful autonomous controllers for digital environments, including mobile interfaces, operating systems, and web browsers. Web navigation, for example, requires handling dynamic content and long sequences of actions, making it particularly challenging. Existing LLM-based agents struggle with long-horizon planning in two main ways. During online execution, they often lose track as new information arrives, lacking a clear and adaptive path toward the final goal. This issue is further exacerbated during reinforcement learning (RL) fine-tuning, where sparse and delayed rewards make it difficult for agents to identify which actions lead to success, preventing them from maintaining coherent reasoning over extended tasks. To address these challenges, we propose two contributions. First, we introduce an agent framework that leverages proprietary models for online planning through subgoal decomposition. Second, we present MiRA (Milestoning your Reinforcement Learning Enhanced Agent), an RL training framework that uses dense, milestone-based reward signals. The real-time planning mechanism improves proprietary models such as Gemini by approximately a 10% absolute increase in success rate (SR) on the WebArena-Lite benchmark. Meanwhile, applying MiRA to the open Gemma3-12B model increases its success rate from 6.4% to 43.0%. This performance surpasses proprietary systems such as GPT-4-Turbo (17.6%) and GPT-4o (13.9%), as well as the previous open-model state of the art, WebRL (38.4%). Overall, our findings demonstrate that combining explicit inference-time planning with milestone-based rewards significantly improves an agent's long-horizon capabilities, paving the way for more robust and general-purpose autonomous systems.

Abstract

Long-Horizon LLM Agents

Explore similar work

A Subgoal-driven Framework for Improving Long-Horizon LLM Agents

AMBER: Training Long-Horizon Web Agents through Append-Only Memory

Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning