cs.AIMar 7, 2026

Enhancing Web Agents with a Hierarchical Memory Tree

Authors: Yunteng Tan, Zhi Gao, Xinxiao Wu

Organizations: Beijing Institute of Technology, Beijing 100081, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language model-based web agents have shown strong potential in automating web interactions through advanced reasoning and instruction following. While retrieval-based memory derived from historical trajectories enables these agents to handle complex, long-horizon tasks, current methods struggle to generalize across unseen websites. We identify that this challenge arises from the flat memory structures that entangle high-level task logic with site-specific action details. This entanglement induces a workflow mismatch in new environments, where retrieved contents are conflated with current web, leading to logically inconsistent execution. To address this, we propose Hierarchical Memory Tree (HMT), a structured framework designed to explicitly decouple logical planning from action execution. HMT constructs a three-level hierarchy from raw trajectories via an automated abstraction pipeline: the Intent level maps diverse user instructions to standardized task goals; the Stage level defines reusable semantic subgoals characterized by observable pre-conditions and post-conditions; and the Action level stores action patterns paired with transferable semantic element descriptions. Leveraging this structure, we develop a stage-aware inference mechanism comprising a Planner and an Actor. By explicitly validating pre-conditions, the Planner aligns the current state with the correct logical subgoal to prevent workflow mismatch, while the Actor grounds actions by matching the stored semantic descriptions to the target page. Experimental results on Mind2Web and WebArena show that HMT significantly outperforms flat-memory methods, particularly in cross-website and cross-domain scenarios, highlighting the necessity of structured memory for robust generalization of web agents.

Figures & tables

Fig. 1: Comparison between Flat Memory and Hierarchical Memory Tree ( HMT ). (a) Flat memory methods retrieve interaction trajectories mixed with original workflows and source-specific implementation details w/o subgoals, leading to workflow mismatch and context pollution when applied to unseen websites. (b) HMT decouples intent from execution using a tree structure. It retrieves stage-aligned subgoals and abstract element descriptions, enabling the agent to plan the correct step and ground actions on the target interface effectively.
Fig. 2: Overview of HMT . The framework consists of a construction pipeline that abstracts raw trajectories into a hierarchical memory tree, and a stage-aware inference mechanism where a Planner selects the logical stage and an Actor grounds the action level description to the target page.
Fig. 3: Structure of the Hierarchical Memory Tree. Unlike flat lists, HMT organizes memory into intent, stage, and action levels.
Fig. 4: Mechanism Analysis. (a) Retrieval recall comparisons show that HMT provides more accurate context. (b) Grounding success rate across generalization splits shows that raw identifiers fail in cross-website and cross-domain settings, while the semantic descriptions (ours) remain robust.
Fig. 5: Visual analysis of a successful cross-website grounding trace demonstrated by HMT . (a) Memory Retrieval: The agent retrieves an abstract action pattern and a semantic descriptor from the hierarchical memory, explicitly discarding the site-specific raw identifier ( #btn-sfo-136 ) from the source trace. (b) Planner Verification & Decision: To prevent workflow mismatch , the Planner verifies that the current page satisfies the stage pre-conditions (e.g., “Flight list visible”) before authorizing the Actor, ensuring actions are only executed in the correct context. (c) Actor Grounding & Execution: Guided by the semantic descriptor, the Actor scans the target DOM on Trip.com . It successfully distinguishes between a distractor advertisement and the correct flight selection button, executing the correct action despite the layout shift.
Fig. 6: Analysis of two representative failure modes in cross-website grounding. (a) Ambiguous Grounding: The agent retrieves a generic descriptor ( text_contains: more ) lacking hierarchical constraints. Consequently, the Actor assigns a higher similarity (score 0.92) to the distractor candidate than to the intended target (score 0.89), leading to an execution error. (b) State Verification Error: The Planner retrieves a strict success criterion expecting a NAVIGATION event (e.g., URL change or page reload). Although the action triggers a visual update (modal popup), the URL remains static. The Planner’s rigid verification logic ( url_change = False and page_reload = False ) fails to recognize the visual progress, incorrectly marking the step as incomplete and initiating a retry loop.
More in this topic

Web Agents

Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.

See all 174 papers

Explore similar work

CardsList
Mar 5, 2026

STRUCTUREDAGENT: Planning with AND/OR Trees for Long-Horizon Web Tasks

Existing LLM-based web agents struggle on complex, long-horizon tasks due to limited in-context memory, weak planning abilities, and greedy behaviors that lead to premature termination. To address these challenges, we propose \SA{}, a hierarchical planning framework that interleaves planning and execution via dynamic \ANDOR\ANDOR trees. The framework separates structural planning from LLM-based reasoning, enabling principled error recovery through node repair, systematic exploration of alternatives via OR nodes, and modular plans that can facilitate human intervention. On WebArena (630 tasks), \SA{} achieves a ∼\sim53% success rate vs.\ ∼\sim46% for AgentOccam, and on complex multi-constraint Amazon shopping tasks, gains reach 10% over the strongest baseline.

ELita Lobo, Xu Chen, Jingjing Meng +5

University of Massachusetts Amherst · Amazon · Virginia Commonwealth University +1

LLM Agent PlanningLong-Horizon LLM AgentsWeb Agents
AskPDF
Feb 28, 2026

M2^2: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight Retrieval

Multimodal Large Language Models (MLLMs) based agents have demonstrated remarkable potential in autonomous web navigation. However, handling long-horizon tasks remains a critical bottleneck. Prevailing strategies often rely heavily on extensive data collection and model training, yet still struggle with high computational costs and insufficient reasoning capabilities when facing complex, long-horizon scenarios. To address this, we propose M2^2, a training-free, memory-augmented framework designed to optimize context efficiency and decision-making robustness. Our approach incorporates a dual-tier memory mechanism that synergizes Dynamic Trajectory Summarization (Internal Memory) to compress verbose interaction history into concise state updates, and Insight Retrieval Augmentation (External Memory) to guide the agent with actionable guidelines retrieved from an offline insight bank. Extensive evaluations across WebVoyager and OnlineMind2Web demonstrate that M2^2 consistently surpasses baselines, yielding up to a 19.6% success rate increase and 58.7% token reduction for Qwen3-VL-32B, while proprietary models like Claude achieve accuracy gains up to 12.5% alongside significantly lower computational overhead.

Dawei Yan, Haokui Zhang, Guangda Huzhang +8

Northwestern Polytechnical University, Xi’an, China · AI Business, Alibaba Group, Hangzhou, China · Xi’an University of Architecture and Technology, Xi’an, China +1

Long-Horizon LLM AgentsWeb AgentsMemory-Augmented Language Model AgentsRetrieval-Augmented ICL
AskPDF
Apr 28, 2026

DRIVE: Modeling Skills at the Reasoning and Interaction Levels for Web Agents under Continual Learning

Web agents require both high-level reasoning (for task decomposition) and low-level interactions (for page elements manipulation) to conduct different tasks. However, these knowledge types differ fundamentally: reasoning knowledge (e.g., booking a flight requires first searching for routes) is abstract and transferable across websites, while interaction knowledge (e.g., clicking the Search button at a specific coordinate on Site A) depends heavily on page-specific contexts. Existing methods store experiences uniformly. This creates a dilemma: abstract representations lose executability on concrete pages, while concrete representations fail to generalize across domains. This entanglement limits capability accumulation: on new websites, agents either fail to recognize reusable task logic due to surface-level differences or attempt infeasible actions from outdated page structures. To disentangle them, we propose DRIVE, a dual-level skill modeling framework separating historical experience into natural language reasoning skills, which capture transferable task logic, and programmatic interaction skills, grounding abstract actions to executable operations. A scene-aware coordination mechanism adaptively retrieves and invokes these dual-level skills based on task semantics. DRIVE also uses skill-level reflection to identify hierarchy-specific failure modes, enabling targeted skill library expansion and refinement. Experiments across five WebArena domains show DRIVE attains an average task success rate of 52.8%, exceeding the skill-free baseline by 7.3 percentage points. Further ablations show reasoning and interaction skills provide distinct, complementary benefits, supporting separation of transferable task logic from executable page-level operations.

Xirui Liu, Sihang Zhou, Yanning Hou +6

College of Intelligence Science and Technology, National University of Defense Technology, Changsha, 410073, Hunan, China · College of Computer Science and Technology, National University of Defense Technology, Changsha, 410073, Hunan, China

Continual Learning for LLM AgentsLLM Agent Skill LearningAgent Skill LearningWeb Agents
AskPDF