Large language model (LLM) agents have demonstrated strong capabilities in automating complex information-seeking and transactional tasks across the open web. However, their real-world reliability remains limited by access barriers, such as dead links, authentication walls, rate limits, and anti-scraping defenses. These barriers block the primary execution path, causing agents to abort prematurely, loop indefinitely, or quietly compromise the user's original constraints. Existing evaluations either implicitly assume that target resources are freely accessible or focus on transient perturbations that simple retries can resolve, failing to capture scenarios where agents must actively reroute around impassable bottlenecks. To bridge this gap, this paper introduces AgentHeal, a comprehensive benchmark evaluating goal-preserving resource recovery, defined as the ability of web agents to detect non-transient access barriers, pivot to legitimate alternative sources, and fulfill the user's exact original objective. Spanning nine barrier families across a counterfactual sandbox and real-website replay, our experiments on leading LLM backbones show that average task success drops sharply from 89.4% to 43.7% under access barriers. Further analysis reveals that agents frequently fall for deceptive look-alikes, fail to escalate to users when necessary, violate access policies, or waste computation on futile retries, providing actionable diagnostics for developing resilient web agents.
Figures & tables
Figure 1. Overview of AgentHeal . Left: the 855 tasks by 4 barrier categories and 9 families; each family contributes sandbox instances and real-web barriers. Right: real-web barriers with the user’s request as the agent receives it ( <link> stands for the link the user provides), the blocked resource as served by the live website, and the legitimate recovery route. A sunburst chart of barrier categories, families, and hosts, next to a table of real-world instances with screenshots and recovery routes.
Benchmark
Env.
Persist.
Altern.
Audit
Abst.
Policy
Cost
WebArena ( Zhou et al., 2024 )
Sim.
✗
✗
✗
❍
✗
✗
VisualWebArena ( Koh et al., 2024a )
Sim.
✗
✗
✗
❍
✗
✗
WorkArena++ ( Boisvert et al., 2024 )
Sim.
✗
✗
✗
❍
✗
✗
Mind2Web ( Deng et al., 2023 )
Real
✗
✗
✗
✗
✗
✗
WebVoyager ( He et al., 2024 )
Real
✗
✗
✗
✗
✗
✗
WAREX ( Kara et al., 2025 )
Real
❍
✗
✗
✗
✗
✓
Table 1. Comparison of AgentHeal with representative benchmarks. Persist.: barriers persist throughout the episode rather than disappearing after a retry. Altern.: alternative resources discovered by the agent. Audit: whether an answer comes from an acceptable source is verified. Abst.: truthful reports of infeasibility are scored. Policy: compliance with access policies is scored. Cost: the cost of recovery is measured. ✓: supported; ❍: partially supported; ✗: not supported.
Figure 2. The AgentHeal construction and evaluation pipeline. Given a user request whose target resource is obstructed by realistic access barriers, we construct evaluation episodes across two settings: a counterfactual sandbox with controlled valid, deceptive, and reference variants, and a real-web replay environment preserving original URLs, response logs, robots.txt rules, and web archives. After verifying discovery routes and archival evidence, we evaluate web agents on their ability to recover valid alternatives ( Recover ), report provenance or infeasibility honestly ( Report ), and respect access policies ( Respect ).
Counterfactual sandbox
Real-web barrier replay
Model
TS 0↑
TS ↑
Recover. ↑
Audit. ↑
Honest. ↑
Violation. ↓
TS ↑
Recover. ↑
Honest. ↑
Violation. ↓
GPT-6.1-Sol
95.6
55.8
48.7
46.0
78.2
7.1
86.2
95.5
82.6
11.7
GPT-6-Luna
93.7
47.0
35.9
32.4
62.8
8.9
68.0
76.1
77.8
12.4
GPT-5.6-Terra
94.7
45.8
42.7
28.5
59.1
7.7
61.1
75.0
62.0
17.2
GPT-4o-mini
70.7
25.8
10.2
9.9
45.3
9.8
49.4
47.7
67.4
11.7
Claude-Haiku-5.5
79.9
56.9
27.6
49.0
88.0
5.5
80.6
71.6
89.1
10.6
Table 2. Main results on AgentHeal (%). TS 0 and TS denote task success without and with barriers. We also report Recovery Rate (Recover.), Audit Accuracy (Audit.), Honest Reporting Rate (Honest.), and Violation Rate (Violation.) across the counterfactual sandbox and the real-web replay suite. Best / second best: bold / underlined; Worst / second worst: gray bold / gray underlined.
Figure 3. Interaction efficiency across models and availability regimes.
Setting
TS 0↑
TS ↑
Recover. ↑
Audit. ↑
Honest. ↑
Violation. ↓
ReAct (w/o guidance)
92.9
57.8
52.0
50.6
68.4
11.0
w/ system prompt hint
96.9
70.7
62.7
64.3
82.7
8.3
w/ Claude Code
93.8
59.8
53.3
54.3
70.7
11.4
Table 3. Effect of explicit guidance and agentic harnesses on DeepSeek-V4.1-Flash in the counterfactual sandbox.
Figure 4. Model performance comparison. (a) Capability versus reliability. (b) Weighted overall performance versus model release date.
Tool
Function
Availability
search(query)
returns titles, addresses, and snippets of the first five search results
all
fetch(url, method, data, headers)
returns the response as text with numbered links and forms
all
download(url)
saves a file and returns its path and hash
all
wait(seconds)
waits, e.g., to honor a stated waiting period
all
ask_user(question)
asks the user a question or for an action only the user can perform