Computer-use agents plan on a screenshot for several seconds and then click at a position on the screen; if the page has changed in the meantime, the click lands on an unintended element. Prior work treats the planning interval as the source of this risk. Measuring planned clicks on 31 public web pages, we find that the risk lies in page loading: 0.4% of clicks planned on fully loaded pages became stale within 30 seconds, against 17% of clicks planned in the first second after a page opened. On these pages, waiting about one second for network activity to stop removes most of this risk, yet the wait rules of eight of ten widely used open-source agents leave more than 1% of clicks stale. Commercial pages keep loading for longer: in the HTTP Archive's public crawl, 15% of 65 desktop booking and buying home pages went 500 ms without network requests within 3 seconds of their HTML being parsed, against 63% of 3,028 .gov pages, and 52% shifted their layout within 6.5 seconds of a network-idle wait capped at 3 seconds, against 8% (61% against 26% among pages recorded in full). Where a check before the click is needed, as before irreversible actions and on pages that are still loading, a pixel comparison verifies the target's position as accurately as a frontier vision-language model at one eighth of the latency, and a 4B model matches the frontier model in verifying whether the previous action succeeded. We release a benchmark generator, a break-even formula for when a check pays, and three rules for agent developers: https://github.com/multimodal-ai-engineering/preflight-bench
Figures & tables
Figure 1: One airline’s home page in a desktop Chrome Incognito window, in screenshots we took by hand on 28 October 2026; brand names are blurred. (a) A screenshot taken while the page loads: the search form is complete, and an agent planning on it would click the search button (box and dot; the dotted line marks its height). (b) The same page once it has loaded: a credit-card offer was inserted under the menu and pushed the form down 52 px, so the planned point lies on a checkbox that switches the search to loyalty points. The HTTP Archive’s crawl of this page on 1 October recorded the timing: a network-idle wait capped at 3 s ends at 3.5 s, the layout shifts 1.3 s later, and the network goes quiet only at 8.1 s. Rates across sites are in Section 5.1 .
Figure 2: Left: share of planned clicks that remain valid on 31 public pages, by page state. Right: median wait of each rule after the page opens, and the share of clicks it leaves stale (95% CI). Only waiting for network activity to stop reaches 1%.
Wait rule
Wait
Stale (95% CI)
Agent (wait rule)
Stale (95% CI)
Wait
no wait
0 s
14.4% (12.6–16.5)
Anthropic CU demo (2 s delay)
2.0% (1.3–2.9)
2.02 s
0.5 s delay
0.50
2.5% (1.7–3.5)
browser-use (load or idle)
4.0% (3.1–5.2)
0.94
load event
0.43
3.0% (2.2–4.2)
OpenAI CUA sample (no wait)
15.4% (13.5–17.4)
0.35
layout stable 0.5 s
0.65
2.8% (2.0–3.8)
Stagehand v4 (no wait)
12.5% (10.8–14.5)
0.34
boxes stable 0.5 s
0.81
1.8% (1.2–2.7)
Skyvern (load, 1 s, animations)
0.46% (0.21–1.0)
2.03
1 s delay
1.00
3.1% (2.3–4.2)
Agent S3 (1 s delay)
11.7% (10.1–13.6)
1.02
Table 1: Left: wait rules for the planning screenshot after a page opens, timed from DOM-ready. Right: post-navigation wait rules of ten open-source agents (code reviewed October 2026, default settings), reimplemented in our harness and applied to the same 31 pages; the agents themselves were not executed. Stale: share of planned clicks whose target moved, disappeared or was covered within 6.5 s.
Figure 3: Desktop home pages in the HTTP Archive crawl (October 2026) whose layout shifted within 6.5 s of the screenshot, by wait rule (95% CI). Unrecorded time counts as no shift, so each point is a lower bound; most .gov windows, and *most windows without the cap, are not recorded in full.
Home pages
Pages
Quiet within 3 s
Time to quiet
Last shift
Shifted after the wait
booking and buying
65
15%
7.4 s
5.1 s
52% (40–65) [61%]
main host only †
57
11%
7.8 s
5.8 s
60% (47–72) [64%]
e-commerce sample
4,972
19%
6.4 s
2.2 s
22% (21–23) [30%]
.gov
3,028
63%
2.1 s
0.65 s
7.8% (6.8–8.7) [26%]
study sites
21
71%
1.9 s
0.45 s
4.8% (0–14)
Table 2: Desktop home pages under rule 1 (network idle, 3 s cap). Quiet: 500 ms without requests within 3 s of DOM-ready. Time to quiet (uncapped) and last layout shift: medians from DOM-ready. Shifted: within 6.5 s of the screenshot (95% CI); in brackets, among fully recorded pages where enough remain. † Post hoc.
Question
Check
Detected
False alarms
Latency (ms)
Target still in place? (1,064 stale clicks)
pixel check
82%
4%
312
PUSV, tuned to 2% false alarms
24%
0.3%
5
Qwen3-VL 2B
0.2%
0%
1,450
Qwen3-VL 4B
66%
37%
2,425
Claude Haiku 5.5
98%
95%
1,276
Claude Opus 5.5
83%
12%
2,366
Table 3: Decisions of each check on held-out moving pages: share of problems detected (recall), share of valid cases incorrectly flagged (false alarms), and median latency. Small open models ran on balanced subsets; hosted models on every pair.
Figure 4: One agent step under current practice (top) and under the three rules (bottom). Most stale clicks arise when the screenshot is captured while the page is still loading. Before the click, a pixel check confirms that the target remains where it was planned; after an action that commits a change, a small model or optical character recognition (OCR) confirms that it succeeded.
Browser-use agents are widely used for everyday tasks. They enable automated interaction with web pages through structured DOM based interfaces or vision language models operating on page screenshots. However, web pages often change between planning and execution, causing agents to execute actions based on stale assumptions. We view this temporal mismatch as a time of check to time of use (TOCTOU) vulnerability in browser-use agents. Dynamic or adversarial web content can exploit this window to induce unintended actions. We present a large scale empirical study of TOCTOU vulnerabilities in browser-use agents using a benchmark that spans synthesized and real world websites. Using this benchmark, we evaluate 10 popular open source agents and show that TOCTOU vulnerabilities are widespread. We design a lightweight mitigation based on pre-execution validation. It monitors DOM and layout changes during planning and validates the page state immediately before action execution. This approach reduces the risk of insecure execution and mitigates unintended side effects in browser-use agents.
Skim is a speculative execution framework for web agents that exploits the predictable structure of purpose-built websites. Today's web-agent expense is not intrinsic to the tasks but a property of how agents are composed: frontier-model inference, browser rendering, and ReAct-style planning are applied to every step of every task regardless of complexity. Skim's key observation is that websites enforce stable URL patterns, answer formats, and task-to-trajectory mappings across queries of the same type, so most queries can bypass these heavyweight components entirely. An offline profiler captures these patterns once per site. At runtime, Skim matches each query to a template, synthesizes the destination URL, and extracts the answer with a small model. A lightweight verifier gates each fast-path output against the query and schema; rare misspeculations cascade to the full agent, warm-started by the fast path's final URL to preserve upstream trajectory progress. Across standard web-agent benchmarks paired with three backboneagents (WebVoyager, AgentOccam, BrowserUse), Skim reduces median per-task cost by 1.9x and latency by 33.4% with no accuracy loss.
Mike Wong, Kevin Hsieh, Suman Nath +1
Princeton University · Princeton, USA · Microsoft Research +1
Large language models are increasingly deployed as autonomous agents that interact with the web through browsers. While recent progress has been driven by benchmarks that evaluate end-to-end task success, these evaluations largely overlook two fundamental sources of difficulty in real web browsing: complex actions over rich user interfaces and visual perception of dynamically rendered content, especially in workflows that span multiple websites. We introduce CAP, a scalable benchmark for evaluating browser agents on cross-site, human-like web tasks that require non-trivial UI interactions and visual understanding. Specifically, we adopt a decomposition-and-recomposition pipeline that first abstracts each website into a structured site card capturing user-facing functions, complex execution operations, and perceptual requirements, and then recomposes these components into realistic cross-site workflows. Each task is therefore grounded in multiple specific operations on each website, enabling fine-grained diagnosis. Built on this framework, we construct 420 tasks across 108 real-world websites and 24 domains under careful quality control. Experiments on state-of-the-art browser agents using our verifiable agent-as-a-judge evaluation framework show low success rates and reveal that perception-heavy interactions remain a major bottleneck, exposing substantial gaps between current agents and real-world web browsing demands.
Zejun Xu, Taiyi Chen, Jin Li +13
Macau University of Science and Technology · FellouAI · Tsinghua University +3