cs.AISep 11, 2026

AutoTailor: Automatic, User-Aligned Capability Selection and Adaptation for Web Agents

Authors: Xinyun Cao, Adriana Szekeres, Fazle Elahi Faisal

Organizations: University of Michigan, Ann Arbor · Microsoft Research, Redmond, WA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Web agents can utilize reusable tools to reduce the cost and latency of low-level browser interaction, but automatically discovered tool collections can be large, redundant, and poorly aligned with user demand. We present AutoTailor, a meta-agentic framework for constructing and maintaining a compact set of trajectory-derived Model Context Protocol (MCP) APIs. Offline, AutoTailor converts web trajectories into parameterized browser-automation programs, applies a Quality Filter to remove APIs with unsuitable granularity and redundant functionality, and applies a Usage Likelihood Filter to prioritize broadly useful capabilities while preserving semantic coverage. Online, Dynamic Reselection monitors task outcomes and API usage, identifies recurring coverage gaps, adds relevant candidates, and prunes persistently unused capabilities. We evaluate AutoTailor on 106 WebArena Postmill tasks. Offline filtering reduces the initial 1,283 unrefined APIs to 87, and Dynamic Reselection produces a 33-API set. With reasoning and acting (ReAct) fallback, this set achieves 90.6% correctness, compared with 87.5% for ReAct alone, while reducing average total request-token cost by 57.8% and latency by 29.4%. Without ReAct, it achieves 60.1% correctness, marginally matching the performance of unrefined set, while reducing request-token usage by 94.9%. Together, these results show that static filtering produces a compact inventory of APIs expected to support core, high-likelihood tasks, while dynamic reselection further tailors that inventory to observed user needs. This combination improves accuracy and latency while sharply reducing token usage and end-to-end cost, demonstrating the value of user-aligned capability management for efficient web agents.

Figures & tables

Figure 1: Overview of AutoTailor system architecture: trajectory-derived web actions are converted into reusable MCP APIs, filtered and deduplicated into a compact capability bank, and then dynamically reselected to match user demand while maintaining functional coverage for downstream runtime web agents.
Figure 2: Overview of trajectory crawling. Given a target website, the trajectory crawler systematically explores available interactions and collects diverse multi-step web trajectories, each represented as an ordered sequence of actions. The resulting corpus captures trajectories of varying lengths and task structures; the inset illustrates an example trajectory consisting of navigation, search, item selection, and cart interaction.
 ReAct only (reference)SinitS_{\mathrm{init}} w/ ReActSQFS_{\mathrm{QF}} w/ ReActSULFS_{\mathrm{ULF}} w/ ReActSDRS_{\mathrm{DR}} w/ ReAct
MetricAllSucc.AllSucc.AllSucc.AllSucc.AllSucc.
Correctness0.87510.88010.83510.87310.9061
Avg Task Latency (s)102.6396.6593.3172.9085.9764.2986.9175.5172.4569.17
Avg # MCP API Calls1.091.081.341.231.731.631.761.671.671.68
Avg # Tooling LLM Calls3.043.073.523.394.093.954.414.123.983.90
Avg Tooling Request Tokens29.70k29.94k2.39m2.30m403.30k387.98k205.13k188.89k98.46k95.89k
Table 1: Functional evaluation with the ReAct Fallback on all 106 WebArena Postmill tasks, comparing the static API-selection stages with the dynamically reselected capability set. ReAct only provides a ReAct-only reference without a trajectory-derived API set. “All” and “Succ.” denote averages over all tasks and successful tasks, respectively. Bold indicates the best value across all evaluated conditions for each metric and averaging scope - higher is better for correctness and the number of MCP API calls; lower is better for all other metrics.
 SinitS_{\mathrm{init}} w/o ReActSQFS_{\mathrm{QF}} w/o ReActSULFS_{\mathrm{ULF}} w/o ReActSDRS_{\mathrm{DR}} w/o ReAct
MetricAllSucc.AllSucc.AllSucc.AllSucc.
Correctness0.60410.59910.50210.6011
Avg Task Latency (s)99.1158.0384.4841.9886.8544.3079.5036.93
Avg # MCP API Calls1.931.441.761.421.711.431.991.44
Avg # Tooling LLM Calls4.053.584.273.854.354.104.663.51
Avg Tooling Request Tokens2.78m2.43m452.58k383.82k225.07k191.24k142.86k90.11k
Table 2: Functional evaluation without ReAct fallback on all 106 WebArena Postmill tasks. Annotations same as Table 1 .
More in this topic

Web Agents

Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.

See all 174 papers

Explore similar work

CardsList
Oct 26, 2026

Back to Basics: A Skill-Centric Approach to Generalist Web Agents

Browser agents do not just need more practice. They need to practice the right things. We introduce MiniWeb, a realistic miniature of the web comprising 65 interconnected websites across 15 domains, where agents navigate, search, and transact as people do. Its 369 human-annotated tasks cover everyday workflows and label 37 reusable interaction primitives grouped into 7 skill families, allowing us to diagnose which skills fail rather than only whether a task succeeds. Benchmarking ten open-weight agents, we find that drag & gesture is consistently the weakest family, with no agent solving more than 24% of its instances. To address this gap, we introduce MiniWebGen, a framework that autonomously generates verified, skill-centric training data from MiniWeb. Fine-tuning Qwen3.5-4B on just 3.1K verified trajectories improves single-primitive success by 24.9pp across 13 unseen websites, with gains across every skill family. However, these trajectories teach short-horizon interactions, which do not directly translate to full-task success. We build on this result with MiniWebAgent, which pairs the trained executor with a planner that decomposes full tasks into the skill-level steps it learned to perform. This approach improves task success by 19.4pp points on unseen MiniWeb sites and transfers to external benchmarks, with gains of 9.3pp on WebArena-Lite, 16.1pp on Online-Mind2Web, and 20.0pp on WebVoyager, without training on their trajectories. We provide a simple recipe for compact browser agents: practice reusable skills directly, then compose them to solve unfamiliar tasks.

Minh Pham-Dinh, Md Farhan Ishmam, Hernan Yupanqui +2

University of Utah · Islamic University of Technology

Web AgentsLLM Agent Skill LearningAgent Skill LearningLLM Agent TrainingWeb Agent Benchmarks
AskPDF
May 28, 2026

Revisiting Observation Reduction for Web Agents: Comprehensive Evaluation with a Lightweight Framework

HTML observations in LLM-based web agents are extremely long, and while many reduction methods have been proposed, it remains unclear which methods reduce overall agent latency while maintaining performance. The main obstacle is the high cost of end-to-end evaluation: in our experiments, evaluating 11 methods across 32 configurations on 33 tasks of WorkArena L1 required 232.4 cumulative hours. To address this, we propose a lightweight evaluation framework based on the Minimal Failure Set (MFS), the minimal set of HTML elements whose removal causes task failure. We define coverage as the fraction of instances in which a reduction method fully retains the MFS, which serves as a proxy metric that requires neither web access nor LLM inference. We validate that coverage strongly correlates with end-to-end success rate, with over 100×\times speedup in cumulative evaluation time on both benchmarks. Using this framework, we find that extractive HTML reduction methods require either high computation cost or domain-specific optimization to reduce agent latency while maintaining performance. Building on this, we optimize a pruning program on MFS training data, achieving 2.2×\times faster per-step latency on WorkArena L1 while retaining 84% of the original success rate, and 3.1×\times faster on WebLinx while retaining 89%.

Masafumi Enomoto, Ryoma Obara, Haochen Zhang +1

NEC Corporation

Web Agent BenchmarksLLM Agent EvaluationWeb Agents
AskPDF
May 15, 2026

Skim: Speculative Execution for Fast and Efficient Web Agents

Skim is a speculative execution framework for web agents that exploits the predictable structure of purpose-built websites. Today's web-agent expense is not intrinsic to the tasks but a property of how agents are composed: frontier-model inference, browser rendering, and ReAct-style planning are applied to every step of every task regardless of complexity. Skim's key observation is that websites enforce stable URL patterns, answer formats, and task-to-trajectory mappings across queries of the same type, so most queries can bypass these heavyweight components entirely. An offline profiler captures these patterns once per site. At runtime, Skim matches each query to a template, synthesizes the destination URL, and extracts the answer with a small model. A lightweight verifier gates each fast-path output against the query and schema; rare misspeculations cascade to the full agent, warm-started by the fast path's final URL to preserve upstream trajectory progress. Across standard web-agent benchmarks paired with three backboneagents (WebVoyager, AgentOccam, BrowserUse), Skim reduces median per-task cost by 1.9x and latency by 33.4% with no accuracy loss.

Mike Wong, Kevin Hsieh, Suman Nath +1

Princeton University · Princeton, USA · Microsoft Research +1

LLM Inference EfficiencyLLM Agent Workflow OptimizationWeb Agents
AskPDF