Web Agents Should Use Typed Actions Instead of Click-Based Browsing
Organizations: The Ohio State University · University of British Columbia · Microsoft Research
Abstract
This position paper argues that building a reliable agentic Web requires shifting from low-level interaction primitives to typed actions supported by a semantic layer. Today's web agents primarily operate through clicks, keystrokes, and DOM manipulation, which leads to brittle long-horizon behavior, high execution cost, and limited auditability. We propose web verbs as a concrete design for this layer. A verb exposes a web operation as a typed function with structured inputs, structured outputs, and documented behavior, whether it is backed by a server-side Web API or a maintained client-side workflow. Verb calls can carry preconditions, postconditions, policy tags, and logging hooks, allowing agents to synthesize concise programs with explicit control flow and data flow and to produce checkable execution traces. Using representative case studies, we illustrate how verb-level composition can produce correct, reproducible outcomes, while browser agents using low-level interaction primitives may produce brittle behavior or incorrect reasoning. We conclude with a call to action on standardization, developer tooling, and community processes needed to make this semantic layer deployable and trustworthy at web scale.
Figures & tables
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Website | Verb | Description |
|---|---|---|
| Amazon | clearCart | Clears all items from the Amazon shopping cart. |
| addItemToCart | Adds an item to the cart and returns cart information. @param item : The item to search for and add to cart @return CartResult containing cart items and their prices. | |
| GoogleMaps | get_direction | Gets travel information between two locations. @param source : The starting location @param destination : The destination location @return DirectionResult containing travel time, distance, and route. |
| get_nearestBusinesses | Retrieves a list of nearby businesses based on a reference point and description. @param referencePoint : The location from which to find nearby businesses (e.g., ”Seattle, WA”). @param businessDescription : The type of business to search for (e.g., ”coffee shop”). @param maxCount : The maximum number of businesses to return. @return NearestBusinessesResult containing a list of the nearest businesses. | |
| createList | Creates a new list on Google Maps and adds the specified places to it. @param listName : The name of the list to create (e.g., ”Anchorage 2025”). @param places : A list of place names to add to the list. @return true if the list was created successfully and all places were added, false otherwise. | |
| deleteList | Deletes saved lists from Google Maps. @param listName : All lists whose names contain listName will be deleted. @return true if no matching list was found or all such lists were deleted successfully, false otherwise. |
Web Agents
Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.
See all 176 papers