cs.ETApr 1, 2026

Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures

Authors: Lewis Tham, Nicholas Mac Gregor Garcia, Jungpil Hahn

Organizations: Unbrowse AI · School of Computing, National University of Singapore

Abstract

Autonomous agents increasingly interact with the web, yet most websites remain designed for human browsers -- a fundamental mismatch that the emerging ``Agentic Web'' must resolve. Agents must repeatedly browse pages, inspect DOMs, and reverse-engineer callable routes -- a process that is slow, brittle, and redundantly repeated across agents. We observe that every modern website already exposes internal APIs (sometimes called shadow APIs) behind its user interface -- first-party endpoints that power the site's own functionality. We present Unbrowse, a shared route graph that transforms browser-based route discovery into a collectively maintained index of these callable first-party interfaces. The system passively learns routes from real browsing traffic and serves cached routes via direct API calls. In a single-host live-web benchmark of equivalent information-retrieval tasks across 94 domains, fully warmed cached execution averaged 950,ms versus 3{,}404,ms for Playwright browser automation (3.6×\times mean speedup, 5.4×\times median), with well-cached routes completing in under 100,ms. A three-path execution model -- local cache, shared graph, or browser fallback -- ensures the system is voluntary and self-correcting. A three-tier micropayment model via the x402 protocol charges per-query search fees for graph lookups (Tier3), a one-time install fee for discovery documentation (Tier1), and optional per-execution fees for site owners who opt in (Tier~2). All tiers are grounded in a necessary condition for rational adoption: an agent uses the shared graph only when the total fee is lower than the expected cost of browser rediscovery.

Figures & tables

Explore similar work

CardsList
  1. Skim: Speculative Execution for Fast and Efficient Web Agents

    May 15, 2026Mike Wong, Kevin Hsieh, Suman Nath +1LLM Inference EfficiencyLLM Agent Workflow Optimization

  2. WebNavigator: Global Web Navigation via Interaction Graph Retrieval

    Mar 20, 2026Xuanwang Zhang, Yuteng Han, Jinnan Qi +5Web AgentsGraph-Based Planning

  3. AutoTailor: Automatic, User-Aligned Capability Selection and Adaptation for Web Agents

    Sep 11, 2026Xinyun Cao, Adriana Szekeres, Fazle Elahi FaisalWeb AgentsLLM Agents