When 200 OK Is Not an Answer: Silent Fetch Failures in Agentic Web Content Acquisition

Shuyang Zhang

cs.IROct 31, 2026

When 200 OK Is Not an Answer: Silent Fetch Failures in Agentic Web Content Acquisition

Authors: Shuyang Zhang

Organizations: The Hong Kong Polytechnic University

Abstract

Web agents and answer engines fetch pages at question time, and liveness checks of cited sources treat an HTTP 200 response as success. We measure how often that assumption fails and what it does to measurements built on fetched pages. We fetched 10,307 URLs in three days with a polite static client and a rendered browser client: pages from 1,507 sites in five strata, random non-existent paths on every site, and the 4,795 URLs cited by expert-verified claims in ExpertQA. We define a taxonomy of what a 200 response contains, label the static 200 responses automatically, and check the labels against 300 items reviewed by an LLM-based assistant; a human check of 60 of them retained every reviewed usability label after discussion. By the automatic labels, 20.9% of the pages that a static client receives with status 200 do not contain the page's main content: app shells, bot challenges, block pages, error and placeholder pages, and partial content. On the held-out test split, the reviewed sample lowers this estimate from 25.5% to 20.0%. 13.1% of the sites whose random-path probe was answered returned 200. The error propagates: for the cited sources, treating 200 as content overstates the share of claim-source pairs that can be checked against live content by 12.2 points and inflates the apparent silent citation-drift rate from 5.8% to 21.9%. Cheap heuristics catch most of it. Text length, probe comparison and redirect-to-root, combined and tuned on held-out sites, recover 86% of unusable reviewed responses at a false-alarm rate near 5%. Content-reading detectors perform comparably: the best, the text classifier, gains five to seven points of recall at more than twice the false-alarm rate. We argue that agents and their evaluations should report fetch outcomes explicitly instead of inferring them from the status line and will release the labelled data and code.

Figures & tables

	Type	Example	Auto	Rev.
A	Content	article, product, documentation or listing text	3,575	199
B	Block	geo-block (“location not supported”); WAF “request unsuccessful”	83	3
C	Challenge	Cloudflare Turnstile; publisher “Client Challenge”; image captcha	159	12
D	Shell	product grid that only says “Loading…”; live scores missing; empty body	354	48
E	Error or substitute	random path redirected to the homepage; unrelated page served at a random path	341	31
F	Parked	hosting placeholder; parked domain redirecting to a lander	27	6

Table 1. Outcome types for 200 responses, with examples from the reviewed sample. Auto: automatic type of all 4,560 static 200 responses (pages and probes). Rev.: reviewed type of the 300 sampled items.

ID	Population	Sites	Pages
S1	Links listed in llms.txt files ( Gerrish, 2026 )	300	595
S2	Tranco ( Le Pochat et al., 2019 ) top 10k	400	631
S3	Markdown-serving sites in the top 10k	107	186
S4	Tranco ranks 10,001–100,000	400	622
S5	Tranco ranks 100,001–1,000,000	300	464
P	URLs cited in ExpertQA ( Malaviya et al., 2024 )	3,383	4,795

Table 2. Populations. Pages are fetched with both clients; every site in S1–S5 also receives two random-path probes.

Figure 1. Outcome of the static fetch for every page, by population. Grey: status 200 with main content. Blue, orange and green: status 200 without usable content (automatic labels,

r<0.5

); their total share of all pages is printed on the right. Other colours: explicit outcomes.

	Pages	200	Unus.	CI	Reviewed	Probe	Blocked	Agree
S1	595	90.6	9.8	7–13	5.4	10.5	5.9	96
S2	631	69.4	26.3	22–31	23.1	13.3	17.1	79
S3	186	91.4	17.1	11–24	7.0	17.3	3.8	87
S4	622	66.1	20.9	17–25	19.0	13.7	18.6	88
S5	464	71.1	16.4	12–21	12.5	13.1	14.7	89
P	4,795	52.1	23.2	19–28	25.3	–	25.7	91

Table 3. Static outcomes by population, in %. 200: share of pages answered with 200. Unusable: share of those 200 responses that are unusable (automatic labels,

r<0.5

), with a 95% site-bootstrap interval, and reweighted from the reviewed test sample (test split only, 36–57 pages per population, no interval). Probe: share of sites whose random-path probe was answered that answered with 200. Blocked: pages answered 401/403/429. Agree: usability agreement of automatic and reviewed labels on the reviewed items (pages and probes).

Figure 2. Cumulative distribution of retention

r

(mean share of the rendered page’s sentences and numeric facts found in the static text) for pages where both clients returned content.

	Reviewed sample				Automatic		Oracle
Detector	Recall	CI	Pages	FA	Recall	FA	Recall
H0	0.00	0.00–0.00	0.00	0.00	0.00	0.00	–
H1	0.25	0.14–0.37	0.25	0.02	0.26	0.03	–
H2	0.76	0.66–0.85	0.81	0.07	0.65	0.04	0.66
H3	0.35	0.26–0.46	0.07	0.03	0.15	0.02	0.35
H4	0.10	0.05–0.16	0.06	0.00	0.08	0.00	–

Table 4. Detectors on the test split. Recall on unusable and false alarm (FA) on usable items, on the reviewed sample (99 unusable, 195 usable) and on all test items with automatic labels (272 / 684). Thresholds fixed on validation at FA

\leq

5%. CI: 95% domain bootstrap. Last column: recall at FA

\leq

5% with the threshold chosen on the reviewed sample itself (descriptive only). Pages: recall on the 68 unusable pages, excluding the 31 random-path probes (FA is unchanged, as all usable items are pages).

Figure 3. Recall by reviewed type (types with at least 12 unusable items) and false-alarm rate on usable items, reviewed test sample. Thresholds fixed on validation.

Policy	Usable %	Missed %	Rendered %	Time (s)
Never render	74.3	25.7	0.0	2.5
Always render	80.3	0.0	100.0	20.6
Render if H-best flags	81.4	7.8	19.6	5.8
Render if M2 flags	81.6	6.4	22.9	6.6
Render if M3 flags	81.8	6.2	23.6	6.7

Table 5. Fetch policies on the 920 test-split pages with a static 200 (automatic labels as outcome). Usable: page content obtained. Missed: unusable static response used without escalation. Rendered: share of pages rendered. Time: mean seconds per page (static plus rendered when escalated).

		Status only	H-best	M1	M2	M3	Labels
All cited	Live, usable	52.6	45.1	45.4	–	–	40.5
	Live, unusable	–	7.5	7.2	–	–	12.2
	Silent drift	21.9	10.2	11.4	–	–	5.8
Test split	Live, usable	56.1	42.7	46.8	41.0	40.4	38.3
	Live, unusable	–	13.5	9.3	15.1	15.7	17.8
	Silent drift	31.9	12.6	18.1	11.5	9.1	6.3

Table 6. Cited sources (pairs). Share of pairs whose static fetch is live and usable or live and unusable, and silent-drift share among usable live pairs whose 2023 passage supported the claim, in %, under each way of judging 200 responses. M2 and M3 were trained on train-split items, so they are shown on the test split only. Dead: 11.0% of all pairs and 12.0% of test pairs; failed (blocked, other errors, transport failures, robots.txt exclusions): 36.4% and 31.8%.

		Pairs %		Drift %
System	Pairs	Live	Usable	Status	Labels
Post hoc, Google	1,888	55.5	45.3	11.5	3.8
Retrieve-and-read, Google	1,553	57.2	46.9	16.5	5.3
Post hoc, Sphere	1,517	47.7	33.4	30.1	10.3
Retrieve-and-read, Sphere	1,198	48.4	33.4	33.3	6.2

Table 7. Cited sources by ExpertQA answer system. Live: static 200, in % of pairs; usable: also labelled usable. Drift: silent-drift share counting every 200 as content (status) and only usable responses (labels).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

	S1	S2	S3	S4	S5	P
$n$ (unus./us.)	15/30	19/37	5/33	22/30	14/32	24/33
H-best	0.93 / 0.03	0.79 / 0.05	0.80 / 0.03	0.95 / 0.10	0.86 / 0.03	0.79 / 0.09
M1	0.60 / 0.00	0.37 / 0.08	0.60 / 0.09	0.59 / 0.10	0.50 / 0.06	0.71 / 0.00
M2	0.93 / 0.10	0.84 / 0.16	1.00 / 0.03	0.95 / 0.13	0.93 / 0.09	0.88 / 0.27
M3	0.80 / 0.00	0.68 / 0.27	1.00 / 0.03	0.86 / 0.13	0.79 / 0.16	0.88 / 0.18

Appendix

Table 8. Recall / false-alarm rate by population on the reviewed test sample, thresholds fixed on validation.

Web Agents

Web agents use language and vision models to navigate websites, gather information, and complete tasks through interactive interfaces.

See all 186 papers

Oct 26, 2026

Proof-of-Agentness: Towards Verifying Agents on the Web

CAPTCHAs and bot detection have helped protect Web services from automated abuse for decades by distinguishing humans from bots. However, the emergence of LLM-powered agents introduces a new setting: Web services increasingly accommodate agents as legitimate users. Without a way to distinguish agents from other automated clients such as simple bots and scripts, these services risk large-scale automated junk submissions that waste service resources and consume users' attention or tokens. In this paper, we investigate a novel question for today's Web: "How can a Web service obtain evidence that a client is an agent?" We introduce the concept of "Proof-of-Agentness" (PoA), a mechanism for distinguishing agents from other automated clients, motivated by the parallel between human attention and an agent's inference resources. We outline two approaches: active verification through challenge-response and passive analysis of service-visible signals. Eight language models achieve an overall accuracy of 96.5% on our challenges, with an average overhead of 1,647 tokens. For passive analysis, we conduct an IRB-approved user study comparing the Web interactions of twelve human participants and representative Bot and Agent implementations, achieving an average classification accuracy of 95.8% across four classifiers, motivating further study.

Zuyao Xu, Xiang Li

Nankai University, Tianjin, China

LLM Agent Verification Web Agents

Ask PDF

Feb 5, 2026

Do Web Agents Investigate Before They Decide?

Autonomous web agents are increasingly deployed in moderation and policy enforcement, where correct decisions often depend on evidence that is not immediately visible and must be actively investigated. Yet existing benchmarks largely assume task critical information is immediately accessible. They do not measure investigative competence: recognizing when visible context is insufficient, retrieving hidden evidence, and integrating it into a final decision. We introduce MIRAGE, a benchmark of 750 multi step decision tasks across three domains: Wikipedia Forensics, Shopping Admin adjudication, and Reddit Moderation. Each task has two layers: a visible surface context that often points to the wrong action, and a hidden context, reachable only by active investigation, that contains the decisive evidence. We decompose agent performance into Investigation, Reasoning, and Decision Accuracy, complemented by an Investigative Hallucination Rate. We evaluate eight LLM agents across two model generations. Three patterns emerge. First, agents reach relevant pages but rarely extract the decisive evidence on them. Second, procedural hints improve investigation but do not consistently improve decisions on Wikipedia tasks, where decisive evidence often contradicts surface impressions. Third, 12.6% of trajectories cite fabricated facts. We call these the Navigation Discovery Gap, Collapse under Contradiction, and Investigative Hallucination. These patterns persist across model scale, generation, and reasoning architecture.

Syed Nazmus Sakib, Nafiul Haque, Tapodhir Karmakar Taton +2

Department of Robotics and Mechatronics Engineering, University of Dhaka · Department of Computer Science and Engineering, University of Dhaka

Web Agent Benchmarks Hallucination in Language Models LLM Agent Evaluation Web Agents

Ask PDF

Jul 11, 2026

Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these interfaces as decision surfaces on a fixed sample of 100 questions from the 254-question SealQA-Hard subset, using one frozen GPT-5.4 agent, a fixed orchestration harness, and a shared page-fetch backend across Brave, Tavily, and Firecrawl. A Kimi-K2.6 oracle labels visible URL-level evidence; a separate answer audit yields 25, 25, and 26 correct answers out of 100. These counts indicate similar observed accuracy but do not establish equivalence. Under the tested configurations, Brave exposes more pre-fetch support alongside a larger snippet surface; Tavily has a larger rank-1 share among trajectory-pooled supporting observations; and Firecrawl is associated with broader exploration. First-search query-level metrics distinguish support availability from ranking, while contradiction exposure complements contradiction-to-gold ratios. A retrospective oracle union covers 44/100 questions, versus 26/100 for the best individual provider: 18 percentage points of headroom. The observed evidence and action differences motivate evaluating search APIs jointly with agent policy and retrieval budget.

Sriram Selvam, Anneswa Ghosh

Agentic Search IR Evaluation Evidence Retrieval

Ask PDF

Abstract

Web Agents

Explore similar work

Proof-of-Agentness: Towards Verifying Agents on the Web

Do Web Agents Investigate Before They Decide?

Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents