Web information extraction (WIE) is the task of automatically extracting data from web pages, offering high utility for various applications. The evaluation of WIE systems has traditionally relied on benchmarks built from HTML snapshots captured at a single point in time. However, this offline evaluation paradigm fails to account for the temporally evolving nature of the web; consequently, performance on these static benchmarks often fails to generalize to dynamic real-world scenarios. To bridge this gap, we introduce \dataset, a new benchmark designed for evaluating WIE systems directly against live websites. Based on trusted and permission-granted websites, we curate natural language queries that require information extraction of various data categories, such as text, images, and hyperlinks. We further design these queries to represent four levels of complexity, based on the number and cardinality of attributes to be extracted, enabling a granular assessment of WIE systems. In addition, we propose Visual Grounding Scraper (VGS), a novel multi-stage agentic framework that mimics human cognitive processes by visually narrowing down web page content to extract desired information. Extensive experiments across diverse backbone models demonstrate the effectiveness and robustness of VGS. We believe that this study lays the foundation for developing practical and robust WIE systems.
Figures & tables
Figure 1: Overview of the conventional WIE paradigm’s limitations and our solutions involving a new benchmark and an extraction method. (A) While existing offline benchmarks are constructed from static HTML snapshots, LiveWeb-IE evaluates WIE systems on live websites to reflect the evolving nature of the web. (B) On a complex live web page, methods that process full HTML often fail, whereas VGS leverages visual cues from the rendered page for accurate information extraction.
Benchmarks
Task
Modality
Online
# Domain
# Website
# Query
SWDE ( Hao et al., 2011 )
Closed IE
Text
✗
8
80
32
WEIR ( Bronzi et al., 2013 )
Closed IE
Text
✗
4
40
32
DS1 ( Omari et al., 2017 )
Closed IE
Text
✗
4
30
11
Expanded SWDE ( Lockard et al., 2019 )
Open IE
Text
✗
3
21
748
PLAtE ( San et al., 2023 )
Closed IE
Text
✗
1
43
3
LiveWeb-IE (Ours)
On-Demand IE
Text & Image
✓
8
15 (46 † )
342
Table 1: Comparison between LiveWeb-IE and other web information extraction benchmarks. The query in ClosedIE corresponds to attributes, and in OpenIE to predicates. † denotes the page groups, clusters of structurally similar web pages within the website.
Figure 2: Dataset construction pipeline for LiveWeb-IE . We first select a diverse set of websites and group the web pages within each website by layout. We then annotate the attributes, queries, and values for each page group, followed by a human verification process to ensure data quality.
Figure 3: The framework of VGS. It sequentially narrows the observation space, from identifying target attributes, grounding the region, pinpointing the exact items, and generating the XPaths.
Figure 4: The task type and data category distribution.
Models
Method
Type I
Type II
Type III
Type IV
Overall
P
R
F1
P
R
F1
P
R
F1
P
R
F1
P
R
F1
Proprietary Models
Gemini-2.5-Flash
COT
32.04
49.01
34.36
32.19
54.80
35.17
9.66
13.99
8.19
10.16
10.90
7.40
20.36
31.33
20.53
Reflexion
33.00
48.31
35.96
39.87
39.59
39.40
10.29
13.00
8.30
12.83
23.11
8.67
23.53
29.66
22.39
AutoScraper
36.52
42.35
37.25
31.47
55.76
34.39
10.94
18.61
10.23
14.36
23.75
9.66
22.41
34.82
22.02
VGS
47.83
55.85
49.02
42.36
53.76
44.82
43.61
43.49
42.92
38.60
38.32
38.13
42.81
47.46
43.44
Table 2: Main experimental results on LiveWeb-IE , measured by Precision (P), Recall (R), and F1 score (F1). The best score for each metric is highlighted in bold .
Table 3: Ablation study on visual grounding and element pinpointing. We report the F1 score.
Figure 5: F1 score comparison on existing web information extraction benchmarks. VGS generally outperforms baselines across different backbone models. Full results are shown in Table 5
Figure 6: F1 score comparison of VGS and baseline methods across three data categories.
Figure 7: Distribution of error types for VGS across different backbone models.
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: F1 score and extraction time comparison between VGS and a direct extraction approach across the four task types. The extraction time is measured in seconds (s) per page group.
Figure 9: Performance variation across distances.
Model
Method
SWDE-sub
SWDE-2025
Δ F1
P
R
F1
P
R
F1
GPT-4o
CoT
95.05
85.0
82.62
97.94
70.32
68.95
-13.67
Reflexion
91.98
95.53
90.70
98.47
73.24
73.86
-16.93
AutoScraper
99.47
90.79
92.66
99.06
77.48
77.83
-14.83
VGS (ours)
98.12
96.27
94.53
98.21
78.94
86.38
-8.15
Qwen-2.5-72B
CoT
99.47
71.58
73.32
95.08
57.38
56.33
-16.99
Appendix
Table 4: Performance comparison between the SWDE-sub and SWDE-2025. The SWDE-sub refers to a snapshot manually sampled from the original SWDE dataset, where the attributes and values remain identical to the live web pages. SWDE-2025 denotes the corresponding web pages captured in November 2025. Δ F1 represents the difference in F1 scores, obtained by subtracting SWDE-sub from SWDE-2025, which highlights the consistent performance decline across all approaches.
Figure 10: Efficiency comparison between VGS and LLM-based wrapper generation baselines. The character length represents the average HTML character length of the web pages for each data sample.
Models
Method
SWDE
Expanded SWDE
P
R
F1
P
R
F1
Proprietary Models
Gemini 2.5 Flash
COT
92.62
76.82
75.33
88.47
65.08
63.74
Reflexion
94.31
84.53
84.40
89.50
80.14
79.22
AutoScraper
93.87
78.79
78.07
88.58
74.14
73.44
VGS
96.20
86.58
88.42
95.76
81.88
83.19
Appendix
Table 5: Results on existing benchmarks.
Domain
Website
Attributes
# web pages
NBAPlayer
espn
name
10
nba
name
10
usatoday
height, name
10
yahoo
height, name, weight
10
wiki
height, name, weight
10
Movie
allmovie
director, title
10
Appendix
Table 6: Statistics of SWDE-sub, a manually curated subset where target values remain identical to recent web pages.
Figure 11: An example of a web page excluded due to text overlap.
Domain
Website
Attribute
# web page
# Group
Academic
ArXiv
author
87
3
author profile link
pdf link
subject
title
ICLR
abstract link
292
2
Appendix
Table 7: Dataset statistic (Part 1): Academic and Auto domains.
Figure 12: VGS prompt template for attribute identification.
Figure 13: VGS prompt template for visual grounding.
Figure 14: VGS prompt template for element pinpointing (element scanning).
Figure 15: VGS prompt template for element pinpointing (element selection).
Figure 16: VGS prompt template for Xpath synthesis.
Figure 17: CoT prompt template for top-down operation.
Figure 18: CoT prompt template for synthesis operation.
Figure 19: Reflexion prompt template for top-down operation.
Figure 20: Reflexion prompt template for self-reflection operation.
Figure 21: Reflexion prompt template for synthesis operation.
Figure 22: AutoScraper prompt template for top-down operation.
Figure 23: AutoScraper prompt template for step-back operation.
Figure 24: AutoScraper prompt template for synthesis operation.
Figure 25: LLM Extractor prompt template.
Figure 26: Complete VGS trajectory for Type I task on LiveWeb-IE .
Figure 27: Complete VGS trajectory for Type II task on LiveWeb-IE .
Figure 28: Complete VGS trajectory for Type III task on LiveWeb-IE .
Figure 29: Complete VGS trajectory for Type IV task on LiveWeb-IE .
Figure 30: Comparison of layout changes on the DPLA website over time. (A) illustrates the web page from a past version (January 2018), which was retrieved from the Web Archive ( https://web.archive.org ), a digital archive preserving over 1 trillion historical web page snapshots, while (B) depicts the current version (November 2025) of the same URL. The attributes “title”, “article image”, and “article link” retain identical values across both versions, but the visual layout and underlying DOM structure of the page have diverged. By targeting attributes whose information remains consistent regardless of layout changes, we enable a stable evaluation of a WIE system’s information extraction performance on web pages as they exist at the current moment.
Figure 31: Comparison of layout changes on the TheMealDB website over time. (A) illustrates a specific meal page from a past version (October 2017), while (B) depicts the current version (November 2025) of the same URL. The attributes “flag”, “ingredient image”, “ingredient link”, “meal image”, and “meal” retain identical values across both versions, but the visual layout and underlying DOM structure of the page have diverged.
Web content extraction - isolating a page's main content from surrounding boilerplate - is a prerequisite for search indexing, retrieval-augmented generation, NLP dataset construction, and large language model training. Progress in this area has been constrained by the limitations of existing evaluation benchmarks, which are small (100-800 pages), restricted to news articles, or based on web pages from over a decade ago. We introduce the Web Content Extraction Benchmark (WCXB), a dataset of 2,008 web pages from 1,613 domains spanning seven structurally distinct page types: articles, forums, products, collections, listings, documentation, and service pages. The dataset includes a 1,497-page development set and a 511-page held-out test set with matched page type distributions. Ground truth annotations were produced through a five-stage pipeline: LLM-assisted drafting, automated verification, four-pass frontier model review, snippet and quality verification scripts, and human review. We evaluate 13 extraction systems - 11 heuristic and 2 neural - and find that while top systems converge on articles (F1 = 0.93), performance diverges sharply on structured page types (F1 = 0.41-0.84), revealing blind spots invisible to existing article-only benchmarks. The dataset is released under CC-BY-4.0 with HTML source files, ground truth annotations, page type labels, and baseline results.
The abundant and heterogeneous nature of web content necessitates automated information extraction, and generating scrapers that can be reused across similar web pages offers an effective solution for scalable data extraction. In this work, we propose Co-Scraper, a two-stage framework capable of handling the hierarchical complexity of long HTML documents. By integrating a query-aware DOM pruning mechanism with stable extraction strategy induction, Co-Scraper can effectively transforms web content into executable programmatic wrappers using a fine-tuned Qwen3-8B model. On the test set of SWDE, Co-Scraper achieves state-of-the-art performance with an F1 score of 94.78% and a reuse success rate of 90.39%. This framework significantly enhances the accuracy and resilience of data extraction, providing a highly efficient approach for web data acquisition tasks.
Shoupeng Wang, Jiantao Qiu, Wuyang Zhang +1
Shanghai Artificial Intelligence Laboratory, OpenDataLab · University of Science and Technology of China
Modern web scraping struggles with dynamic, interactive websites that require more than static HTML parsing. Current methods are often brittle and require manual customization for each site. To address this, we introduce Webscraper, a framework designed to handle the challenges of modern, dynamic web applications. It leverages a Multimodal Large Language Model (MLLM) to autonomously navigate interactive interfaces, invoke specialized tools, and perform structured data extraction in environments where traditional scrapers are ineffective. Webscraper utilizes a structured five-stage prompting procedure and a set of custom-built tools to navigate and extract data from websites following the common ``index-and-content'' architecture. Our experiments, conducted on six news websites, demonstrate that the full Webscraper framework, equipped with both our guiding prompt and specialized tools, achieves a significant improvement in extraction accuracy over the baseline agent Anthropic's Computer Use. We also applied the framework to e-commerce platforms to validate its generalizability.
Guan-Lun Huang, Yuh-Jzer Joung
Dept. of Information Management, National Taiwan University, Taipei, Taiwan