ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation

William Brach; Francesco Zuppichini; Marco Vinciguerra; Lorenzo Padoan

cs.IRFeb 16, 2026

ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation

Authors: William Brach, Francesco Zuppichini, Marco Vinciguerra, Lorenzo Padoan

Organizations: ScrapeGraphAI, San Francisco, United States

Abstract

Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-in ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 18 000+ unique schemas across 15 named languages plus a long-tail Other category, with English and Traditional Chinese covering 88% of detected content, each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels (semantic correctness is out of scope, and raw HTML is deferred beyond v1.0). We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support.

Figures & tables

Figure 1: Schema complexity distributions in ScrapeGraphAI-100k (depth, key count, elements, cyclomatic complexity, composite score) across 93,695 extraction events, highlighting a dense core and a long tail of complex schemas.

Figure 2: Schema validation rate versus complexity (depth, key count, composite score), the dashed line marks the corpus mean and shows declining validity at higher complexity.

Figure 3: Response size distribution for 93,695 extraction outputs (log-scale y-axis), with most responses under 100 KB and a long tail approaching 1 MB.

Dataset	Size	Lang.	Input	Output
C4	172B	English	Text
FineWeb	15T	English	Text
WebMainBench	7,887	English	HTML	Markdown
WCE Benchmark	$\sim$ 7K	English	HTML	Text
SWDE	124k	English	HTML	KV
OpenCeres	2M	English	HTML	Triples

Table 1: Comparison of ScrapeGraphAI-100k with existing datasets.

Figure 4: Overall BLEU vs. model size on the evaluation set. Circles are Qwen3 baselines (1.7B/4B/30B-A3B) and the star is our fine-tuned 1.7B model, which gains +25.7% over the 1.7B baseline and comes within 3.0% of the 30B-A3B (3.3B active) MoE model.

Validity Metrics
Metric	Qwen3 1.7B	Qwen3 4B	Qwen3-30B-A3B (3.3B active)	Ours
Valid JSON	0.9544	0.9681	0.9913	0.9558
Schema compliant	0.7954	0.8301	0.9363	0.9167
Key-based Metrics
Key precision	0.7332	0.8029	0.8997	0.8988
Key recall	0.7477	0.8069	0.8956	0.8864

Table 2: Extraction accuracy on the fine-tuning subset test split (N=2,714 after context-length filtering). Best results per metric in bold .

\downarrow

indicates lower is better.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Identifiers
Feature Name	Description
id	Unique identifier for each extraction event
source	Source URL (domain-masked if sensitive)
schema_hash	SHA256 hash for deduplication
Input Content
prompt	Natural language instruction

Appendix

Table 3: Dataset Feature Summary

Metric	Mean	Std	Median	P25	P75	Max
Exec. Time (s)	14.85	75.29	8.49	4.86	15.19	14,345
Response Size (B)	19,772	48,476	10,007	790	21,687	991,323
Schema Size (B)	1,161	1,012	1,080	1,058	1,106	56,254
Schema Depth	4.93	0.74	5.0	5.0	5.0	9.0
Schema Keys	41.40	42.10	37.0	37.0	37.0	3,060
Schema Elements	47.95	55.24	41.0	41.0	41.0	3,769

Appendix

Table 4: Descriptive Statistics of ScrapeGraphAI-100k Dataset (N=93,695)

Figure 5: Correlation matrix of dataset metrics, showing strong correlations among complexity measures and a negative relationship between complexity and validation.

Figure 6: Content language distribution (15 named languages plus an Other bucket) for 92,968 entries after removing unknown detections. Percentages are computed over the filtered subset.

Language	Count	Percentage (%)
eng_Latn	49,187	52.91
yue_Hant	33,043	35.54
kor_Hang	5,389	5.80
deu_Latn	1,005	1.08
nld_Latn	669	0.72
ltz_Latn	463	0.50

Appendix

Table 5: Top content languages in ScrapeGraphAI-100k after filtering unknown detections (N=92,968).

Schema-Constrained Generation

See all 12 papers

ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
Jul 31, 2026Boyang Zhang, Adrian Lyjak, Eli Stewart +2LLM Grounding LLM Agent Evaluation
SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extraction
May 4, 2026Sin Yu Bonnie Ho, Arlie Coles, Erik Larsson +3Retrieval-Augmented Generation LLM Inference Acceleration
Microsoft
STAGE: Source-grounded Text-to-JSON Artifact GEneration
Jun 18, 2026Sunghee Ahn, Guijin Son, Youngjae YuGrounded Text Generation Synthetic Data Generation
Seoul National University

Jul 31, 2026J/K move · Enter open · S save

ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction

Boyang Zhang, Adrian Lyjak, Eli Stewart, Zhaoqi Li+1

LLM Grounding LLM Agent Evaluation Document Information Extraction

Open paper AskPDF arXiv

Enterprise workflows increasingly rely on agents for schema-guided extraction: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on HuggingFace and GitHub.

Abstract

Schema-Constrained Generation

Explore similar work

ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction

SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extraction

STAGE: Source-grounded Text-to-JSON Artifact GEneration