Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-in ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 18 000+ unique schemas across 15 named languages plus a long-tail Other category, with English and Traditional Chinese covering 88% of detected content, each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels (semantic correctness is out of scope, and raw HTML is deferred beyond v1.0). We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support.
Figures & tables
Figure 1: Schema complexity distributions in ScrapeGraphAI-100k (depth, key count, elements, cyclomatic complexity, composite score) across 93,695 extraction events, highlighting a dense core and a long tail of complex schemas.
Figure 2: Schema validation rate versus complexity (depth, key count, composite score), the dashed line marks the corpus mean and shows declining validity at higher complexity.
Figure 3: Response size distribution for 93,695 extraction outputs (log-scale y-axis), with most responses under 100 KB and a long tail approaching 1 MB.
Dataset
Size
Lang.
Input
Output
C4
172B
English
Text
FineWeb
15T
English
Text
WebMainBench
7,887
English
HTML
Markdown
WCE Benchmark
∼ 7K
English
HTML
Text
SWDE
124k
English
HTML
KV
OpenCeres
2M
English
HTML
Triples
Table 1: Comparison of ScrapeGraphAI-100k with existing datasets.
Figure 4: Overall BLEU vs. model size on the evaluation set. Circles are Qwen3 baselines (1.7B/4B/30B-A3B) and the star is our fine-tuned 1.7B model, which gains +25.7% over the 1.7B baseline and comes within 3.0% of the 30B-A3B (3.3B active) MoE model.
Metric
Qwen3 1.7B
Qwen3 4B
Qwen3-30B-A3B (3.3B active)
Ours
Validity Metrics
Valid JSON
0.9544
0.9681
0.9913
0.9558
Schema compliant
0.7954
0.8301
0.9363
0.9167
Key-based Metrics
Key precision
0.7332
0.8029
0.8997
0.8988
Key recall
0.7477
0.8069
0.8956
0.8864
Table 2: Extraction accuracy on the fine-tuning subset test split (N=2,714 after context-length filtering). Best results per metric in bold . ↓ indicates lower is better.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Feature Name
Description
Identifiers
id
Unique identifier for each extraction event
source
Source URL (domain-masked if sensitive)
schema_hash
SHA256 hash for deduplication
Input Content
prompt
Natural language instruction
Appendix
Table 3: Dataset Feature Summary
Metric
Mean
Std
Median
P25
P75
Max
Exec. Time (s)
14.85
75.29
8.49
4.86
15.19
14,345
Response Size (B)
19,772
48,476
10,007
790
21,687
991,323
Schema Size (B)
1,161
1,012
1,080
1,058
1,106
56,254
Schema Depth
4.93
0.74
5.0
5.0
5.0
9.0
Schema Keys
41.40
42.10
37.0
37.0
37.0
3,060
Schema Elements
47.95
55.24
41.0
41.0
41.0
3,769
Appendix
Table 4: Descriptive Statistics of ScrapeGraphAI-100k Dataset (N=93,695)
Figure 5: Correlation matrix of dataset metrics, showing strong correlations among complexity measures and a negative relationship between complexity and validation.
Figure 6: Content language distribution (15 named languages plus an Other bucket) for 92,968 entries after removing unknown detections. Percentages are computed over the filtered subset.
Language
Count
Percentage (%)
eng_Latn
49,187
52.91
yue_Hant
33,043
35.54
kor_Hang
5,389
5.80
deu_Latn
1,005
1.08
nld_Latn
669
0.72
ltz_Latn
463
0.50
Appendix
Table 5: Top content languages in ScrapeGraphAI-100k after filtering unknown detections (N=92,968).
Enterprise workflows increasingly rely on agents for schema-guided extraction: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on HuggingFace and GitHub.
Extracting structured data from unstructured text using large language models (LLMs) becomes challenging when target schemas are large and complex. In such cases, including the full schema in the prompt increases cost and latency, risks lost-in-the-middle performance degradation, and can exceed context length limits. We propose SchemaRAG, a retrieval-augmented generation (RAG) framework that dynamically prunes the output schema space for schema-conditioned information extraction tasks by leveraging schema metadata and few-shot examples when available. We evaluate SchemaRAG on real-world healthcare and e-commerce datasets. Our results show that SchemaRAG can achieve up to an 8.8% increase in micro-F1, a 47% reduction in latency, and a 48% reduction in token costs, demonstrating its practicality for large-schema extraction.
From financial filings to clinical records, legacy industries rely heavily on long, unstructured documents to store high-value information. Reliably extracting this information into structured, machine-readable representations is a key prerequisite to making the contents accessible to automated systems. JSON is a natural target for such structured extraction, yet constructing reliable and scalable text-to-JSON training data remains challenging. To address this gap, we propose STAGE (Spreadsheet-grounded Text-to-JSON Artifact GEneration), a source-grounded data generation pipeline that constructs reports and JSON schema by using LLMs for scalable synthesis while validating ground-truth values against the underlying spreadsheet. Evaluations on STAGE-Eval, our source-grounded benchmark with an 851-example test set, show that STAGE produces stronger training data than existing approaches. This improves Qwen3-4B exact match from 31.37% to 74.27% and value accuracy from 45.46% to 90.69%.