Coverage-Aware Web Crawling for Domain-Specific Supplier Discovery via a Web--Knowledge--Web Pipeline

Yijiashun Qi; Yijiazhen Qi; Tanmay Wagh

cs.LGFeb 27, 2026

Coverage-Aware Web Crawling for Domain-Specific Supplier Discovery via a Web--Knowledge--Web Pipeline

Authors: Yijiashun Qi, Yijiazhen Qi, Tanmay Wagh

Organizations: University of Michigan Ann Arbor, USA · The University of Hong Kong Hong Kong, China · Santa Clara University Santa Clara, USA

Abstract

Identifying the full landscape of small and medium-sized enterprises (SMEs) in specialized industry sectors is critical for supply-chain resilience, yet existing business databases suffer from substantial coverage gaps -- particularly for sub-tier suppliers and firms in emerging niche markets. We propose a \textbf{Web--Knowledge--Web (W $\to$ K $\to$ W)} pipeline that iteratively (1)~crawls domain-specific web sources to discover candidate supplier entities, (2)~extracts and consolidates structured knowledge into a heterogeneous knowledge graph using domain-adapted few-shot LLM prompting, and (3)~uses the knowledge graph's topology and coverage signals to guide subsequent crawling toward under-represented regions of the supplier space. To quantify discovery completeness, we introduce a coverage estimation framework inspired by ecological species-richness estimators (Chao1, ACE) adapted for web-entity populations. Experiments on the semiconductor equipment manufacturing sector (NAICS 333242) demonstrate that the W $\to$ K $\to$ W pipeline achieves the highest precision (0.165) and F1 (0.123) among all methods while using only 144 pages -- 32% fewer than the 213-page baseline budget -- building a knowledge graph of 664 entities and 542 relations with 100% relation type-consistency.

Figures & tables

Relation	Source	Target	Definition
supplies_to	CO	CO	Provides components/equipment
partners_with	CO	CO	Formal partnership or JV
produces	CO	PROD	Manufactures or sells product
located_in	CO	LOC	HQ, office, or facility
belongs_to_sector	CO	SEC	Operates in sector

Table 1. Relation type definitions with schema-level type constraints. These definitions are embedded in the LLM extraction prompt.

Method	Disc.	TP	Prec.	Rec.	F1	Pages
BFS Crawler	160	20	0.125	0.103	0.113	213
Focused Crawler	236	18	0.076	0.092	0.084	213
W $\to$ K (no feedback)	160	20	0.125	0.103	0.113	213
W $\to$ K $\to$ W	115	19	0.165	0.097	0.123	144

Table 2. Supplier discovery performance (NAICS 333242). Baselines use a 213-page budget; W

\to

\to

W uses 144 pages. Precision, recall, and F1 are computed against a ground truth of 195 companies.

Iter	Obs.	$\hat{S}$	$\hat{C}$	$C_{\text{true}}$	Error	$f_{1}$ / $f_{2}$
1	219	2,949	7.4%	8.7%	1.3%	209/8
2	420	2,897	14.5%	9.7%	4.8%	379/29
3	514	3,136	16.4%	9.7%	6.6%	458/40
4	553	3,379	16.4%	9.7%	6.6%	493/43
5	664	4,042	16.4%	9.7%	6.7%	587/51

Table 3. Coverage estimation trajectory over 5 iterations.

\hat{S}

: Chao1 estimate;

\hat{C}

: estimated coverage ratio;

C_{\text{true}}

: true coverage (TP/GT size); Error:

|\hat{C}-C_{\text{true}}|

Figure 2. Cumulative entity and company discovery over five W

\to

\to

W iterations. The discovery rate varies across iterations as the crawler reaches different types of web sources.

Iter	Pages	New Ent.	Total	Companies	Relations
1	24	219	219	50	170
2	44	201	420	71	334
3	18	94	514	77	423
4	13	39	553	85	452
5	45	111	664	115	542

Table 4. Per-iteration discovery breakdown showing cumulative growth in companies and the entity types discovered.

Relation Type	Count	Consistent	Consist. %
supplies_to	2	2	100.0%
partners_with	33	33	100.0%
produces	261	261	100.0%
located_in	85	85	100.0%
belongs_to_sector	161	161	100.0%
Total	542	542	100.0%

Table 5. Relation type-consistency in the final KG (

G_{5}

, 542 relations). Type-consistent relations satisfy schema constraints on source and target entity types.

Relation Type	C	I	A	$n$	$P_{L}$	$P_{S}$
supplies_to	2	0	0	2	100.0%	100.0%
partners_with	9	8	3	20	52.9%	45.0%
produces	16	3	1	20	84.2%	80.0%
located_in	8	12	0	20	40.0%	40.0%
belongs_to_sector	16	3	1	20	84.2%	80.0%
Overall	51	26	5	82	66.2%	62.2%

Table 6. Relation extraction precision from stratified annotation (up to 20 samples per type, 82 total).

P_{L}

: lenient precision (ambiguous excluded).

P_{S}

: strict precision (ambiguous = incorrect).

Iter	Entities	Loss	Preds	Link Conf.	Nbr. Growth
1 $\to$ 2	219	0.366	0	—	—
2 $\to$ 3	420	0.353	20	0.0%	6.2%
3 $\to$ 4	514	0.391	20	0.0%	5.9%
4 $\to$ 5	553	0.359	20	0.0%	0.0%

Table 7. Retroactive DistMult prediction analysis. Link conf. = predicted links confirmed in future KGs. Nbr. growth = fraction of predicted entities gaining new connections at

t{+}1

Figure 3. Michaelis-Menten scalability projection. Left: discovery curve with fitted model. Right: projected coverage ratio. The model projects diminishing returns beyond

{\sim}20

iterations.

Explore similar work

Snippet-Driven Supply Chain Discovery with LLMs: Scaling Visibility in China
May 27, 2026Hiroto Fukada, Takayuki MizunoRelation Extraction KG Construction
Graduate Institute for Advanced Studies (SOKENDAI) · National Institute of Informatics · Tokyo, Japan
Beyond Vector Similarity: A Structural Analysis of Graph-Augmented Retrieval for Industrial Knowledge Graphs
Jun 4, 2026Grama ChethanLLM Reasoning with Graphs LLM Planning
Siemens Digital Industries Software
Data Scout: Targeted Web Crawling for Domain-Specific Pretraining Corpora
Sep 4, 2026Chirag Garg, Eelaaf Zahid, Farhan Ahmed +3Domain-Adaptive Pretraining
IBM Research

May 27, 2026J/K move · Enter open · S save

Snippet-Driven Supply Chain Discovery with LLMs: Scaling Visibility in China

Hiroto Fukada, Takayuki Mizuno

Graduate Institute for Advanced Studies (SOKENDAI) · National Institute of Informatics · Tokyo, Japan

Relation Extraction KG Construction LLM Information Extraction Supply Chain Management Knowledge Graphs

Open paper AskPDF arXiv

Financial and economic research often relies on structured supply-chain disclosures and commercial databases. In China, supplier--customer disclosure is typically limited to major partners of listed firms, leaving unlisted firms and long-tail inter-firm links poorly captured in structured data. Public web evidence can partly complement this gap through corporate, government, and trade-media disclosures; however, full-text web mining at scale is costly because pages are often inaccessible or expensive to process with large language models (LLMs). We propose a snippet-driven method for constructing a supply chain knowledge graph (SCKG), with firms as nodes and inter-firm relationships as edges. Web search snippets are query-biased summaries returned with search results. We use them as a scalable first-pass evidence layer for LLM-based relationship extraction. We evaluate the pipeline in terms of extraction efficiency and coverage. For extraction efficiency, exhaustive full-text chunking discovers 19.8

\times

more unique relationships than snippets, but requires 251.2

\times

more input tokens and yields higher redundancy. For coverage, we use 130,685 Chinese firms as search seeds, covering Shanghai/Shenzhen-listed firms and large unlisted firms as of 2024. In the listed-firm subset, the resulting SCKG covers 7.2

\times

more firms and 9.3

\times

more relationships than the CSMAR disclosure-based benchmark, while revealing heavy-tailed degree patterns. Retained provenance metadata make the SCKG an auditable complement to disclosure-based databases.

Iter	Pages	New Ent.	Total	Companies	Relations
1	24	219	219	50	170
2	44	201	420	71	334
3	18	94	514	77	423
4	13	39	553	85	452
5	45	111	664	115	542

Iter	Pages	New Ent.	Total	Companies	Relations
1	24	219	219	50	170
2	44	201	420	71	334
3	18	94	514	77	423
4	13	39	553	85	452
5	45	111	664	115	542

Abstract

Explore similar work

Snippet-Driven Supply Chain Discovery with LLMs: Scaling Visibility in China

Beyond Vector Similarity: A Structural Analysis of Graph-Augmented Retrieval for Industrial Knowledge Graphs

Data Scout: Targeted Web Crawling for Domain-Specific Pretraining Corpora

Iter	Pages	New Ent.	Total	Companies	Relations
1	24	219	219	50	170
2	44	201	420	71	334
3	18	94	514	77	423
4	13	39	553	85	452
5	45	111	664	115	542