Automatic End-to-End Data Integration using Large Language Models

Aaron Steiner; Christian Bizer

cs.CLMar 11, 2026

Automatic End-to-End Data Integration using Large Language Models

Authors: Aaron Steiner, Christian Bizer

Organizations: Data and Web Science Group, University of Mannheim, Mannheim, Germany

Abstract

Designing data integration pipelines typically requires substantial manual effort from data engineers to configure pipeline components and label training data. While LLMs have shown promise in handling individual steps of the integration process, their potential to replace all human input across end-to-end data integration pipelines has not been investigated. As a step toward exploring this potential, we present an automatic data integration pipeline that uses GPT-5.2 to generate all artifacts required to adapt the pipeline to specific use cases. These artifacts are schema mappings, value mappings for data normalization, training data for entity matching, and validation data for selecting conflict resolution heuristics in data fusion. We compare the performance of this LLM-based pipeline to the performance of human-designed pipelines along three case studies requiring the integration of video game, music, and company related data. Our experiments show that the LLM-based pipeline is able to produce similar results, for some tasks even better results, as the human-designed pipelines. End-to-end, the human and the LLM pipelines produce integrated datasets of comparable size and density. Having the LLM configure the pipelines costs approximately $10 per case study, which represents only a small fraction of the cost of having human data engineers perform the same tasks.

Figures & tables

Case Study	Source	Rows	Attrs	Density	Attributes
Games	DBpedia	46,580	6	91.0%	wiki_ref, title, launch_yr, studio, system, franchise
	Metacritic	20,494	8	97.7%	mc_id, game_title, year_published, made_by, console, press_rating, player_rating, age_rating
	Sales	7,877	10	98.7%	rec_id, prod_title, launch_dt, studio, dist, hw, press_score, comm_rating, age_classification, units_sold_mm
	Target Schema	–	12	–	id, name, releaseYear, developer, genres, publisher, platform, criticScore, userScore, ESRB, globalSales, series
Companies	Forbes	2,000	7	99.2%	forbes_url, company, url, region, business_segment, asset_value, sales_figure
	DBpedia	10,085	9	66.4%	entity_uri, org_name, established, nation, headquarters, sector, keypeople_name, total_assets_val, annual_income

TABLE I: Dataset statistics for the three case studies.

Case Study	Human	LLM	Label-based	Instance-based
Games	1.00	1.00	.23	.00
Companies	1.00	1.00	.35	.32
Music	1.00	1.00	.38	.24
Average	1.00	1.00	.32	.19

TABLE II: Schema matching F1 scores (macro-averaged across datasets).

		LLM Mappings			Code-based
	Source	Col.	Norm.	Total	Col.	Norm.	Total
Games	DBpedia	1	38,268	46,170	2	83,439	91,341
	Metacritic	2	38,712	38,712	3	59,204	59,206
	Sales	1	7,877	7,877	3	23,631	23,631
Companies	DBpedia	1	6,395	6,878	2	16,429	16,963
	Forbes	1	1,895	1,957	1	1,895	1,957

TABLE III: Value normalization statistics by method and case study.

		Human	Human	LLM
Use Case	Dataset Pair	Config	Labels	Labels
Games	DBpedia–Metacritic	.930	.826	.849
Games	DBpedia–Sales	.927	.839	.979
Companies	DBpedia–Forbes	.857	.954	.939
Companies	Forbes–FullContact	.870	.898	.897
Music	Discogs–MusicBrainz	.800	.991	.990

TABLE IV: Entity matching F1 scores on held-out test sets. Human Config : the data engineers’ pipeline. Human Labels and LLM Labels : identical ML matchers trained on human-labeled vs. LLM-labeled training data.

	Human	Human	LLM	LLM
Use Case	Config	Val. Set	Val. Set	+ RAG
Games	.832	.866	.866	.866
Companies	.803	.861	.709	.744
Music	.766	.721	.708	.708
Average	.800	.816	.761	.773

TABLE V: Data fusion test accuracy by validation set origin.

Metric	Games	Companies	Music
Data Sources	3	3	3
Total Input Records	74,951	14,016	37,255
Fused Record Groups	7,235	1,031	4,178
Output Records	65,518	12,768	30,885
Fusion Ratio	11.0%	8.1%	13.5%
Row Gain vs. Largest	+18,938	+2,683	+8,258

TABLE VI: End-to-end integration metrics for the three case studies.

	Output Records		Fusion Ratio		Row Gain %		Density $\Delta$
Use Case	Human	LLM	Human	LLM	Human	LLM	Human	LLM
Games	65,794	65,518	11.8	11.0	+41.3	+40.7	+2.7	+4.5
Companies	11,517	12,768	15.1	8.1	+14.2	+26.6	$-$ 5.2	+5.6
Music	30,779	30,885	14.6	13.5	+36.0	+36.5	+5.8	$-$ 1.8

TABLE VII: End-to-end integration metrics comparing human versus LLM pipelines.

	Execution		Configuration
Use Case	LLM	Human	LLM	Human
Games	181 s	73 s	1.9 h	$\geq$ 18 h
Companies	38 s	34 s	1.7 h	$\geq$ 20 h
Music	45 s	124 s	2.2 h	$\geq$ 20 h
Average	88 s	77 s	1.9 h	$\geq$ 19 h

TABLE VIII: Average runtime per use case. Execution includes normalization, matching, and fusion. Configuration refers to one-time artifact generation. Human configuration times are lower-bound estimates.

Pipeline Step	Games	Companies	Music	Total
Schema Matching	$0.02	$0.02	$0.02	$0.06
Normalization	$0.13	$0.36	$0.17	$0.66
Training Set Generation	$1.33	$1.96	$1.66	$4.95
Fusion Validation LLM	$0.22	$0.15	$0.23	$0.60
Fusion Validation RAG	$7.46	$6.07	$7.45	$20.98
Total per Use Case	$9.16	$8.56	$9.53	$27.25

TABLE IX: LLM usage costs per use case.

Data Integration

See all 15 papers

MaDI-Bench: An End-to-End Data Integration Benchmark
Jun 29, 2026Aaron Steiner, Ralph Peeters, Christian BizerBenchmark Design Data Integration
Data and Web Science Group University of Mannheim Mannheim, Germany
SINT-Flow: Schema Integration using Large Language Model Workflows
Jul 27, 2026Keti Korini, Christian BizerSemantic Mapping Relational Databases
University of Mannheim Mannheim, Germany
DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data Pipelines
Jul 18, 2026Runming He, Zhen Hao Wong, Hao Liang +4LLM Grounding Code Generation
Peking University · Institute for Advanced Algorithms Research, Shanghai · Zhongguancun Academy

Jun 29, 2026J/K move · Enter open · S save

MaDI-Bench: An End-to-End Data Integration Benchmark

Aaron Steiner, Ralph Peeters, Christian Bizer

Data and Web Science Group University of Mannheim Mannheim, Germany

Benchmark Design Data Integration Data Fusion

Open paper AskPDF arXiv

Data integration is the process of combining data from multiple, heterogeneous sources into a consistent, unified representation. Data integration involves a sequence of interdependent tasks including schema matching, value normalization, blocking, entity matching, and data fusion. Existing table-based benchmarks either evaluate these steps in isolation or cover only incomplete versions of the data integration pipeline, omitting specific steps. The lack of public end-to-end data integration benchmarks hinders research on data integration methods that address the integration process as a whole and account for the interdependencies among the different tasks. This paper fills this gap by introducing the Mannheim Data Integration Benchmark (MaDI-Bench), the first benchmark for the end-to-end integration of relational tables covering all steps of the integration process. MaDI-Bench contributes (i) a set of end-to-end data integration tasks spanning several application domains, each requiring the full schema matching, value normalization, entity matching, and data fusion pipeline, and (ii) a generic method for deriving task variants that mitigates rapid benchmark saturation as data integration systems advance. We validate the benchmark using human-engineered pipelines, a best-of-breed pipeline, an LLM workflow, and a pipeline written by a coding agent. The validation demonstrates the utility of the benchmark for measuring the step-wise as well as the end-to-end performance of data integration pipelines. All benchmark artifacts are available for public download.

Abstract

Data Integration

Explore similar work

MaDI-Bench: An End-to-End Data Integration Benchmark

SINT-Flow: Schema Integration using Large Language Model Workflows

DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data Pipelines