Given a query table, existing data lake discovery techniques seek to identify data sources related to the query table through schema similarity and joinability. The discovered sources, however, are not always useful for downstream analyses; while some improve task generalization, some have the potential to inadvertently introduce harmful noise or distribution shifts. We formalize source selection as the problem of finding a subset of the discovered sources that maximizes downstream task utility. We propose Splice, a target- and cost-aware source selection system that adopts the idea of gene splicing to identify sources which provide additional signals that the union of all sources misses. To speed up the process, we develop two surrogate variants, Splice-DP and Splice-Grad, that estimate the marginal contribution of subsets to task utility by leveraging data profiles in a meta-learning approach and gradient alignment, respectively. On classification tasks over two real-world benchmarks derived from SOTA table union search tools, Splice shows 3-5 percentage points improvement in binary F1 scores compared to the case when all discovered tables are considered. On five additional real-world ML tasks spanning classification and regression, Splice improves task utility over the all-sources baseline by 1.8-13.7 points, while Splice-DP and Splice-Grad match this quality with 3-7x fewer model evaluations (up to 77x at scale).
Figures & tables
Figure 1. An overview of Splice : a gene-splicing-based source selection mechanism that actively adds and removes sources from an active set to output the subset of sources having the best downstream performance.
Task
dϕ
Feature groups
Surrogate
Tabular classification, regression
28
Size & feature shape: dataset size, PCA-95% dim, first-PC skew, first-PC kurt, per-feature skew & kurt (min/max/mean/std); Target alignment: per-feature target correlations (min/max/mean/std); Target structure: class imbalance, base rate, Bayes-error gap (classification), mean, standard deviation (regression)
Linear Regression
Image classification
11
Target alignment: JS divergence, ℓ2 domain gap, centroid dist. (all, tail-class), variance ratio vs target; Class balance & coverage: label & tail-class coverage, log min/mean class counts, head/tail ratio, within-class compactness
Gradient Boost
Text classification
15
Size & balance: log size, effective n , positive class fraction; Target alignment: JS divergence vs target, ℓ2 centroid distances ( μglobal,μ+,μ− ), cosine similarities to target ( μglobal,μ+,μ− ), variance ratio vs target; Class separation: Fisher separation, ∣ sep − target sep ∣ , Fisher ratio, within-class compactness
Gradient Boost
Table 1. Data profile summary. dϕ = profile dimension. Features are semicolon-separated groups, with italic labels indicating what each group captures. Surrogate model column refers to regressor used to predict marginal gains from meta-features.
Oracle
Offline cost
Per-query
Exact ( Splice )
None
O(Ttrain)
Data-profile ( Splice-DP )
O(n⋅Ttrain)
O(dϕ)
Gradient-match ( Splice-Grad )
O((n+1)⋅Tref)
O(dg)
Table 2. Cost comparison of marginal gain oracles. Tref : cost of fitting a reference model (one linear/small head) ( <<Ttrain ).
Method
ACSIncome
ACSPubCov.
ACSTravelTime
Amazon
Wilds
Single
72.81 ± 0.22
62.55 ± 0.04
500.67 ± 1.33
0.8337 ± 0.0071
0.0903 ± 0.0077
AllSrc
71.39 ± 0.21
51.12 ± 0.03
500.28 ± 1.31
0.8314 ± 0.0118
0.2114 ± 0.0096
Random
70.17 ± 0.18
51.68 ± 0.02
609.28 ± 1.49
0.8163 ± 0.0108
0.1408 ± 0.0119
Greedy
72.98 ± 0.28
62.55 ± 0.04
496.42 ± 1.28
0.8470 ± 0.0066
0.1463 ± 0.0056
DsDM
72.81 ± 0.19
64.61 ± 0.04
496.72 ± 1.27
0.8550 ± 0.0069
0.2132 ± 0.0072
Grasp
73.13 ± 0.21
64.76 ± 0.04
484.45 ± 1.29
0.8614 ± 0.0054
0.2303
Table 3. Profit by task and dataset, mean ± 95% CI over 30 target splits (ACS) or 10 model seeds (Wilds, Amazon). ACSTravelTime reports MSE (lower is better); all other columns are higher-is-better. Best per column in bold ; Second best are underlined .
Dataset
Learner
AllSrc
Single
Greedy
DsDM
Grasp
Splice
ACSInc.
LogReg
71.39
72.81
72.98
72.81
73.13
73.14
XGBoost (transfer)
78.71
82.21
81.18
77.54
80.02
80.16
LightGBM (transfer)
78.63
82.39
81.17
77.94
79.93
80.03
XGBoost (nat)
79.33
82.73
83.59
82.73
—
81.41
LightGBM (nat)
79.12
83.45
82.69
83.11
—
82.25
ACSPub.
LogReg
51.12
62.55
62.55
64.61
64.76
64.81
Table 4. Generalizability to downstream learner. Bold=best.
Figure 2. Runtime of methods (in seconds) on different tasks.
Figure 3. Number of models explored on different tasks.
Figure 4. Effect of number of models explored on profit.
Dataset
Method
Profit
#Evals
Sequential (s)
Parallel (s)
ACSIncome
Splice
74.610
8,353
4,979s
1,155s (4.3 × )
Splice-DP
74.778
367
522s
92s (5.7 × )
Splice-Grad
74.318
326
243s
51s (4.8 × )
ACSPubCov
Splice
66.825
11,030
6,274s
1,216s (5.2 × )
Splice-DP
65.845
401
552s
158s (3.5 × )
Splice-Grad
65.677
323
375s
71s (5.2 × )
Table 5. Results on the entire ACS data ( 50 sources) #Evals = full model trainings; parallel speedup in parentheses.
Figure 5. Splice runtime speedup due to parallelization.
Figure 6. Effect of number of sources (SynPubCov).
Figure 7. Sensitivity of Splice to budget B and maximum number of swaps kmax .
Dataset
SANTOS + Single
SANTOS + All
SANTOS + Splice
YDNPA (Spending)
0.737
0.722
0.768
IPO (Payments)
0.736
0.715
0.745
Table 6. Splice on table groups discovered by Santos . ‘Self-train UB’: F1 attained by 5-fold CV on target table.
f
ACSIncome
ACSPubCov.
ACSTrvlTime
Amazon
Wilds
0.05
73.754
64.608
487.095
0.8466
0.1822
0.10
73.754
64.345
485.206
0.8494
0.2244
0.20
74.835
64.608
485.206
0.8484
0.2485
0.30
75.112
64.581
485.243
0.8519
0.2359
0.70
74.835
64.608
486.903
0.8518
0.2337
Table 7. Effect of validation set size on Splice test profit. f= fraction of target used for selection; ACSTravelTime MSE ( ↓ ).
ACSInc.
ACSPub.
ACSTrvl ↓
Amazon
Wilds
Cor-all (rand)
71.281
51.168
501.617
0.8247
0.1742
Cor-all ( k -C)
74.107
51.980
489.788
0.8340
0.2125
Cor-all (GM)
103.480 †
56.271
1566.02
0.7755
0.1493
Hyb (rand- k )
69.091
53.015
619.643
0.8309
0.1649
Hyb (rand- Splice )
73.368
64.691
488.493
0.8444
0.1602
Hyb (best- Splice )
74.890
64.691
484.122
0.8579
0.2034
Table 8. Point-level and hybrid selection, 10 seeds. † : degenerate classifier. Bold: better than Splice .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Median γ
% Sub
% Part-comp
% Opp-sign
% Filtered
ACSIncome
1.97
77
11
11
26
ACSPubCov
2.37
76
11
11
47
ACSTravelTime
1.07
53
14
33
41
Amazon
1.71
78
14
8
24
IPO
2.43
69
10
21
39
Wilds
1.44
69
14
17
12
Appendix
Table 9. Submodularity ratio diagnostics across benchmarks. Substitutes ( γ>1 , sum of individual gains exceeds joint gain), Partial-complements ( 0<γ≤1 , joint gain exceeds or matches the sum), Opposite-sign ( γ<0 , individual gains and joint gain have opposite signs, only possible in non-monotone settings). % Filtered reports the fraction of all 500 sampled pairs filtered before γ computation because the joint gain was non-positive.
Benchmark
vs.
Δ (95% CI)
p
ACSIncome
AllSrc
+1.749[1.63,1.87]
<10−4
DsDM
+0.333[0.15,0.51]
0.007
Grasp
+0.008[−0.07,0.09]
0.836
Splice-DP
−0.063[−0.16,0.04]
0.473
ACSPubCov
AllSrc
+13.690[13.65,13.73]
<10−4
Grasp
+0.044[0.02,0.07]
0.005
Appendix
Table 10. Paired difference in profit, Splice minus the competing method, on identical target splits/seeds. Positive favours Splice . Δ is the mean paired difference and the 95% CI is Δ± 1.96 SE over replicates. p from a two-sided paired t -test, Holm-corrected over all comparisons against Splice within each benchmark.
State
Sel.
Rows
Mismatch
Shift
Solo
Marginal
CA
729,432
0.211
4.21
62.23
−5.89
ME
✓
23,699
0.254
3.26
61.55
+0.09
ID
✓
30,490
0.362
3.65
59.79
+0.53
MI
✓
186,896
0.230
3.01
58.05
+1.81
MD
93,984
0.279
3.10
57.86
−1.04
GA
194,141
0.358
3.74
57.75
−5.39
Appendix
Table 11. ACSPubCov source profiles, ordered by solo profit.
Src
Imgs
Cls
Cov
Shift
Solo
Marginal
Selected by Splice
28
3,719
17
0.452
10.06
0.0669
+0.029
2
4,100
24
0.548
8.14
0.0574
+0.062
10
3,690
22
0.516
8.47
0.0549
+0.070
9
2,620
23
0.516
7.92
0.0477
+0.090
20
2,420
15
0.419
9.31
0.0279
+0.038
Appendix
Table 12. Wilds source profiles. Cls: classes present in the source; Cov: fraction of target classes covered.
Category
Sel.
Pos.
Mismatch
Solo
Marginal
Industrial & Sci.
0.842
0.027
0.829
−0.008
Health & Household
0.812
0.003
0.817
−0.028
Kindle Store
✓
0.798
0.017
0.807
+0.036
Grocery & Gourmet
✓
0.830
0.015
0.805
+0.012
Software
0.688
0.127
0.804
−0.030
Home & Kitchen
0.828
0.013
0.791
−0.016
Appendix
Table 13. Amazon source profiles, ordered by solo profit. Pos. : positive-class rate.