RelICL: Training-free Relational Learning with Tabular Foundation Models

Simon Forbat; Rainer Gemulla

cs.LGOct 1, 2026

RelICL: Training-free Relational Learning with Tabular Foundation Models

Authors: Simon Forbat, Rainer Gemulla

Organizations: Data and Web Science Group University of Mannheim Mannheim, Germany

Abstract

Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables' columns as features. This approach is appealing because it directly benefits from improvements to or customization of the underlying tabular foundation model. In this paper, we identify two key problems with DFS: feature explosion and interaction blindness. The first problem arises because the number of DFS features grows quickly as the schema becomes more complex, limiting scalability and performance. The second problem arises because column-wise aggregates do not account for feature interactions, limiting performance. We propose and explore an alternative method termed RelICL, which keeps the benefits of DFS but alleviates these two problems. At its heart, RelICL propagates and fuses information step by step through the schema graph, using the same tabular foundation model that is eventually used for prediction to do so. In our experimental study using RelBench tasks, RelICL was on par with the strongest approach based on deep feature synthesis.

Figures & tables

Table 1

	$\mathcal{T}(R)$	$\mathcal{T}(\mathcal{T}_{\mathrm{enc}}(R))$
Dataset	AUROC	AUROC	$\Delta$
bioresponse	87.7 $\pm$ 1.0	87.5 $\pm$ 0.8	-0.3 $\pm$ 0.3
churn	92.8 $\pm$ 0.8	92.6 $\pm$ 0.6	-0.2 $\pm$ 0.2
compass	72.3 $\pm$ 0.7	72.4 $\pm$ 0.7	+0.1 $\pm$ 0.1
credit-g	82.2 $\pm$ 1.8	83.1 $\pm$ 0.4	+0.9 $\pm$ 1.4
jasmine	89.8 $\pm$ 1.1	86.8 $\pm$ 0.4	-3.0 $\pm$ 0.6

Table 3: Replacing the feature columns by a row embedding (test AUROC (%), higher is better, mean

\pm

SEM over three seeds).

\mathcal{T}(R)

runs TabICL on the table

R

\mathcal{T}(\mathcal{T}_{\mathrm{enc}}(R))

replaces all feature columns by TabICL’s respective row embedding.

\Delta

is the paired difference against the original data of the same seed. The last row averages the ten datasets, so its SEM is taken across datasets rather than across seeds.

	Pretrained (rel.)		Pretrained (tab.)
Model	RT	PluRel	RDBLearn	TabPFN-Rel	RelICL
RT	–	0 6 / 18 ∗	0 0 / 8	0 2 / 10	0 7 / 18 ∗
PluRel	11 / 18 ∗	–	0 0 / 8	0 2 / 10	0 5 / 18 ∗
RDBLearn	0 8 / 8 0	0 8 / 8 0	–	0 1 / 15	0 5 / 15
TabPFN-Rel	0 8 / 10	0 8 / 10	13 / 15	–	10 / 21
RelICL	11 / 18 ∗	13 / 18 ∗	10 / 15	11 / 21	–

Table 4: Head-to-head wins over all regression and classification tasks combined. The baselines report different subsets of the tasks and metrics, so an average over all tasks would score each model on different tasks. We therefore compare the models pair by pair, only on the tasks both report. A cell counts the tasks the row model won over the column model divided by the total number of shared tasks. A pair shares a task only where both models report a common metric for it. RT and PluRel share no regression task with RDBLearn and TabPFN-Rel. Bold marks the model that won the pair. Tabs. 6 and 8 provide the more details.

Table 5: Regression, MAE (lower is better) / $R^{2}$ (%, higher is better). The table gives the per-task numbers behind the regression part of Tab. 4 . It shows both metrics because RT and PluRel report only $R^{2}$ , while RDBLearn and TabPFN-Rel report only MAE. RelICL reports both, so it can be compared with either group. Bold marks the best value per task and metric.	Table 6: RelICL with the TabPFN backbone.
Pretrained (rel.) Pretrained (tab.) Task RT PluRel RDBLearn TabPFN-Rel RelICL item-ltv – / 32.5 – / 40.5 48.56 / – 47.80 / – 43.97 / 28.6 user-ltv – / 36.9 – / 18.5 14.54 / – 14.40 / – 14.78 / 21.0 ad-ctr – / 4.5 – / 4.9 0.03 / – 0.03 / – 0.04 / 18.3 user-attendance – – – 0.24 / – 0.24 / 5.1 driver-position – / 52.4 – / 55.5 – 3.76 / – 3.68 / 25.2 item-sales – / 14.0 – / 20.0 0.06 / – 0.06 / – 0.04 / 69.2 post-votes – / 33.9 – / 25.5 0.07 / – 0.07 / – 0.06 / 25.6 site-success – / 4.5 – / 38.6 0.42 / – 0.39 / – 0.40 / -22.0 study-adverse – / 2.6 – / 1.6 43.91 / – 42.60 / – 32.42 / 54.0	Pretrained (tab.) RelICL (TabPFN) 43.41 / 29.2 15.48 / 16.5 0 0.05 / 0.1 0 0.30 / 17.2 0 4.00 / 13.4 0 0.04 / 50.1 0 0.08 / 17.0 0 0.43 / -7.1 29.97 / 56.9
– Not reported.

Table 5: Regression, MAE (lower is better) /

R^{2}

(%, higher is better). The table gives the per-task numbers behind the regression part of Tab. 4 . It shows both metrics because RT and PluRel report only

R^{2}

, while RDBLearn and TabPFN-Rel report only MAE. RelICL reports both, so it can be compared with either group. Bold marks the best value per task and metric.

Table 7: Classification (AUROC (%), higher is better). The table gives the per-task numbers behind the classification part of Tab. 4 . Bold marks the best value per task.	Table 8: RelICL with the TabPFN backbone.
Pretrained (rel.) Pretrained (tab.) Task RT PluRel RDBLearn TabPFN-Rel RelICL item-churn 70.9 72.5 82.1 82.8 78.5 user-churn (ama) 64.0 65.0 67.6 70.2 65.4 user-clicks 59.5 47.9 69.0 61.5 64.3 user-visits 61.8 63.4 65.5 66.9 66.2 user-ignore – – – 70.1 88.5 user-repeat – – – 76.9 79.3 driver-dnf 81.2 81.0 – 71.5 73.6 driver-top3 89.3 88.4 – 79.3 81.4 user-churn (hm) 62.8 66.0 68.0 70.6 68.4 user-badge 80.1 82.0 85.3 86.4 86.3 user-engagement 75.7 86.2 89.4 90.6 90.0 study-outcome 51.8 51.8 71.6 73.1 73.4	Pretrained (tab.) RelICL (TabPFN) 78.4 64.6 58.7 65.7 87.9 77.7 74.2 78.4 67.8 84.9 89.5 73.6
– Not reported.

Table 7: Classification (AUROC (%), higher is better). The table gives the per-task numbers behind the classification part of Tab. 4 . Bold marks the best value per task.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 2 — Figure 2: The bounded depth task for three task rows. Each line links a key column to the matching column of the next table, and every reference is one-to-one. The label of a task row equals the bit in $R_{3}$ , three references away. The highlighted query row is the one the model has to predict.

Panel of Figure 3 — Figure 3: Showcases a pathological task which cannot be solved through DFS features. Entity 1 is positive, so $b$ increases with $a$ ; entity 2 is negative, so $b$ decreases. Column $a$ holds $0,1,2$ and column $b$ holds $-1,0,1$ in both classes, so the classes are separated only by which value of $b$ is next to which value of $a$ . Entity 3 is the highlighted query row, whose label the model has to predict from its three signal rows.

Panel of Figure 4 — Figure 4: Junction table elimination. The junction table $R$ is on the many side of its references to $C_{1}$ and $C_{2}$ . RelICL replaces it by two copies, each of which keeps one reference and carries the columns of the other neighbor inline.

Dataset	OpenML ID	Rows	Features	Positive class (%)
bioresponse	4134	3751	1776	54.2
churn	40701	5000	20	14.1
compass	42193	5278	13	47.0
credit-g	31	1000	20	70.0
jasmine	41143	2984	144	50.0
kr-vs-kp	3	3196	36	52.2

Appendix

Table 9: The ten OpenML datasets of Sec. C.2 .

	$\mathcal{T}(R)$	$\mathcal{T}(\mathcal{T}_{\mathrm{enc}}(R))$		$\mathcal{T}(\operatorname{reduce}(\mathcal{T}_{\mathrm{enc}}(R)))$
Dataset	AUROC	AUROC	$\Delta$	AUROC	$\Delta$
bioresponse	87.7 $\pm$ 1.0	87.5 $\pm$ 0.8	-0.3 $\pm$ 0.3	86.1 $\pm$ 1.0	-1.6 $\pm$ 0.4
churn	92.8 $\pm$ 0.8	92.6 $\pm$ 0.6	-0.2 $\pm$ 0.2	91.4 $\pm$ 0.6	-1.4 $\pm$ 0.2
compass	72.3 $\pm$ 0.7	72.4 $\pm$ 0.7	+0.1 $\pm$ 0.1	72.3 $\pm$ 0.7	0.0 $\pm$ 0.1
credit-g	82.2 $\pm$ 1.8	83.1 $\pm$ 0.4	+0.9 $\pm$ 1.4	79.4 $\pm$ 0.4	-2.8 $\pm$ 1.5
jasmine	89.8 $\pm$ 1.1	86.8 $\pm$ 0.4	-3.0 $\pm$ 0.6	86.5 $\pm$ 0.8	-3.3 $\pm$ 0.4

Appendix

Table 10: Replacing the feature columns by a row embedding (test AUROC (%), higher is better, mean

\pm

SEM over three seeds).

\mathcal{T}(R)

runs TabICL on the table

R

\mathcal{T}(\mathcal{T}_{\mathrm{enc}}(R))

replaces all feature columns by the row embedding (

d=512

\mathcal{T}(\operatorname{reduce}(\mathcal{T}_{\mathrm{enc}}(R)))

then applies a Gaussian random projection (

d^{\prime}=32

, see Sec. D.3 ). Both

\Delta

are paired differences against the original data of the same seed. The last row averages the ten datasets, so its SEM is taken across datasets rather than across seeds.

Dataset	Tables	Rows	Columns
rel-amazon	3	23,218,245	15
rel-avito	8	24,653,915	42
rel-event	5	44,822,992	131
rel-f1	9	97,606	67
rel-hm	3	16,931,173	37
rel-stack	7	5,399,818	51

Appendix

Table 11: RelBench databases

Dataset	Task	Train	Val	Test	Pos. rate (%)
rel-amazon	item-churn	2,536,014	177,689	166,842	36.9
rel-amazon	user-churn	4,708,383	409,792	351,885	60.6
rel-avito	user-clicks	59,454	21,183	47,996	1.5
rel-avito	user-visits	86,619	29,979	36,129	85.1
rel-event	user-ignore	19,239	2,013	1,958	13.0
rel-event	user-repeat	3,842	268	246	44.7

Appendix

Table 12: RelBench’s (binary) classification tasks

Dataset	Task	Train	Val	Test	Target (mean $\pm$ SD)
rel-amazon	item-ltv	2,707,679	166,978	178,334	77.13 $\pm$	664.61
rel-amazon	user-ltv	4,708,383	409,792	351,885	16.78 $\pm$	55.44
rel-avito	ad-ctr	5,100	1,766	1,816	0.05 $\pm$	0.11
rel-event	user-attendance	19,239	2,013	1,958	0.26 $\pm$	0.64
rel-f1	driver-position	7,453	499	760	11.93 $\pm$	5.21
rel-hm	item-sales	5,488,184	105,542	105,542	0.08 $\pm$	0.58

Appendix

Table 13: RelBench’s regression tasks

Regression	Classification
rel-f1/driver-position	rel-f1/driver-top3
rel-event/user-attendance	rel-event/user-repeat
rel-trial/study-adverse	rel-event/user-ignore
rel-trial/site-success	rel-trial/study-outcome
rel-avito/ad-ctr	rel-avito/user-clicks
rel-hm/item-sales	rel-hm/user-churn

Appendix

Table 14: Tasks the hyperparameter search ran on.

Hyperparameter	Search space	Selected (default values)
Anchor budget fraction a	$\{0.25,0.75\}$	$0.75$
Train sample size	$\{16384,32768,65536\}$	$32768$
Context table sample size	$\{32768,65536\}$	$65536$
Count features after optimization b	$\{\text{true},\text{false}\}$	true
Count feature windows (days) b	$\{\text{none},[1,7,28],[1,3,7,28]\}$	none
Relative time encoding	$\{\text{true},\text{false}\}$	true

Appendix

Table 15: Hyperparameters, their search space, and the value selected for RelICL’s reported configuration.

Kind	Model	item-ltv		user-ltv		ad-ctr		user-attendance		driver-position		item-sales		post-votes		site-success		study-adverse
Trained	GraphSAGE	50.05 /	–	14.31 /	–	0.04 /	–	0.26 /	–	4.02 /	–	0.06 /	–	0.07 /	–	0.40 /	–	44.47 /	–
Trained	RelGNN	48.77 /	–	14.23 /	–	0.04 /	–	0.24 /	–	3.80 /	–	0.05 /	–	0.07 /	–	0.30 /	–	44.46 /	–
Trained	RelGT	48.92 /	–	14.27 /	–	0.04 /	–	0.25 /	–	3.92 /	–	0.05 /	–	0.07 /	–	0.33 /	–	43.99 /	–
Trained	DS+LightGBM	41.12 /	–	13.93 /	–	0.04 /	–	0.28 /	–	3.96 /	–	0.04 /	–	0.07 /	–	0.41 /	–	40.58 /	–
Pretrained (rel.)	RT	– /	32.5	– /	36.9	– /	4.5	–		– /	52.4	– /	14.0	– /	33.9	– /	4.5	– /	2.6
Pretrained (rel.)	RT (target DB)	– /	32.2	– /	38.3	– /	8.0	–		– /	58.7	– /	30.9	– /	35.0	– /	5.1	– /	3.1

Appendix

Table 16: All published models, regression, MAE (lower is better) /

R^{2}

(%, higher is better)

Kind	Model	item-churn	user-churn (ama)	user-clicks	user-visits	user-ignore	user-repeat	driver-dnf	driver-top3	user-churn (hm)	user-badge	user-engagement	study-outcome
Trained	GraphSAGE	82.8	70.4	65.9	66.2	81.6	76.9	72.6	75.5	69.9	88.9	90.6	68.6
Trained	RelGNN	82.6	71.0	68.2	66.2	86.2	79.6	75.3	85.7	70.9	89.0	90.8	71.2
Trained	RelGT	82.5	70.4	68.3	66.8	81.6	76.1	75.9	83.5	69.3	86.3	90.5	68.6
Trained	DS+LightGBM	81.8	67.6	64.3	64.5	84.2	75.4	69.8	82.4	69.0	86.2	90.3	72.0
Pretrained (rel.)	RT	70.9	64.0	59.5	61.8	–	–	81.2	89.3	62.8	80.1	75.7	51.8
Pretrained (rel.)	PluRel	72.5	65.0	47.9	63.4	–	–	81.0	88.4	66.0	82.0	86.2	51.8

Appendix

Table 17: All published models, classification (AUROC (%), higher is better)

		Pretrained (rel.)		Pretrained (tab.)
Kind	Model	RT	PluRel	RDBLearn	TabPFN-Rel	RelICL
Pretrained (rel.)	RT	–	3 / 8 ∗	–	–	5 / 8 ∗
Pretrained (rel.)	PluRel	5 / 8 ∗	–	–	–	3 / 8 ∗
Pretrained (tab.)	RDBLearn	–	–	–	0 / 7	2 / 7
Pretrained (tab.)	TabPFN-Rel	–	–	6 / 7	–	4 / 9
Pretrained (tab.)	RelICL	3 / 8 ∗	5 / 8 ∗	5 / 7	5 / 9	–

Appendix

Table 18: Head-to-head wins, regression

		Pretrained (rel.)		Pretrained (tab.)
Kind	Model	RT	PluRel	RDBLearn	TabPFN-Rel	RelICL
Pretrained (rel.)	RT	–	0 3 / 10	0 0 / 8	0 2 / 10	0 2 / 10
Pretrained (rel.)	PluRel	0 6 / 10	–	0 0 / 8	0 2 / 10	0 2 / 10
Pretrained (tab.)	RDBLearn	0 8 / 8 0	0 8 / 8 0	–	0 1 / 8	0 3 / 8
Pretrained (tab.)	TabPFN-Rel	0 8 / 10	0 8 / 10	0 7 / 8 0	–	0 6 / 12
Pretrained (tab.)	RelICL	0 8 / 10	0 8 / 10	0 5 / 8 0	0 6 / 12	–

Appendix

Table 19: Head-to-head wins, classification

Trained

Pretrained (rel.)

Pretrained (tab.)

Kind

Model

GraphSAGE

RelGNN

RelGT

DS+LightGBM

RT (target DB)

PluRel

KumoRFM-2

RDBLearn

TabPFN-Rel (API)

TabPFN-Rel

RelICL

Trained

GraphSAGE

–

0 0 / 9

0 3 / 9

–

0 1 / 9

0 4 / 7 0

0 4 / 9

0 3 / 9

0 1 / 9

Trained

RelGNN

0 8 / 9 0

–

0 5 / 9 0

0 4 / 9 0

–

0 1 / 9

0 4 / 7 0

0 5 / 9 0

0 4 / 9

Trained

RelGT

0 8 / 9 0

0 2 / 9

–

0 4 / 9 0

–

0 1 / 9

0 4 / 7 0

0 4 / 9

0 3 / 9

Trained

DS+LightGBM

0 5 / 9 0

0 4 / 9 0

–

0 3 / 9

0 6 / 7 0

0 5 / 9 0

0 3 / 9

Pretrained (rel.)

–

0 1 / 8 0 ∗

0 3 / 8 0 ∗

–

0 5 / 8 0 ∗

Appendix

Table 20: Head-to-head wins against all published models, regression

Trained

Pretrained (rel.)

Pretrained (tab.)

Kind

Model

GraphSAGE

RelGNN

RelGT

DS+LightGBM

PluRel

KumoRFM-2

RDBLearn

TabPFN-Rel (API)

TabPFN-Rel

RelICL

Trained

GraphSAGE

–

0 2 / 12

0 7 / 12

0 9 / 12

0 8 / 10

0 5 / 12

0 6 / 8 0

0 3 / 12

0 7 / 12

0 6 / 12

Trained

RelGNN

10 / 12

–

0 9 / 12

11 / 12

0 8 / 10

0 6 / 12

0 6 / 8 0

0 8 / 12

0 9 / 12

Trained

RelGT

0 5 / 12

0 3 / 12

–

10 / 12

0 8 / 10

0 4 / 12

0 6 / 8 0

0 4 / 12

0 5 / 12

0 8 / 12

Trained

DS+LightGBM

0 3 / 12

0 1 / 12

0 2 / 12

–

0 8 / 10

0 1 / 12

0 5 / 8 0

0 1 / 12

0 3 / 12

0 5 / 12

Pretrained (rel.)

0 2 / 10

–

0 3 / 10

0 0 / 10

0 0 / 8

0 2 / 10

Appendix

Table 21: Head-to-head wins against all published models, classification

Explore similar work

A Fair Benchmarking of Deep Relational Database Learning Models
Jul 4, 2026Kazi F. Akhter, Bharath Ajendla, Manar D. SamadRelational Deep Learning Relational Databases
Department of Computer Science Tennessee State University Nashville, TN, USA · SAP AI Research and Innovation SAP Labs Palo Alto, CA, USA
OpenRFM: Dissecting Relational In-Context Learning
Jun 3, 2026Zhikai Chen, Junyu Yin, Jialiang Gu +5Relational Deep Learning In-Context Learning
Michigan State University · George Mason University · Georgia Institute of Technology +1
Advancing Open and Reproducible Relational Learning: RelArena- $α$ , TabPFN-Rel and RPI
Aug 17, 2026Adrian Hayler, Klemens Flöge, Alan Arazi +44Relational Deep Learning Tabpfn

Jul 4, 2026, cs.DBJ/K move · Enter open · S save

A Fair Benchmarking of Deep Relational Database Learning Models

Kazi F. Akhter, Bharath Ajendla, Manar D. Samad

Department of Computer Science Tennessee State University Nashville, TN, USA · SAP AI Research and Innovation SAP Labs Palo Alto, CA, USA

Relational Deep Learning Relational Databases Benchmark Datasets Multi--Task Learning

Open paper AskPDF arXiv

Relational databases (RDBs) are the primary data infrastructure in many enterprises, yet recent deep learning methods designed for RDBs have been evaluated under inconsistent experimental protocols, making fair comparison difficult. We present one of the first systematic benchmarking studies of recently released deep learning methods for RDBs, evaluating them across five relational databases, with one classification and one regression task for each. We refactor all deep RDB models to allow the full range of experimental procedures to be applied consistently across all methods. Our findings indicate that the relational transformer (RT) approach delivers the strongest overall performance on both classification and regression tasks compared to the state-of-the-art graph-based modeling and learning of RDBs. Even for single-table learning tasks, deep learning methods designed for RDBs outperform the leading tabular foundation model, TabPFN 2.5. Extending learning from a single table (hop = 0) to multiple tables (hop = 1, 2) by connecting neighboring tables in relational databases enhances performance, but the additional benefit from higher hops diminishes as computational overhead grows. Deep RDB learning methods have the potential to challenge state-of-the-art tabular foundation models, especially on large-scale enterprise data. The source code for this benchmarking study is publicly available.

Abstract

Explore similar work

A Fair Benchmarking of Deep Relational Database Learning Models

OpenRFM: Dissecting Relational In-Context Learning

Advancing Open and Reproducible Relational Learning: RelArena-ααα, TabPFN-Rel and RPI

Advancing Open and Reproducible Relational Learning: RelArena- $α$ , TabPFN-Rel and RPI