Relational In-Context Learning via Synthetic Pre-training with Structural Prior

Yanbo Wang; Jiaxuan You; Chuan Shi; Muhan Zhang

cs.LGMar 4, 2026

Relational In-Context Learning via Synthetic Pre-training with Structural Prior

Authors: Yanbo Wang, Jiaxuan You, Chuan Shi, Muhan Zhang

Organizations: Institute for Artificial Intelligence, Peking University · University of Illinois at Urbana-Champaign · Institute of Computing Technology, Beijing University of Post and Telecommunication · State Key Laboratory of General Artificial Intelligence

Abstract

Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce, and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, we introduce RDB-PFN, the first relational foundation model trained purely via synthetic data. Inspired by Prior-Data Fitted Networks (PFNs), where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a Relational Prior Generator to create an infinite stream of diverse RDBs from scratch. Pre-training on over 2 million synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine in-context learning. Experiments show that RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming state-of-the-art tabular foundation models evaluated on the same DFS-linearized inputs, while using a lightweight architecture and fast inference. The code is available at https://github.com/MuLabPKU/RDBPFN.

Figures & tables

Figure 1: Overview of the RDB-PFN Framework. The top panel illustrates our Universal Relational Prior , which synthesizes diverse relational databases via a hierarchical decomposition: Schema (LayerDAG), Structure (Hybrid SCM), and Content (Hierarchical SCM). The bottom panel depicts the Two-Stage Curriculum Learning protocol, where the model first establishes a statistical backbone on single-table data before adapting to the complex topological signals of linearized relational data.

Figure 2: Resource Efficiency Frontier. Comparison of model complexity across Inference Latency (X-axis), Parameter Count (Y-axis), and Pre-training Data Volume (Bubble Size). “Lite” baselines denote single-estimator configurations with ensembling disabled. RDB-PFN (red point) uses a compact 0.7M-parameter backbone and is pre-trained on only

\sim

2M synthetic datasets. Compared with full/default tabular foundation-model baselines, it is 3.0–6.7

\times

faster at inference; among baselines with disclosed pre-training budgets, it uses 2.5–4.4% as many pre-training datasets.

Panel of Figure 3 — Figure 3: Relational Few-Shot Performance across Evaluation Protocols. We report aggregated normalized performance across 19 relational tasks (higher is better). (a) Single-Estimator Protocol: all baselines are constrained to one estimator (ensembling disabled). RDB-PFN clearly surpasses all baselines. (b) Default Protocol: baselines run with their official default inference pipelines (which may include test-time ensembling), while RDB-PFN remains a single forward-pass estimator. RDB-PFN maintains superior average performance while offering 3.0x – 6.7x faster inference, positioning it on the optimal frontier of the efficiency-accuracy trade-off.

Figure 4: Single-Table Performance Analysis. We compare classical baselines, specialized tabular foundation models, and two RDB-PFN variants. The single-table benchmark serves as a transfer check rather than the primary target setting. RDB-PFN trails specialized single-table TFMs, while the continued relational synthetic pre-training improves over its single-table-only variant, suggesting a positive transfer.

Panel of Figure 5 — Figure 5: Structural Correlation Patterns. Correlation heatmaps for real/synthetic single-table data and real/synthetic DFS-linearized RDBs. Linearized RDBs exhibit more block-like feature correlations induced by relational aggregations, and our synthetic RDB prior reproduces a similar qualitative pattern.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Tables	Columns	Rows
Amazon	3	15	24,291,489
AVS	3	24	349,967,371
Diginetica	5	28	3,672,396
Outbrain	8	31	4,778,954
Retailrocket	3	11	23,033,676
Stackexchange	7	49	5,399,818

Appendix

Table 1: Statistics of relational database datasets.

Dataset	Task Description	#Train / #Val / #Test
Amazon	User Churn Prediction	1,045,568 / 149,205 / 152,486
AVS	Customer Retention Prediction	109,341 / 24,261 / 26,455
Diginetica	Click-through-rate Prediction	108,570 / 6,262 / 5,058
Outbrain	Click-through-rate Prediction	69,709 / 8,715 / 8,718
RetailRocket	Conversion-rate Prediction	80,008 / 9,995 / 9,997
Stackexchange	User Churn Prediction	142,877 / 88,164 / 105,612

Appendix

Table 2: Statistics of relational database tasks.

Dataset	# Samples	# Feats	Dataset	# Samples	# Feats
Bioresponse	3,434	419	Default-Credit (Num)	13,272	20
Diabetes130US	71,090	7	Default-Credit (Cat)	13,272	21
Higgs	940,160	24	Electricity (Num)	38,474	7
MagicTelescope	13,376	10	Electricity (Cat)	38,474	8
MiniBooNE	72,998	50	Eye_Movements (Num)	7,608	20
Albert	58,252	31	Eye_Movements (Cat)	7,608	23

Appendix

Table 3: Statistics of the Single-Table Classification Datasets used for verification. (Num) and (Cat) denote two variants of the same dataset: one containing exclusively numerical features and one containing categorical features, respectively.

Panel of Figure 6 — Figure 6: Additional Baseline Analysis. Left: comparison after strengthening classical baselines with tuned XGBoost and AutoGluon using the best_quality preset under a 5-minute time budget. Right: comparison with Griffin on the subset of overlapping tasks, using an idealized reference that selects the best-performing Griffin checkpoint for each task group. These additional comparisons preserve the qualitative conclusion while clarifying the strength and protocol differences of the baselines.

Figure 7: Diversity of Generated Temporal Patterns. We visualize the row density over time for four distinct synthetic tables. By composing primitives from our Temporal Vocabulary (Trend, Seasonality, Spike), the prior generates complex, non-i.i.d. distributions.

Figure 8: Main benchmark results under PR-AUC. We report PR-AUC to complement ROC-AUC on imbalanced binary classification tasks. The qualitative comparison remains similar to the ROC-AUC results in the main text.

Panel of Figure 9 — Figure 9: Adaptation and larger effective support. (a) Target-task fine-tuning further improves over strict ICL, while cross-task fine-tuning gives a smaller but positive gain. (b) Chunked-support ensembling increases the effective support size beyond the pre-training context budget, improving performance with an expected runtime trade-off.

Panel of Figure 10 — Figure 10: Relational prior and backbone ablations. (a) Simplifying components of the relational synthetic prior generally degrades performance. (b) The default interleaved attention design outperforms weaker blockwise variants, while reduced-size models follow the expected capacity trend.

DFS Config	Diginetica	Outbrain	Rel F1	Rel F1	Rel Hm	Stackexchange	Stackexchange	Average
	Ctr	Small Ctr	Driver Dnf	Driver Top3	User Churn	Churn	Upvote	Average
DFS-1	0.5225	0.4914	0.4955	0.5445	0.5294	0.5592	0.8555	0.5711
DFS-2	0.7004	0.5352	0.7188	0.8115	0.6648	0.8477	0.8527	0.7330
DFS-2-MaxMinMode	0.6029	0.5363	0.7222	0.8148	0.6212	0.8327	0.8447	0.7107
DFS-2-MeanCount	0.6135	0.4906	0.7195	0.8205	0.6708	0.8419	0.8419	0.7141

Appendix

Table 4: DFS configuration sensitivity. We evaluate RDB-PFN at context size 1024 under different DFS configurations on small relational tasks. DFS-1 uses 1-hop relational features, while DFS-2 is the default 2-hop setting. DFS-2-MaxMinMode and DFS-2-MeanCount keep the 2-hop neighborhood but restrict the aggregation functions to the indicated subsets. The default 2-hop configuration performs best on most tasks, suggesting that both relational depth and aggregation choice affect performance, although the sensitivity is task-dependent.

Model	Parameters (M)	Pretraining Data (M)	Inference Time (s)
Our Model
RDB-PFN	0.7	2	34
Tabular Foundation Models (Full / Ensemble)
TabICL v1.1	27.1	80	229
Mitra	72	45	164
TabPFN v2.5	10.7	Undisclosed	156

Appendix

Table 5: Efficiency and complexity comparison of baseline models. Total inference time is calculated as the cumulative time required to evaluate all 19 relational benchmark tasks using a fixed 500-shot context (

N=500

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6036	0.5115	0.5792	0.5073	0.6530	0.5628	0.5743	0.4824	0.7326	0.6032
XGBoost	0.5757	0.5049	0.5000	0.5109	0.6417	0.5505	0.4884	0.5357	0.7317	0.6279
XGBoost-Tuned	0.5871	0.5099	0.5000	0.5046	0.6369	0.5271	0.4922	0.5230	0.7134	0.5968
AutoGluon-Medium	0.5818	0.5059	0.5000	0.5038	0.6516	0.5339	0.4979	0.5254	0.7169	0.6170

Appendix

Table 6: Raw Results for Context Size 64

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6030	0.5234	0.5400	0.5002	0.6953	0.5816	0.5578	0.4754	0.7445	0.5909
XGBoost	0.5801	0.5154	0.5486	0.4989	0.6814	0.5657	0.4905	0.5635	0.7731	0.6049
XGBoost-Tuned	0.5708	0.5241	0.5158	0.4987	0.6448	0.5531	0.4951	0.5531	0.7021	0.6132
AutoGluon-Medium	0.6071	0.5137	0.4972	0.5082	0.6693	0.5741	0.4922	0.5477	0.7407	0.6012

Appendix

Table 7: Raw Results for Context Size 128

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6230	0.5189	0.5826	0.5177	0.7135	0.5812	0.5728	0.5250	0.7709	0.6257
XGBoost	0.5992	0.5163	0.5697	0.5142	0.7056	0.5792	0.5063	0.5631	0.8075	0.6265
XGBoost-Tuned	0.6050	0.5123	0.5331	0.5095	0.7074	0.5791	0.5326	0.5863	0.7642	0.6429
AutoGluon-Medium	0.6075	0.5246	0.5173	0.5055	0.7274	0.5683	0.5006	0.5868	0.7797	0.6122

Appendix

Table 8: Raw Results for Context Size 256

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6392	0.5277	0.6084	0.5174	0.7292	0.6064	0.6129	0.5016	0.7821	0.6919
XGBoost	0.6219	0.5289	0.6143	0.5105	0.7289	0.5978	0.5707	0.5799	0.8100	0.6939
XGBoost-Tuned	0.6408	0.5215	0.5756	0.5079	0.7297	0.6116	0.5785	0.6057	0.7848	0.7089
AutoGluon-Medium	0.6479	0.5280	0.5475	0.5223	0.7278	0.6073	0.5666	0.5852	0.7750	0.6731

Appendix

Table 9: Raw Results for Context Size 512

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6360	0.5289	0.6125	0.5183	0.7425	0.6124	0.5973	0.5169	0.7913	0.6807
XGBoost	0.6200	0.5319	0.6198	0.5127	0.7410	0.5969	0.5409	0.5932	0.8174	0.6947
XGBoost-Tuned	0.6405	0.5307	0.6371	0.5179	0.7448	0.6065	0.5930	0.6167	0.8209	0.7277
AutoGluon-Medium	0.6749	0.5299	0.5953	0.5208	0.7537	0.5952	0.5373	0.5974	0.8101	0.7075

Appendix

Table 10: Raw Results for Context Size 1024

Part 1: First 12 Datasets
Model	Bioresponse	Diabetes130us	Higgs	Magictelescope	Miniboone	Albert	Bank	California	Compas	Covertype	Covertype	Credit
							Marketing		Two Years		(v2)
Random Forest	0.8256	0.5861	0.7307	0.9107	0.9620	0.6899	0.8579	0.9318	0.6547	0.8579	0.8222	0.8386
XGBoost	0.8300	0.5666	0.7329	0.9046	0.9703	0.6647	0.8432	0.9397	0.6470	0.8480	0.8147	0.8148
AutoGluon	0.8146	0.6159	0.7208	0.9078	0.9650	0.6845	0.8582	0.9355	0.7210	0.8459	0.8104	0.8455
Mitra	0.8083	0.6341	0.7525	0.9250	0.9735	0.6969	0.8708	0.9475	0.7302	0.8456	0.8252	0.8518

Appendix

Table 11: Single-Table Benchmark Results

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.5394	0.3691	0.0245	0.2002	0.7463	0.4534	0.0205	0.1505	0.9465	0.5481
XGBoost	0.5306	0.3676	0.0098	0.1943	0.7221	0.4114	0.0151	0.1607	0.9388	0.5372
AutoGluon	0.5269	0.3655	0.0098	0.1956	0.7388	0.4162	0.0154	0.1602	0.9401	0.5538
Mitra	0.5259	0.3702	0.0286	0.1973	0.7794	0.4102	0.0206	0.1603	0.9365	0.4980

Appendix

Table 12: Raw PR-AUC Results for Context Size 64

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.5418	0.3760	0.0154	0.1935	0.7807	0.4642	0.0181	0.1503	0.9497	0.5305
XGBoost	0.5149	0.3744	0.0104	0.1935	0.7291	0.4375	0.0155	0.1733	0.9363	0.5356
AutoGluon	0.5478	0.3690	0.0098	0.2008	0.7596	0.4676	0.0154	0.1733	0.9497	0.5424
Mitra	0.5760	0.3821	0.0342	0.1917	0.8114	0.4759	0.0189	0.1760	0.9412	0.5401

Appendix

Table 13: Raw PR-AUC Results for Context Size 128

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.5677	0.3745	0.0173	0.2024	0.7893	0.4703	0.0195	0.1668	0.9571	0.5513
XGBoost	0.5472	0.3694	0.0115	0.2014	0.7816	0.4551	0.0179	0.1879	0.9487	0.5659
AutoGluon	0.5612	0.3796	0.0104	0.1994	0.8168	0.4720	0.0158	0.1815	0.9564	0.5286
Mitra	0.6271	0.3860	0.0302	0.1979	0.8478	0.5165	0.0184	0.1845	0.9533	0.5544

Appendix

Table 14: Raw PR-AUC Results for Context Size 256

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.5799	0.3811	0.0190	0.2029	0.8101	0.5081	0.0242	0.1591	0.9591	0.6023
XGBoost	0.5815	0.3760	0.0137	0.2002	0.8032	0.4917	0.0207	0.1956	0.9583	0.6223
AutoGluon	0.5812	0.3817	0.0113	0.2109	0.8180	0.5167	0.0195	0.1848	0.9559	0.5858
Mitra	0.6553	0.3913	0.0304	0.1972	0.8622	0.5713	0.0225	0.1894	0.9574	0.6023

Appendix

Table 15: Raw PR-AUC Results for Context Size 512

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.5817	0.3826	0.0226	0.2012	0.8194	0.5157	0.0222	0.1671	0.9612	0.6042
XGBoost	0.5921	0.3815	0.0176	0.2050	0.8190	0.5065	0.0217	0.1983	0.9648	0.6444
AutoGluon	0.6157	0.3844	0.0140	0.2100	0.8401	0.5072	0.0184	0.1884	0.9630	0.6174
Mitra	0.6510	0.4054	0.0333	0.1969	0.8642	0.5708	0.0193	0.2081	0.9628	0.6241

Appendix

Table 16: Raw PR-AUC Results for Context Size 1024

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
RDBPFN-Finetune	0.7337	0.5561	0.7600	0.5322	0.8054	0.6576	0.6459	0.6598	0.8282	0.7456
RDBPFN-Cross-RDB-Finetune	0.7153	0.5630	0.7193	0.5305	0.7858	0.6455	0.6398	0.6486	0.8182	0.7572
TabPFNv2.5-Finetune	0.7280	0.5520	0.6887	0.5357	0.7982	0.6475	0.5884	0.6346	0.8228	0.6445
Part 2: Remaining 9 Datasets and Average

Appendix

Table 17: Fine-tuning and cross-task adaptation results. We compare RDB-PFN fine-tuned on the target task, RDB-PFN cross-task fine-tuned on other RDB tasks excluding the target task, and TabPFNv2.5 target-task fine-tuning. The fine-tuning data can be spread across the training set, but the context size remains limited to 1,024.

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6532	0.5289	0.6454	0.5207	0.7650	0.6186	0.6036	0.5340	0.8017	0.6963
XGBoost	0.6383	0.5341	0.6494	0.5118	0.7624	0.6006	0.5563	0.6058	0.8233	0.6947
AutoGluon	0.6850	0.5324	0.6058	0.5274	0.7658	0.6269	0.5417	0.6115	0.8082	0.7020
Mitra	0.7118	0.5571	0.7205	0.5123	0.7943	0.6495	0.5604	0.6436	0.8116	0.7304

Appendix

Table 18: Raw Results for Effective Context Size 2048 via Chunked-Support Ensembling

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6620	0.5378	0.6826	0.5207	0.7737	0.6207	0.6131	0.5362	0.8026	0.6809
XGBoost	0.6529	0.5424	0.6715	0.5111	0.7711	0.6040	0.5571	0.6153	0.8222	0.6675
AutoGluon	0.6968	0.5446	0.6488	0.5213	0.7797	0.6258	0.5523	0.6056	0.8240	0.7371
Mitra	0.7146	0.5618	0.7448	0.5184	0.7959	0.6415	0.5578	0.6496	0.8126	0.7119

Appendix

Table 19: Raw Results for Effective Context Size 4096 via Chunked-Support Ensembling

Part 1: First 10 Datasets
Model	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Random Forest	0.6629	0.5401	0.6918	0.5167	0.7783	0.6240	0.6157	0.5362	0.8027	0.6809
XGBoost	0.6577	0.5412	0.6853	0.5123	0.7739	0.6131	0.5519	0.6199	0.8165	0.6675
AutoGluon	0.7134	0.5448	0.6515	0.5205	0.7952	0.6348	0.5703	0.6353	0.8179	0.7358
TabPFNv2.5	0.6856	0.5593	0.7127	0.5296	0.8144	0.6624	0.6489	0.6061	0.8404	0.7477

Appendix

Table 20: Raw Results for Effective Context Size 8192 via Chunked-Support Ensembling

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6295	0.5142	0.5646	0.5152	0.6747	0.5674	0.5692	0.4936	0.7407	0.6132
Simpler DAG Generator	0.5984	0.5050	0.5594	0.5075	0.6694	0.5674	0.5554	0.5080	0.7191	0.5768
Remove Temporal Component	0.6051	0.5104	0.5639	0.5073	0.6612	0.5706	0.5703	0.5031	0.7240	0.6019
Simpler Edge Generation	0.6126	0.5084	0.5464	0.5036	0.6757	0.5746	0.5776	0.4989	0.7218	0.5873

Appendix

Table 21: Raw Relational Prior Ablation Results for Context Size 64

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6598	0.5293	0.5727	0.5079	0.7211	0.5875	0.5597	0.5530	0.7486	0.6211
Simpler DAG Generator	0.6014	0.5213	0.5711	0.5069	0.7341	0.6005	0.5258	0.5559	0.7331	0.6116
Remove Temporal Component	0.5736	0.5261	0.5710	0.5065	0.7316	0.5974	0.5517	0.5513	0.7166	0.6258
Simpler Edge Generation	0.6470	0.5282	0.5502	0.5019	0.7370	0.6031	0.5609	0.5416	0.7315	0.6207

Appendix

Table 22: Raw Relational Prior Ablation Results for Context Size 128

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6770	0.5224	0.6312	0.5274	0.7451	0.6096	0.5844	0.5990	0.7979	0.6810
Simpler DAG Generator	0.6503	0.5133	0.6180	0.5245	0.7523	0.6246	0.5629	0.5911	0.7731	0.6508
Remove Temporal Component	0.6615	0.5203	0.6225	0.5227	0.7498	0.6171	0.5698	0.5966	0.7562	0.6744
Simpler Edge Generation	0.6660	0.5239	0.5956	0.5227	0.7486	0.6209	0.5775	0.6072	0.8005	0.6485

Appendix

Table 23: Raw Relational Prior Ablation Results for Context Size 256

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.7313	0.5431	0.6081	0.5271	0.7543	0.6306	0.6069	0.6192	0.8010	0.7398
Simpler DAG Generator	0.6868	0.5365	0.6121	0.5247	0.7612	0.6409	0.5985	0.6165	0.7858	0.7001
Remove Temporal Component	0.6991	0.5425	0.6163	0.5203	0.7713	0.6420	0.6057	0.6123	0.7643	0.7117
Simpler Edge Generation	0.6979	0.5388	0.6130	0.5203	0.7659	0.6428	0.6199	0.6189	0.8037	0.7009

Appendix

Table 24: Raw Relational Prior Ablation Results for Context Size 512

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.7360	0.5454	0.6707	0.5351	0.7564	0.6308	0.6065	0.6503	0.8012	0.7510
Simpler DAG Generator	0.6969	0.5428	0.6397	0.5349	0.7673	0.6401	0.6142	0.6434	0.7822	0.7223
Remove Temporal Component	0.6965	0.5464	0.6650	0.5355	0.7782	0.6439	0.6193	0.6463	0.7547	0.7383
Simpler Edge Generation	0.6961	0.5395	0.6598	0.5216	0.7663	0.6419	0.6251	0.6482	0.8049	0.7086

Appendix

Table 25: Raw Relational Prior Ablation Results for Context Size 1024

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6005	0.5086	0.5645	0.5104	0.6717	0.5604	0.5753	0.4934	0.7337	0.5816
Cols First	0.5202	0.5138	0.5672	0.4982	0.4928	0.4889	0.5862	0.4712	0.5866	0.5327
Rows First	0.4461	0.4884	0.4810	0.4995	0.4711	0.5265	0.6197	0.3851	0.7061	0.4739
Small Capacity	0.5643	0.5081	0.5834	0.5030	0.6571	0.5506	0.5781	0.4909	0.7218	0.6289

Appendix

Table 26: Raw Backbone and Model-Size Ablation Results for Context Size 64

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6150	0.5252	0.5471	0.5098	0.7178	0.5981	0.5743	0.5301	0.7551	0.6127
Cols First	0.5083	0.5049	0.5722	0.5189	0.4945	0.4745	0.5806	0.4886	0.6092	0.5461
Rows First	0.4664	0.4874	0.4900	0.5013	0.4787	0.5451	0.6199	0.3841	0.7002	0.4457
Small Capacity	0.5813	0.5185	0.5495	0.5113	0.7062	0.5776	0.5549	0.5252	0.7287	0.6595

Appendix

Table 27: Raw Backbone and Model-Size Ablation Results for Context Size 128

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.6665	0.5196	0.5986	0.5292	0.7358	0.6127	0.5880	0.5824	0.7906	0.6512
Cols First	0.4753	0.5116	0.5619	0.5290	0.4965	0.4879	0.6025	0.5004	0.6261	0.5083
Rows First	0.4833	0.4859	0.4885	0.5019	0.4872	0.5302	0.6216	0.3837	0.7038	0.3970
Small Capacity	0.6141	0.5123	0.5823	0.5282	0.7295	0.5688	0.5560	0.5694	0.7567	0.6797

Appendix

Table 28: Raw Backbone and Model-Size Ablation Results for Context Size 256

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.7131	0.5388	0.5836	0.5352	0.7445	0.6333	0.6187	0.6021	0.8024	0.7132
Cols First	0.5110	0.5141	0.5759	0.5315	0.4985	0.4931	0.6120	0.5274	0.6704	0.4972
Rows First	0.5047	0.4843	0.4919	0.5038	0.5037	0.5538	0.6233	0.3828	0.7042	0.4200
Small Capacity	0.6257	0.5388	0.5722	0.5327	0.7484	0.6112	0.5945	0.5691	0.7669	0.7076

Appendix

Table 29: Raw Backbone and Model-Size Ablation Results for Context Size 512

Part 1: First 10 Datasets
Variant	Amazon	Avs	Diginetica	Outbrain	Rel Amazon	Rel Amazon	Rel Avito	Rel Avito	Rel Event	Rel Event
	Churn	Repeater	Ctr	Small Ctr	Item Churn	User Churn	User Clicks	User Visits	User Ignore	User Repeat
Original	0.7166	0.5494	0.6638	0.5402	0.7482	0.6369	0.6265	0.6376	0.8003	0.7301
Cols First	0.5365	0.5099	0.5890	0.5137	0.4954	0.5106	0.6121	0.5706	0.6681	0.5115
Rows First	0.5241	0.4822	0.4905	0.5006	0.5087	0.5465	0.6231	0.3826	0.7072	0.4001
Small Capacity	0.6410	0.5265	0.6089	0.5359	0.7516	0.6090	0.5879	0.6131	0.7464	0.7196

Appendix

Table 30: Raw Backbone and Model-Size Ablation Results for Context Size 1024

Explore similar work

PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining
Jul 31, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathyRelational Foundation Models Synthetic Data Pretraining
SAP Labs, LLC., Palo Alto 94304, United States.
Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data
Jul 31, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathySynthetic Data Pretraining Synthetic Data Generation
SAP Labs, LLC. Palo Alto, CA, United States
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Oct 13, 2026Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal +5Relational Foundation Models Relational Deep Learning
Stanford University · SAP

Jul 31, 2026J/K move · Enter open · S save

PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining

Mohammad Sadeq Abolhasani, Viswanath Ganapathy

SAP Labs, LLC., Palo Alto 94304, United States.

Relational Foundation Models Synthetic Data Pretraining Relational Reasoning In-Context Learning Curriculum Learning

Open paper AskPDF arXiv

Relational Foundation Models (RFMs) require large-scale synthetic relational databases for pretraining, but existing approaches tightly couple data generation with the model training pipeline. We study whether PluRel, a general-purpose synthetic relational database generator, can serve as an external data source for RDB-PFN, a relational in-context learner originally pretrained with a 600K-task single-table warm-up followed by an approximately 1.8M-task adaptation stage. We build a conversion pipeline that maps PluRel-generated databases, including externally constructed binary prediction tasks, into the RDB-PFN training format and evaluate three curriculum strategies: SCHEMA-GUIDED FIRST (real-world schema then fully synthetic), FULLY SYNTHETIC (diverse synthetic schemas throughout), and SCHEMA-GUIDED LAST (fully synthetic then real-world schema). Using only approximately 5,500 relational databases (approximately 33K tasks), roughly 55x fewer tasks than the original protocol, and no single-table warm-up, our best curriculum (SCHEMA-GUIDED FIRST) achieves 0.6346 average ROC-AUC across 19 real benchmark tasks at 1024-shot context, recovering 87.6% of the published RDB-PFN performance (0.7245). At 64-shot context, the gap narrows to 93.8% (0.6116 vs. 0.6517). Our results demonstrate that external synthetic generators can provide useful pretraining signals for RFMs when combined with appropriate curriculum design and that exposure to a real-world schema early in training is substantially more effective than late-stage schema adaptation.

Abstract

Explore similar work

PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions