Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations

Mohammad Sadeq Abolhasani; Viswanath Ganapathy

cs.LGOct 16, 2026

Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations

Authors: Mohammad Sadeq Abolhasani, Viswanath Ganapathy

Organizations: SAP Labs

Abstract

Prior-data fitted networks (PFNs) show that pretraining on large populations of synthetic tasks can enable generalization to unseen datasets with little or no downstream fine-tuning. This paradigm is effective for transformer-based tabular and relational foundation models such as RDB-PFN, but remains largely unexplored for graph-based relational models built on message passing. We pretrain the graph-based relational foundation model Griffin on large collections of synthetic relational databases and ask whether curriculum learning, which benefits transformer-based relational learners, also improves message-passing representations. Rather than fine-tune on downstream tasks, we use the pretrained encoder as a general-purpose relational representation learner and pass its latent representations to a frozen in-context predictor (\ticl{}); no component is trained on the target task. On 16 real-world relational classification tasks, evaluated over five seeds against raw-feature and untrained-encoder baselines, a purely synthetic curriculum that begins with schema-guided data and continues with schema-agnostic data gives the strongest frozen in-context performance\rev{, outperforming real-data-only pretraining and every unordered single-stage mixture of the corpora; when appended after real-data pretraining, it yields the best encoders overall}. These results provide initial evidence that PFN-style synthetic pretraining transfers to graph-based relational foundation models.

Figures & tables

Curriculum	Com1	Com2	Othr1	Othr2	Avg	Rank
$\mathrm{PT}_{\mathrm{C2O2}}\!\rightarrow\!\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	0.628 $\pm$ 0.013	–	0.716 $\pm$ 0.006	–	0.672 $\pm$ 0.004	1
$\mathrm{PT}_{\mathrm{C1O1}}\!\rightarrow\!\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	–	0.618 $\pm$ 0.030	–	0.730 $\pm$ 0.005	0.655 $\pm$ 0.018	2
$\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	0.584 $\pm$ 0.013	0.613 $\pm$ 0.016	0.709 $\pm$ 0.012	0.729 $\pm$ 0.009	0.648 $\pm$ 0.010	3
$\mathrm{PT}_{\mathrm{C2O2}}$	0.569 $\pm$ 0.005	–	0.699 $\pm$ 0.021	–	0.634 $\pm$ 0.012	4
$\mathrm{PT}_{\mathrm{C1O1}}$	–	0.575 $\pm$ 0.018	–	0.716 $\pm$ 0.008	0.622 $\pm$ 0.014	8
$\mathrm{PT}_{\textsc{sa}}$	0.508 $\pm$ 0.029	0.582 $\pm$ 0.019	0.704 $\pm$ 0.008	0.717 $\pm$ 0.007	0.614 $\pm$ 0.010	9

Table 1: Frozen in-context ROC-AUC by task family (context 1024), mean

\pm

standard deviation over five seeds. Curricula initialized on real data are evaluated only on their held-out families. Avg is the mean over the run’s evaluated tasks.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Task	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)
Com1: diginetica-downsample-ctr	0.665	–	0.605	0.607	–	0.550	0.596	–	0.512	0.552	0.542	0.490
Com1: rel-hm-user-churn	0.565	–	0.566	0.578	–	0.587	0.565	–	0.541	0.565	0.495	0.542
Com1: retailrocket-cvr	0.877	–	0.901	0.858	–	0.945	0.906	–	0.773	0.838	0.835	0.495
Com1: seznam-charge	0.435	–	0.330	0.270	–	0.279	0.278	–	0.252	0.245	0.351	0.226
Com1: seznam-prepay	0.597	–	0.518	0.532	–	0.534	0.510	–	0.462	0.544	0.503	0.282
Com2: amazon-churn	–	0.640	0.623	–	0.639	–	0.674	0.613	0.629	0.657	0.565	0.500

Appendix

Table 2: Per-task ROC-AUC under frozen in-context prediction (context 1024), mean over five seeds. Empty cells: the family was used for pretraining and is excluded from evaluation. Columns: (1)

\mathrm{PT}_{\mathrm{C2O2}\rightarrow\textsc{sg}\rightarrow\textsc{sa}}

, (2)

\mathrm{PT}_{\mathrm{C1O1}\rightarrow\textsc{sg}\rightarrow\textsc{sa}}

, (3)

\mathrm{PT}_{\textsc{sg}\rightarrow\textsc{sa}}

, (4)

\mathrm{PT}_{\mathrm{C2O2}}

, (5)

\mathrm{PT}_{\mathrm{C1O1}\cup\textsc{sg}\cup\textsc{sa}}

, (6)

\mathrm{PT}_{\mathrm{C2O2}\cup\textsc{sg}\cup\textsc{sa}}

, (7)

\mathrm{PT}_{\textsc{sg}\cup\textsc{sa}}

, (8)

\mathrm{PT}_{\mathrm{C1O1}}

, (9)

\mathrm{PT}_{\textsc{sa}}

, (10)

\mathrm{PT}_{\textsc{sg}}

, (11) RandGriffin , (12) RawFeat .

Curriculum	Com1 $+$ Othr1 (10 tasks)	Com2 $+$ Othr2 (6 tasks)
$\mathrm{PT}_{\mathrm{C2O2}}\!\rightarrow\!\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	0.672 $\pm$ 0.004	–
$\mathrm{PT}_{\mathrm{C1O1}}\!\rightarrow\!\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	–	0.655 $\pm$ 0.018
$\mathrm{PT}_{\textsc{sg}}\!\rightarrow\!\mathrm{PT}_{\textsc{sa}}$	0.646 $\pm$ 0.010	0.652 $\pm$ 0.010
$\mathrm{PT}_{\mathrm{C2O2}}$	0.634 $\pm$ 0.012	–
$\mathrm{PT}_{\mathrm{C1O1}}$	–	0.622 $\pm$ 0.014
$\mathrm{PT}_{\textsc{sa}}$	0.606 $\pm$ 0.011	0.627 $\pm$ 0.013

Appendix

Table 3: ROC-AUC averaged over fixed task subsets (context 1024), mean

\pm

standard deviation over five seeds. Each column compares all runs that were evaluated on that subset.

Conclusion	Comparison	$\boldsymbol{\Delta}$	SE	$\boldsymbol{\Delta/\mathrm{SE}}$	Tasks
Pretraining helps (RQ1)	$\mathrm{PT}_{\textsc{sg}\rightarrow\textsc{sa}}$ vs. RandGriffin	$+$ 0.058	0.005	12.4	all 16
	$\mathrm{PT}_{\textsc{sg}}$ vs. RandGriffin	$+$ 0.015	0.005	3.2	all 16
	RandGriffin vs. RawFeat	$+$ 0.051	0.004	13.3	all 16
More data helps (RQ5)	$\mathrm{PT}_{\mathrm{C1O1}\rightarrow\textsc{sg}\rightarrow\textsc{sa}}$ vs. $\mathrm{PT}_{\mathrm{C1O1}}$	$+$ 0.033	0.010	3.2	Com2 $+$ Othr2
	$\mathrm{PT}_{\mathrm{C2O2}\rightarrow\textsc{sg}\rightarrow\textsc{sa}}$ vs. $\mathrm{PT}_{\mathrm{C2O2}}$	$+$ 0.038	0.006	6.7	Com1 $+$ Othr1
Synthetic vs. real alone (RQ2)	$\mathrm{PT}_{\textsc{sg}\rightarrow\textsc{sa}}$ vs. $\mathrm{PT}_{\mathrm{C2O2}}$	$+$ 0.012	0.007	1.7	Com1 $+$ Othr1

Appendix

Table 4: Key comparisons with their seed standard error.

\Delta

is the difference of the two means on the task set in the last column; SE is the standard error of the difference over five seeds per arm.

	B	C	D	SALT ( sg )
Databases	5,623	1,408	2,112	500
Tables per DB (min/max/avg)	2 / 4 / 3.0	2 / 4 / 3.0	5 / 8 / 6.5	4 / 4 / 4
Total rows per DB (min–max)	150–600	50–400	50–400	500–5,000
Columns per table (min/max/avg)	2 / 13 / 7	4 / 20 / 10	4 / 22 / 7	2 / 13 / 7
Tasks per DB	$\sim$ 6 (binary)	$\sim$ 6 (binary)	$\sim$ 6 (binary)	7 (multi-class)

Appendix

Table 5: Synthetic pretraining corpora. sa

=

\cup

\cup

D (9,143 DBs); sg

=

SALT (500 DBs). (Moved from Section 3 ; content unchanged.)

Encoder / sampling		Optimization / evaluation
Message-passing layers	4	Batch size	256
Hidden dim.	512	Learning rate / weight decay	$3\!\times\!10^{-4}$ / $2\!\times\!10^{-4}$
Subgraph hops	2	Max / min steps	100k / 5k
Fanout / few-shot fanout	20 / 3	Eval. interval / patience	250 / 20–30
Reversed FK edges	yes	Binarization (synth. / real)	$k{=}2$ / $k{=}0$
Gating	no	TabICL (v2) ensemble / context	8 / $\{512,1024,2048\}$

Appendix

Table 6: Shared configuration across all pretraining stages and frozen TabICL evaluation. (Moved from Section 3 ; content unchanged.)

Relational Foundation Models

Relational foundation models learn reusable representations and predictors for tasks spanning linked tables and databases.

See all 17 papers

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data
Jul 31, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathySynthetic Data Pretraining Synthetic Data Generation
SAP Labs, LLC. Palo Alto, CA, United States
PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining
Jul 31, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathyRelational Foundation Models Synthetic Data Pretraining
SAP Labs, LLC., Palo Alto 94304, United States.
Relational In-Context Learning via Synthetic Pre-training with Structural Prior
Mar 4, 2026Yanbo Wang, Jiaxuan You, Chuan Shi +1Relational ICL Relational Foundation Models
Institute for Artificial Intelligence, Peking University · University of Illinois at Urbana-Champaign · Institute of Computing Technology, Beijing University of Post and Telecommunication +1

Jul 31, 2026J/K move · Enter open · S save

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

Mohammad Sadeq Abolhasani, Viswanath Ganapathy

SAP Labs, LLC. Palo Alto, CA, United States

Synthetic Data Pretraining Synthetic Data Generation Relational Reasoning Prior-Data Fitted Networks Curriculum Learning Relational Deep Learning+1

Open paper AskPDF arXiv

Relational Prior-Data Fitted Networks (PFNs) such as RDB-PFN approximate Bayesian inference over multi-table relational databases by pretraining on millions of synthetic tasks. We investigate three intertwined questions about this paradigm. First, can a structurally different synthetic generator PluRel substitute for RDB-PFN's prior? Second, how much does the order in which synthetic data is presented to the PFN affect downstream performance? Third, how much relational reasoning can a PFN acquire from single-table synthetic pretraining alone, before any relational data is introduced? Using PluRel as the sole synthetic data source across all experiments, we find: (i) a progressive single-table curriculum that gradually widens schema complexity from 7 to 17 columns reaches 0.703 average ROC-AUC on the 23-task tabular benchmark using only approximately 13,300 synthetic tables (approximately 45x fewer single-table datasets than RDB-PFN's reported warm-up recipe), while the same data trained all-at-once collapses to 0.541 ROC-AUC; (ii) a relational curriculum trained from scratch on only approximately 5,500 PluRel databases reaches 0.638 average ROC-AUC on the 19-task RelBench/4DBInfer benchmark, recovering 88% of RDB-PFN's reported performance with approximately 220x less relational synthetic data; and (iii) the single-table curriculum model, evaluated directly on the relational benchmark without any relational adaptation, achieves 0.631, nearly matching the dedicated relational pipeline. Together, these findings suggest that curriculum design and synthetic data diversity may matter more for relational PFN pretraining than the specific relational generator or raw synthetic scale alone.

Abstract

Relational Foundation Models

Explore similar work

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining

Relational In-Context Learning via Synthetic Pre-training with Structural Prior