Relational Deep Learning

Latest papers 44

9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

Jun 3, 2026

OpenRFM: Dissecting Relational In-Context Learning

Relational Foundation Models (RFMs) promise a single pre-trained predictor that, given any relational database, returns predictions in one forward pass via relational in-context learning (ICL). Yet a substantial gap separates open RFMs from their commercial counterparts, and the origin of this gap has not been systematically understood. We dissect a representative framework, the Relational Transformer (RT), from two perspectives. Model side: we show that RT performs relation-level ICL, and a kernel regression view shows it fails when sparse label-cell coverage yields an underdetermined regression. Data side: we ablate RT's pre-training source and find that existing synthetic-only pre-training and in-distribution pre-training drive the same architecture into different regimes, lazy vs. feature-learning. Probing this gap reveals that the missing ingredient is a support-identifiable relational latent in the label-generation process. These two diagnoses translate into (1) a dual-stage ICL architecture that combines the relational backbone with a batch-level ICL layer lifted from a pre-trained tabular foundation model to overcome relation-level label scarcity, and (2) a homophily-aware synthetic plus continual real-data pre-training mixture, augmented with a prototype-based regularization. These choices define OpenRFM, a simple yet effective RFM that improves average task performance by approximately 30% over the RT backbone and surpasses the commercial model KumoRFMv1 on a large set of evaluation tasks.

Zhikai Chen, Junyu Yin, Jialiang Gu +5

Michigan State University · George Mason University · Georgia Institute of Technology +1

Relational Foundation Models Synthetic Data Pretraining Tabular Foundation Models Relational Reasoning In-Context Learning Relational Deep Learning+2

Ask PDF

Jun 2, 2026

RelGT-AC: A Relational Graph Transformer for Autocomplete Tasks in Relational Databases

Relational databases underpin modern enterprise, scientific, and healthcare systems, yet predictive machine learning on such data remains challenging due to their multi-table, heterogeneous, and temporal structure. Relational Deep Learning (RDL) addresses this by representing databases as heterogeneous graphs and applying graph neural networks (GNNs) directly. RelBench v2 recently introduced autocomplete tasks -- a practically motivated task type where the goal is to predict an existing column value from relational context, analogous to an intelligent form-filling assistant. We propose RelGT-AC (Relational Graph Transformer for Autocomplete), extending the RelGT architecture with three targeted contributions: (1) a column masking strategy that prevents trivial solutions by masking the target column during subgraph encoding; (2) a unified task head supporting binary classification, multiclass classification, and regression autocomplete tasks within a single model; and (3) a TF-IDF text encoder that automatically detects and encodes free-text columns, recovering strong lexical signal that categorical encoders discard. Across 7 tasks spanning 3 RelBench v2 datasets (rel-trial, rel-f1, rel-stack), RelGT-AC outperforms the GraphSAGE baseline on all 3 regression autocomplete tasks and achieves up to +10 AUROC points on text-heavy eligibility tasks via the TF-IDF encoder.

Phillip Jiang

Appsofa LLC

Relational Deep Learning Graph Transformer Relational Databases

Ask PDF

May 22, 2026

Incorporating Deep Learning Design in Database Queries

Deep learning over relational databases is conventionally realized by translating data into graph representations and applying graph-based neural networks within external frameworks. This round-trip between the database and external machine learning (ML) systems introduces non-trivial engineering overhead. In effect, these graph neural networks operate on tuple embeddings and manipulate them in ways that capture the interactions induced by relational joins. Given this natural correspondence, there is no fundamental reason why specifying a neural network over relational data should be substantially harder than querying it. We propose an approach that naturally integrates deep learning with database queries. The key idea is to associate each tuple with provenance, represented as a vector embedding with learnable parameters. Queries are lifted to operate jointly on data and embeddings, mapping input relations with embedded tuples to output relations with embedded tuples. This approach provides a declarative foundation for relational deep learning, facilitating integration with database systems, optimization, and wide adoption. We describe RelaNN, a proof-of-concept implementation of this approach built on top of PyTorch and cuDF. We illustrate the utility of RelaNN by implementing various graph-learning models, including graph convolutional networks, heterogeneous graph transformers, hypergraph neural networks and deep homomorphism networks. The simplicity of the programs and their competitive runtime performance demonstrate a concrete path toward making the implementation of state-of-the-art neural networks over databases as simple as writing a query.

Yuval Lev Lubarsky, Dean Light, Boaz Berger +2

Technion · University of Washington · RelationalAI

Relational Deep Learning Relational Databases

Ask PDF

May 22, 2026

RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databases

Relational databases (RDBs) remain the cornerstone of modern data systems and support diverse predictive tasks. Recent relational deep learning (RDL) methods enable end-to-end prediction by converting RDBs into graphs, where rows are represented as nodes and inter-table interactions are represented as edges, and then applying graph-based models for representation learning. Despite the strong capability of RDL, effective self-supervised pre-training for RDBs remains non-trivial. RDB tasks often require multi-faceted information across different perspectives and granularities. For example, user churn classification may rely more on interaction patterns, whereas consumption value prediction requires both user-item behaviors and intrinsic user attributes for fine-grained regression. Such heterogeneous needs challenge RDB representation learning, as pre-training objectives should cover comprehensive information for downstream adaptation. However, existing SSL methods typically derive supervision from a single facet, such as node-level intrinsic attributes or subgraph-level relational structures, providing limited adaptability. To this end, we propose RelPrism, a multi-faceted self-supervised learning framework for RDBs. RelPrism constructs intrinsic, relational, and hybrid attributes from distinct perspectives, and applies multi-granularity clustering to each perspective to form corresponding pseudo-task pools. Pre-training over these pools exposes representations to broader perspectives and granularity levels, yielding a stronger basis for downstream adaptation. Experiments on 14 tasks across 5 real-world datasets show that RelPrism improves ROC-AUC by 4.15% for classification and reduces MAE by 10.75% for regression over state-of-the-art baselines. Our code is available at https://anonymous.4open.science/r/RelPrism.

Jinyu Yang, Cheng Yang, Junze Chen +4

Beijing University Of Posts and Telecommunications Beijing, China · Peng Cheng Laboratory Shenzhen, China · China Telecom Bestpay Beijing, China +1

Representation Learning Self-Supervised Pre-Training Graph SSL Relational Deep Learning Relational Databases

Ask PDF

May 20, 2026

Is Fixing Schema Graphs Necessary? Full-Resolution Graph Structure Learning for Relational Deep Learning

Relational prediction tasks are fundamental in many real-world applications, where data are naturally stored in relational databases (RDBs). Relational Deep Learning (RDL) addresses this problem by modeling RDBs as graphs and applying graph neural networks (GNNs) for end-to-end learning. However, the full-resolution property is commonly adopted as a design principle in graph construction for RDBs to preserve relational semantics, which leads most existing methods to rely on fixed graph structures. In this paper, we propose FROG, a Full-Resolution and Optimizable Graph Structure Learning} framework for RDL that formulates relational structure learning as a learnable table role modeling problem, allowing tables to contribute as nodes and edges in message passing. We further design role-driven message passing mechanisms to capture relational semantics, enabling joint optimization of graph structure and GNN representations. To ensure semantic consistency, we introduce functional dependency constraints that regularize representations across table and entity levels. Extensive experiments demonstrate that our method outperforms existing approaches and reveal how table roles impact downstream tasks, offering new insights into graph construction for RDL

Yi Huang, Qingyun Sun, Jia Li +2

SKLCCSE, School of Computer Science and Engineering, Beihang University, Beijing, China · Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, China

Graph Structure Learning Graph Neural Networks Relational Deep Learning Relational Databases

Ask PDF

May 15, 2026

Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks

Relational databases store much of the world's structured information, and they are essential for driving complex predictive applications. However, deep learning progress on relational data remains limited, as conventional approaches flatten databases into single tables via manual feature engineering, discarding relational context. Relational deep learning (RDL) addresses this by modeling databases as relational entity graphs (REGs) for graph neural networks (GNNs), but remains task- and database-specific. To combine the strengths of both paradigms, we propose a hybrid architecture combining a fine-tuned BART encoder to capture intra-row semantics with a GraphSAGE-based GNN over REGs to inject relational context. Experiments on RelBench show that the GNN substantially enriches BART's row embeddings, achieving a ROC-AUC of 67.40 on the driver-dnf task from the rel-f1 dataset. This performance is competitive with supervised baselines such as LightGBM (68.86) and narrows the gap to RDL (72.62) to within 5.22 points, though a substantial gap remains to state-of-the-art foundation models such as KumoRFM (82.63). These results suggest that lightweight hybrid LM-GNN architectures offer a promising and resource-efficient path towards foundation models for relational databases.

Jingcheng Wu, Ratan Bahadur Thapa, Mojtaba Nayyeri +4

University of Stuttgart, Stuttgart, Germany · Web and Internet Science Research Group, University of Southampton, Southampton, United Kingdom

Relational Foundation Models Tabular Foundation Models Graph Neural Networks Graph Foundation Models Relational Deep Learning Relational Databases

Ask PDF

May 15, 2026

Gaussian Relational Graph Transformer

Relational graph learning enables predictive modeling over relational databases by directly capturing dependencies across interconnected tables. While relational graph Transformers extend the receptive field beyond local message passing, a larger receptive field does not necessarily provide more useful information: sampling must preserve relational structure without introducing excessive semantically irrelevant nodes, while attention must account for the temporal relevance of the retained information. We propose GelGT, a Gaussian relational graph transformer that explicitly addresses these challenges. GelGT introduces a structure-semantic collaborative sampling strategy to preserve structural connectivity while filtering irrelevant semantic information, and incorporates a Gaussian graph attention mechanism with a learnable Gaussian bias on the sampled subgraphs to dynamically encode temporal dependencies. We provide theoretical analysis characterizing the structural preservation, semantic refinement, and temporal discrimination of these mechanisms. Experiments on 7 real-world relational datasets covering 21 prediction tasks show that GelGT consistently outperforms existing relational graph learning methods, with improvements of up to 13.8%.

Zezhong Ding, Jin Li, Xugang Wang +1

School of Artificial Intelligence and Data Science, University of Science and Technology of China (USTC) · Data Darkness Lab, Suzhou Institute for Advanced Research, USTC · School of Biomedical Engineering, USTC +1

Graph Attention Networks Graph Representation Learning Relational Deep Learning Graph Transformer Relational Databases

Ask PDF

Apr 14, 2026

KumoRFM-2: Scaling Foundation Models for Relational Learning

We introduce KumoRFM-2, the next iteration of a pre-trained foundation model for relational data. KumoRFM-2 supports in-context learning as well as fine-tuning and is applicable to a wide range of predictive tasks. In contrast to tabular foundation models, KumoRFM-2 natively operates on relational data, processing one or more connected tables simultaneously without manual table flattening or target variable generation, all while preserving temporal consistency. KumoRFM-2 leverages a large corpus of synthetic and real-world data to pre-train across four axes: the row and column dimensions at the individual table level, and the foreign key and cross-sample dimensions at the database level. In contrast to its predecessor, KumoRFM-2 injects task information as early as possible, enabling sharper selection of task-relevant columns and improved robustness to noisy data. Through extensive experiments on 41 challenging benchmarks and analysis around expressivity and sensitivity, we demonstrate that KumoRFM-2 outperforms supervised and foundational approaches by up to 8%, while maintaining strong performance under extreme settings of cold start and noisy data. To our knowledge, this is the first time a few-shot foundation model has been shown to surpass supervised approaches on common benchmark tasks, with performance further improving upon fine-tuning. Finally, while KumoRFM-1 was limited to small-scale in-memory datasets, KumoRFM-2 scales to billion-scale relational datasets.

Valter Hudovernik, Federico López, Vid Kocijan +4

Relational Foundation Models Relational Deep Learning Few-Shot Learning Relational Databases In-Context Learning

Ask PDF

Mar 9, 2026

Rel-MOSS: Towards Imbalanced Relational Deep Learning on Relational Databases

In recent advances, to enable a fully data-driven learning paradigm on relational databases (RDB), relational deep learning (RDL) is proposed to structure the RDB as a heterogeneous entity graph and adopt the graph neural network (GNN) as the predictive model. However, existing RDL methods neglect the imbalance problem of relational data in RDBs and risk under-representing the minority entities, leading to an unusable model in practice. In this work, we investigate, for the first time, class imbalance problem in RDB entity classification and design the relation-centric minority synthetic over-sampling GNN (Rel-MOSS), in order to fill a critical void in the current literature. Specifically, to mitigate the issue of minority-related information being submerged by majority counterparts, we design the relation-wise gating controller to modulate neighborhood messages from each individual relation type. Based on the relational-gated representations, we further propose the relation-guided minority synthesizer for over-sampling, which integrates the entity relational signatures to maintain relational consistency. Extensive experiments on 12 entity classification datasets provide compelling evidence for the superiority of Rel-MOSS, yielding an average improvement of up to 2.46% and 4.00% in terms of Balanced Accuracy and G-Mean, compared with SOTA RDL methods and classic methods for handling class imbalance.

Jun Yin, Peng Huo, Bangguo Zhu +4

Department of Data Science and Artificial Intelligence, Hong Kong Polytechnic University, Hong Kong SAR, China · National Super Computing Center, Tianjin, China · School of Computer Science and Engineering, Central South University, Hunan, China +1

Relational Deep Learning Class-Imbalanced Learning Relational Databases Node Classification Graph Neural Networks

Ask PDF

Feb 26, 2026

Relatron: Automating Relational Machine Learning over Relational Databases

Predictive modeling over relational databases (RDBs) powers applications, yet remains challenging due to capturing both cross-table dependencies and complex feature interactions. Relational Deep Learning (RDL) methods automate feature engineering via message passing, while classical approaches like Deep Feature Synthesis (DFS) rely on predefined non-parametric aggregators. Despite performance gains, the comparative advantages of RDL over DFS and the design principles for selecting effective architectures remain poorly understood. We present a comprehensive study that unifies RDL and DFS in a shared design space and conducts architecture-centric searches across diverse RDB tasks. Our analysis yields three key findings: (1) RDL does not consistently outperform DFS, with performance being highly task-dependent; (2) no single architecture dominates across tasks, underscoring the need for task-aware model selection; and (3) validation accuracy is an unreliable guide for architecture choice. This search yields a model performance bank that links architecture configurations to their performance; leveraging this bank, we analyze the drivers of the RDL-DFS performance gap and introduce two task signals -- RDB task homophily and an affinity embedding that captures size, path, feature, and temporal structure -- whose correlation with the gap enables principled routing. Guided by these signals, we propose Relatron, a task embedding-based meta-selector that chooses between RDL and DFS and prunes the within-family search. Lightweight loss-landscape metrics further guard against brittle checkpoints by preferring flatter optima. In experiments, Relatron resolves the "more tuning, worse performance" effect and, in joint hyperparameter-architecture optimization, achieves up to 18.5% improvement over strong baselines with 10x lower cost than Fisher information-based alternatives.

Zhikai Chen, Han Xie, Jian Zhang +3

Michigan State University · Amazon

Relational Deep Learning Model Selection Automated Feature Engineering Relational Databases Meta-Learning

Ask PDF

Feb 19, 2026

A Locality Radius Framework for Understanding Relational Inductive Bias in Database Learning

Foreign key discovery and related schema-level prediction tasks are often modeled using graph neural networks (GNNs), implicitly assuming that relational inductive bias improves performance. However, it remains unclear when multi-hop structural reasoning is actually necessary. In this work, we introduce locality radius, a formal measure of the minimum structural neighborhood required to determine a prediction in relational schemas. We hypothesize that model performance depends critically on alignment between task locality radius and architectural aggregation depth. We conduct a controlled empirical study across foreign key prediction, join cost estimation, blast radius regression, cascade impact classification, and additional graph-derived schema tasks. Our evaluation includes multi-seed experiments, capacity-matched comparisons, statistical significance testing, scaling analysis, and synthetic radius-controlled benchmarks. Results reveal a consistent bias-radius alignment effect.

Aadi Joshi, Kavya Bhand

Dept. of Computer Engineering, Vishwakarma Institute of Technology, Pune, India.

Relational Deep Learning Graph Neural Networks

Ask PDF

Feb 3, 2026

PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.

Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik +4

1Stanford University · 2Kumo AI · 3SAP Labs LLC

Relational Foundation Models Synthetic Data Pretraining Synthetic Data Generation Synthetic Tabular Data Generation Relational Deep Learning Relational Databases

Ask PDF

Feb 3, 2026

Grables: When Graph Structure Helps Tabular Prediction

Simple row-wise predictors that score each row independently remain competitive with far more expressive architectures on tabular benchmarks, which are i.i.d. by construction. They cannot, however, represent natural targets on transactional, temporal, and relational tables, where labels depend on other rows. We show that row-wise prediction provably rules out targets driven by global counts, overlaps, and relational patterns. To make ``using structure'' precise across architectures, we introduce grables: a modular interface that separates how a table is lifted to a graph (constructor) from how predictions are computed on that graph (node predictor), placing gradient-boosted trees, neighbourhood aggregation, transformer-style tabular models, and relational GNNs on a single axis and pinpointing where expressive power comes from. The framework yields a hierarchy that answers when structure is needed: row-local models suffice for extension-invariant targets, lightweight neighbourhood aggregation captures shallow inter-row signals, and explicit graph construction is needed for compositional patterns that fixed aggregations cannot express without exponential feature blow-up. Experiments on synthetic, transactional, and relational data confirm the predicted separations, but also show that benchmark performance is a poor proxy for structural capability: row-local models score well on tasks they provably cannot represent, and a stress-test that suppresses row-local cues is needed to expose the gap. Conversely, access to cross-row context does not imply a model uses it. Feeding explicitly extracted inter-row structure to strong tabular learners yields gains exactly where the hierarchy predicts. The practical recipe is to add structure incrementally, and only where the target is extension-sensitive.

Tamara Cucumides, Floris Geerts

Department of Computer Science, University of Antwerp. Belgium.

Tabular ML Tabular Prediction Relational Deep Learning

Ask PDF

Jan 27, 2026

FloydNet: A Learning Paradigm for Global Relational Reasoning

Learning algorithmic computation often requires explicit relational intermediate states, yet many graph processors maintain their primary states on individual entities. We introduce \fnet and Pivotal Attention (PA), which maintain ordered pair states and update a target relation

(i,k)

by attending over candidates formed from

(i,j)

and

(j,k)

for every pivot

j

. Motivated by the pair-and-pivot structure of Floyd--Warshall, PA learns relation composition and pivot weighting in parallel rather than executing its ordered min-plus recurrence. The \kfnet{k} framework extends this operation to ordered

k

-tuples, with Self-Attention and PA as its

k=1

and

k=2

cases at the attention-operation level. Under atomic tuple initialization and invariant readout, we show that \kfnet{k} is no more graph-discriminative than k-FWL; on BREC, each evaluated variant matches the success set of its corresponding WL reference. \fnet further achieves 96.64% mean accuracy under the reported CLRS-30 protocol and a 99.8% optimality rate with 10 samples on held-out non-metric TSP instances.

Jingcheng Yu, Mingliang Zeng, Qiwei Ye

Beijing Academy of Artificial Intelligence

Graph Neural Networks Relational Reasoning Relational Deep Learning Algorithmic Reasoning

Ask PDF