Tabular Foundation Models
Overview
Tabular foundation models aim to make predictions on new tables with little or no task-specific training, often by conditioning on labeled examples in context. Many use transformers or related architectures pretrained on synthetic tasks generated from causal or statistical priors; other approaches learn from real tables, retrieve relevant examples, or adapt frozen models with lightweight heads. Benchmarks such as TabArena and TALENT show strong results on many small- and medium-sized, in-distribution tasks, while gradient-boosted trees and specialized methods remain competitive, especially on large or high-dimensional data. Open challenges include robustness to distribution shift, calibrated uncertainty and fairness, and reducing the memory and inference costs of processing large contexts.
Latest papers
209 papers23 papers in the last four weeks, down 45% on the four weeks before. 0.2% of all new papers.
Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?
Beyond IID: How General Are Tabular Foundation Models, Really?
FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks
KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models
Towards Evaluating Data Priors for Tabular Foundation Models
Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Models
Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees
Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries
Are Tabular Foundation Models Robust to Realistic Query Distribution Shifts in Microbiome Data?
Universal Encoders for Modular Relational Deep Learning
Bounded Context Management for Tabular Foundation Models on Stream Learning
Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function
Tabular Foundation Models for Clinical Survival Analysis via Survival-Aware Adaptation
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
Towards Unified and Data-Efficient Prognostics and Health Management with Tabular Foundation Models
GOTabPFN: From Feature Ordering to Compact Tokenization for Tabular Foundation Models on High-Dimensional Data
Towards Pretraining Text Encoders for TabPFN
SurvPFN: Towards Foundation Models for Survival Predictions
SurvPFN, a prior-data fitted network (PFN), for survival prediction tasks. We pretrain SurvPFN on millions of synthetic survival prediction tasks to learn survival via distributional regression that accounts for censored data. SurvPFN works by (1) generating data with Weibull event times and a non-informative censoring mechanism; (2) integrating a censored event indicator; and (3) minimizing a censored negative log-likelihood. On SurvSet, a collection of real-world survival tasks, SurvPFN is highly competitive with classical and deep survival baselines without per-dataset fitting, a survival-specific architecture, or feature engineering. We show that survival can be treated as a continuous-time distributional regression problem with censored loss, unlocking the power of PFNs for time-to-event predictions.