Tabular Foundation Models

Follow this topic Ask about this topic

Tabular foundation models aim to make predictions on new tables with little or no task-specific training, often by conditioning on labeled examples in context. Many use transformers or related architectures pretrained on synthetic tasks generated from causal or statistical priors; other approaches learn from real tables, retrieve relevant examples, or adapt frozen models with lightweight heads. Benchmarks such as TabArena and TALENT show strong results on many small- and medium-sized, in-distribution tasks, while gradient-boosted trees and specialized methods remain competitive, especially on large or high-dimensional data. Open challenges include robustness to distribution shift, calibrated uncertainty and fairness, and reducing the memory and inference costs of processing large contexts.

Latest papers

188 papers39 papers in the last four weeks, up 44% on the four weeks before. 0.2% of all new papers.

Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules
Mar 9, 2026Jonas Landsgesell, Pascal Knoll, Tizian WenzelTabular Foundation Models Proper Scoring Rules
University of Stuttgart (Stuttgart, Germany) · Ludwig Maximilian University of Munich (Munich, Germany) · Munich Center for Machine Learning (Munich, Germany)
Engineering Regression Without Real-Data Training: Domain Adaptation for Tabular Foundation Models Using Multi-Dataset Embeddings
Mar 5, 2026Lyle Regenwetter, Rosen Yu, Cyril Picard +1Synthetic Data Pretraining Synthetic-to-Real Domain Adaptation
Massachusetts Institute of Technology
MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learning
Feb 23, 2026Wall Kim, Chaeyoung Song, Hanul KimTabular Foundation Models Multimodal Foundation Models
Samsung Electronics, Hwaseong, South Korea · Seoul National University of Science and Technology, Seoul, South Korea
Evaluating SAP RPT-1 for Enterprise Business Process Prediction: In-Context Learning vs. Traditional Machine Learning on Structured SAP Data
Feb 22, 2026Amit LalTabular Foundation Models Tabular Prediction
Microsoft Corporation
Tabular Foundation Models Can Learn Association Rules
Feb 16, 2026Erkan Karabulut, Daniel Daza, Paul Groth +2Association Rule Mining Tabular Foundation Models
University of Amsterdam, The Netherlands · Vrije Universiteit Amsterdam, The Netherlands · Amsterdam UMC, location VUmc, Vrije Universiteit Amsterdam, the Netherlands
TabICLv2: A better, faster, scalable, and open tabular foundation model
Feb 11, 2026Jingang Qu, David Holzmüller, Gaël Varoquaux +1Synthetic Data Pretraining Tabular Foundation Models
SODA Team, INRIA Saclay, Palaiseau, France · Probabl, France
Strengthening LLMs for Tabular Prediction with Structural Priors
Oct 20, 2025Pengxiang Cai, Zihao Gao, Wanchen Lian +2Tabular Foundation Models RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University
General Demographic Pre-trained Models for Enhancing Predictive Performance Across Diseases and Population
Sep 9, 2025Li-Chin Chen, Ji-Tian Sheu, Yuh-Jue ChuangHealthcare Representation Learning
Data Analytics and Digital Transformation Research Center National Taiwan University Taipei, Taiwan · Department of Health Care Management Chang Gung University Taoyuan, Taiwan

Mar 9, 2026J/K move · Enter open · S save

Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules

Jonas Landsgesell, Pascal Knoll, Tizian Wenzel

University of Stuttgart (Stuttgart, Germany) · Ludwig Maximilian University of Munich (Munich, Germany) · Munich Center for Machine Learning (Munich, Germany)

Tabular Foundation Models Proper Scoring Rules Distributional Regression Tabular Regression

Open paper AskPDF arXiv

We study how the objective used to train or fine-tune a tabular foundation model affects its distributional predictions. Models such as TabPFN and TabICL provide forecasts of the full conditional distribution, but regression benchmarks including TabArena and TALENT assess them mainly through RMSE and

R^2

. Such assessments reward accurate conditional means while leaving the remaining distributional information unassessed. We therefore examine proper scoring rules as objectives for training and fine-tuning and as criteria for evaluating and ranking these models. Every strictly proper rule favors the true distribution at the population level, however, this shared optimum does not determine how learning proceeds with a finite sample. Our examples illustrate score-dependent preferences between forecasts, and our fine-tuning experiments show differences in predictive performance under different objectives. Experiments that fine-tune TabPFN and TabICL on several datasets reveal that changing the training objective can change performance across evaluation scores, with non-trivial interactions between the two. By connecting established methods from distributional regression, scoring-rule theory, and foundation-model adaptation, we show that selecting the score is part of specifying the model and can measurably alter the distribution it learns.