Tabular Foundation Models

Tabular foundation models aim to make predictions on new tables with little or no task-specific training, often by conditioning on labeled examples in context. Many use transformers or related architectures pretrained on synthetic tasks generated from causal or statistical priors; other approaches learn from real tables, retrieve relevant examples, or adapt frozen models with lightweight heads. Benchmarks such as TabArena and TALENT show strong results on many small- and medium-sized, in-distribution tasks, while gradient-boosted trees and specialized methods remain competitive, especially on large or high-dimensional data. Open challenges include robustness to distribution shift, calibrated uncertainty and fairness, and reducing the memory and inference costs of processing large contexts.