Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).
Figures & tables
Figure 1: Overall TabArena Elo ratings for the top 16 methods across all 51 benchmark datasets (38 classification and 13 regression). TabFM ranks first among zero-shot tabular foundation models, while TabFM+ ’s multi-view feature expansion, ensembling, and calibration and TabFM-Auto ’s LLM-guided feature engineering with Gemini-3.8-Flash take the top two positions overall without updating any weights.
Figure 2: The TabFM Architecture. (a) Tables are embedded cell-wise ( T×H×E ), compressed by two untied stages alternating column ISAB and row SAB with 8 CLS tokens into row representations R , conditioned on e(y) for train rows, then processed by a 24-layer in-context predictor. (b) Column ISAB attends across rows in linear time, while row SAB attends across features with RoPE.
Configuration
Stage
Module
Width
Heads
Induc.
FFN
Params
Function
Cell Embedder
Spectral proj.
128
–
–
–
0.05 M
Learned Fourier features
Column Embedder
2×3 ISAB
128
4
256
512
3.3 M
Linear row attention
Row Interaction
2×3 SAB
128
8
–
512
1.6 M
RoPE feature attention
ICL Predictor
24× SAB
1024
8
–
4096
402.7 M
Masked in-context mixing
Prediction Head
2-layer MLP
1024
–
–
1024
1.05 M
Output
Table 1: TabFM architecture specification (400M checkpoint, E=dmodel=128 ). All sublayers use sandwich RMSNorm with SwiGLU activations, zero dropout, and no bias parameters.
Figure 3: Synthetic pretraining and curriculum: (a) a sampled SCM graph with randomized functional dependencies, (b) the resulting table with numerical, categorical, and dual targets, and (c) the four-stage curriculum scaling context from 2,048 to 16,384 instances at constant tokens per step.
Table 2: TabArena benchmark leaderboards under repeated 10-fold cross-validation. Ratings are obtained from a unified Bradley–Terry fit anchored to RF (default) at 1000 Elo. G-mean is the geometric mean error across datasets, Wins counts fold-averaged dataset victories, and Improvability is the relative error reduction an oracle selector would achieve.
Figure 4: Task-separated TabArena Elo over 38 classification (top) and 13 regression (bottom) datasets, top 16 methods per suite. Colors mark TabFM variants in blue, peer foundation models in purple, AutoML in green, and tuned baselines in grey.
Figure 5: Pairwise win rates on TabArena: percentage of dataset-fold instances where the row method beats the column method, ties counted as half. Methods are ordered by Elo.
Figure 6: Per-dataset relative error reduction ( 1−errmethod/errTabFM ) over zero-shot TabFM across all 51 TabArena datasets. Bars show TabFM-Auto and diamonds show TabFM+ . TabFM-Auto improves 41 of 51 datasets, ten by more than 10%.
Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.
Deqing Fu, Huangyuan Su, Rajat Sen +4
Google Research · University of Southern California · Google DeepMind +2
Tabular foundation models (TFMs) have recently emerged as a promising paradigm for machine learning on tabular data, offering the ability to generalize across datasets without task-specific training. Since many machine learning datasets can be represented as tables, this raises the question: does TFM capability extend beyond tasks traditionally regarded as tabular? We address this question by using TabPFN v3 on three non-tabular classification problems: handwritten digit recognition on MNIST, language identification of French and German words, and image classification on Tiny ImageNet. In each case, the original data are represented as rows of a table and classification is formulated as prediction of a missing label. We evaluate performance as a function of the number of context samples provided to the pretrained model, with no additional training or fine-tuning. Despite having no explicit access to the spatial or sequential structure characterizing the data, TabPFN v3 in some cases achieves accuracies comparable with that of models or methods geared specifically toward the corresponding tasks.
Goran Nakerst, John Brennan, Wouter Beugeling +1
Institut für Theoretische Physik, Technische Universität Dresden, 01062 Dresden, Germany · Department of Physics, Maynooth University, Ireland · Physikalisches Institut (EP3) and Institute for Topological Insulators, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany
We introduce TabPFN-3.5, our new flagship Tabular Foundation Model. It significantly outperforms its predecessor, TabPFN-3, and all existing baselines across a broad range of tabular problems. TabPFN-3.5 sets a new state of the art on standard tabular prediction in TabArena, and extends it to the data practitioners encounter in practice: non-i.i.d. data with temporal or grouped splits, tables with strings, text and images, high-cardinality categorical features, and wide tables with many features. These gains carry over to our task-specific harnesses: state of the art on relational data and stronger time-series forecasting. For faster inference, our variant TabPFN-3.5-Fast runs up to 3x faster than TabPFN-3 while keeping most of the accuracy gains. In addition, we upgrade TabPFN-3.5-Plus, expanding our multimodal capabilities with advanced text and date handling alongside proprietary inference optimizations. Finally, we release a new version of our Thinking mode, TabPFN-3.5-Thinking, which scales inference-time computation to push the state of the art further. It benefits from our stronger base model and from inference-time improvements that make it up to 12x faster than TabPFN-3-Thinking.