Tabular foundation models (TFMs) pretrain a transformer on large-scale synthetic priors so that a single forward pass yields zero-shot tabular prediction. TabICLv2 samples the class count of its classification prior during pretraining, and the prevailing configuration caps it at ten. We report a counter-intuitive result: setting this cap directly to two---making every pretrained classification task binary---still yields weights that perform zero-shot K-way classification 2. On the 38 TabArena classification datasets (200 bootstrap rounds, anchored at default RF = 1000), the binary two-stage recipe reaches 1590.1 Elo, above the official default at 1568.0 (+22.1), above TabPFN-v2.6 (1577.1) and RealTabPFN-v2.5 (1521.6). A production-scale validation on 60 real-world credit-scoring institutions independently confirms the direction: the same checkpoint improves test AUC on 44 of 60 institutions (paired Wilcoxon p = 0.0007 for AUC, p = 0.0002 for KS) at +1.5 inference cost. Mechanistically, classification in TabICLv2 is carried not by a fixed-cardinality output head but by an in-context comparison operator matching query-row embeddings against class-conditional embeddings, so the cardinality of the output space is set by the labels present in the context, not by the label space seen during pretraining. This makes a binary prior structurally cheaper to generate and cache than a ten-class prior, demotes "number of classes" from an architectural constraint to a tunable hyperparameter, and points to a class-count-agnostic regime of TFM pretraining.
Figures & tables
Figure 1: TabArena Elo top-15 leaderboard on the full 38-dataset classification suite (200 bootstrap rounds, anchored at default RF = 1000). Light-blue/orange/green grouped bars show the Default, Tuned, and Tuned + Ensembled configurations of each method, and the highlighted yellow bar marks the TabICLv2 official default baseline. Our binary two-stage checkpoint (TabICLv2-Custom-Binary-Prior, dark orange) reaches 1590.1 Elo, +22.1 above the official default (1568.0).
Xtrain
→
standardize → feature grouping → column embedding
ytrain
→
label injection (label encoder)
Xtest
→
standardize → column embedding
↓
column-wise attention / set transformer
↓ CLS token
row-wise attention / ICL transformer
Figure 2: TabICLv2 forward pass schematic. Standardization, feature grouping, column embedding, label injection, column-wise and row-wise attention, and the final query-to-class comparison step are shown. The two components that govern the question of this paper are the label encoder and the comparison decoder (§ 3.3 ).
Configuration
Prior K
Steps
Elo ↑
Elo+ / Elo −
Rank ↓
Win rate
TabFM (reference)
10
—
1749.6
+133.5 / −114.7
9.59
0.901
TabICLv2-Custom-Binary-Prior (ours)
2
540K
1590.1
+75.2 / −63.3
17.91
0.806
TabPFN-v2.6 default (reference)
10
—
1577.1
+60.6 / −56.2
18.77
0.796
TabICLv2 official default (baseline)
10
550K
1568.0
+76.5 / −66.8
19.40
0.788
RealTabPFN-v2.5 default (reference)
10
—
1521.6
+54.7 / −52.1
22.80
0.749
Table 1: TabArena classification leaderboard (38 datasets, 200 bootstrap rounds, anchored at default RF = 1000). Elo+ / Elo − are bootstrap intervals; Rank is the mean rank; Win rate is the pairwise win ratio. Prior K is the classification prior’s target class count (for TabFM, the cap reported in [ 7 ] ; for the other reference checkpoints, the paper’s default). Steps gives total pretraining steps: the official default runs the full three-stage schedule (500K + 40K + 10K = 550K); our Custom variant is a from-scratch retrain of stage 1 + stage 2 only (540K, no stage 3). The four reference rows (TabFM, TabPFN-v2.6, the TabICLv2 official default, RealTabPFN-v2.5) use their publicly reported Elo without rerunning; “—” marks the original published schedule.
Figure 3: TabArena score on the full 38-dataset classification suite (binary + multiclass; higher is better), covering all evaluated methods. Our binary two-stage checkpoint (TabICLv2-Custom-Binary-Prior, dark orange) scores +0.013 above the TabICLv2 official default baseline (yellow); the Default / Tuned / Tuned + Ensembled bars show the per-method configurations reported by TabArena.
Checkpoint
Prior K
AUC test
KS test
Train − test gap
Cost (s)
official default (baseline)
10
0.7014
0.3082
0.2784
23.94
ours (binary, 540K)
2
0.7033
0.3134
0.2770
24.30
Δ (ours − default)
+0.0019
+0.0053
−0.0014
+0.36
Table 2: Production credit-scoring suite: 60 institutions (300 features each), institution-equal-weighted macro averages on the test hold-out. Δ is ours minus the official default. Across institutions, ours wins on test AUC in 44 / ties in 1 / loses in 15 (strict sign); the paired two-sided Wilcoxon signed-rank test (normal approximation with tie-averaged ranks) gives p=0.0007 on AUC and p=0.0002 on KS.
We introduce Mitra-v2, a tabular foundation model that delivers state-of-the-art performance on real-world classification and regression problems, from credit-risk scoring and clinical prediction to equipment-failure detection and house-price estimation. Mitra-v2 is trained only on synthetic data, with a pretraining distribution that is much larger and more diverse than Mitra-v1's. Built on a small 2D Transformer backbone, Mitra-v2 supports longer contexts and larger feature spaces. Improved optimization lets it learn from this larger task distribution. We evaluate Mitra-v2 on the TabArena and TALENT benchmarks, comprising more than 300 real-world datasets under two evaluation protocols. On the full TabArena benchmark, Mitra-v2 delivers state-of-the-art performance at the level of the industry-scale TabFM and EXAONE Tabular models, while surpassing TabPFN-3 by a wide margin in both classification and regression. Mitra-v2 matches the 1.6B-parameter TabFM with only 5% of its size (77M parameters), delivering frontier performance at a fraction of the cost. On TALENT, Mitra-v2 remains among the leading models, clearly outperforming TabPFN-3 and TabICLv2. It also ranks first on classification tasks with more than ten classes, even though it was pretrained only on tasks with at most ten classes. These results make Mitra-v2 one of the strongest and most broadly applicable open tabular foundation models released to date. We release the model weights, the inference and fine-tuning code, and our evaluation results under the Apache-2.0 license.
Tabular foundation models, such as TabPFNv2 and TabICL, have recently dethroned gradient-boosted trees at the top of predictive benchmarks, demonstrating the value of in-context learning for tabular data. We introduce TabICLv2, a new state-of-the-art foundation model for regression and classification built on three pillars: (1) a novel synthetic data generation engine designed for high pretraining diversity; (2) various architectural innovations, including a new scalable softmax in attention improving generalization to larger datasets without prohibitive long-sequence pretraining; and (3) optimized pretraining protocols, notably replacing AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 without any tuning surpasses the performance of the current state of the art, RealTabPFN-2.5 (hyperparameter-tuned, ensembled, and fine-tuned on real data). With only moderate pretraining compute, TabICLv2 generalizes effectively to million-scale datasets under 50 GB GPU memory while being markedly faster than RealTabPFN-2.5. We provide extensive ablation studies to quantify these contributions and foster open research by releasing code for inference, pretraining, and synthetic data generation at https://github.com/soda-inria/tabicl.
Jingang Qu, David Holzmüller, Gaël Varoquaux +1
SODA Team, INRIA Saclay, Palaiseau, France · Probabl, France
Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).