We introduce Kumo Tabular, a family of open tabular foundation models for classification and regression, in three sizes: Kumo Tabular-S (27M parameters), Kumo Tabular-M (61M), and Kumo Tabular-L (214M). Pretrained only on artificial tables from the Kumo prior, they can make predictions on new tables without any training. In our approach, a table encoder compresses each row into an embedding, and an in-context learning transformer predicts the unlabeled rows from the labeled ones. Kumo Tabular introduces architectural innovations that improve both training stability and predictive performance, enabling the model to scale up to 150M cells on modern GPUs. Kumo Tabular-L ranks first on eight benchmarks: TabArena, TALENT, BeyondArena, ScoringBench, STRABLE, MulTaBench, RamanBench, and TabBench-Bio. These cover probabilistic regression, tables with over 1M rows or 100K columns, and tables with strings, text, or images. Accuracy improves with model size in all tested comparisons. All three sizes lie on the accuracy-speed frontiers of TabArena and BeyondArena. Kumo Tabular-L predicts faster than the runners-up, LimiX-2 on TabArena (36 times) and TabPFN-3.5 on BeyondArena (1.7 times). Weights and pretraining code are released under OpenMDW-1.1 and Apache 2.0, respectively.
Figures & tables
Figure 1: Kumo Tabular is accurate and fast. On (a) TabArena and (b) BeyondArena, all three sizes lie on the Pareto frontier of Elo and prediction time.
Figure 2: Kumo Tabular-L tops all eight leaderboards. (a) Entries and scores under each leaderboard’s main metric, Kumo Tabular in green. (b) Row and column counts of each dataset (log scale). ScoringBench subsamples to 3,000 rows, and STRABLE and MulTaBench count columns before encoding. Tables of each benchmark are colored black over all tables in gray. Above the dashed line, tables have more columns than rows. See Table 3 for benchmark versions, evaluation protocols, ranking procedures, and subsampling limits.
Figure 3: Architecture of Kumo Tabular . (1) The token of cell (i,j) encodes the cells in columns j , j+1 , and j+3 of row i , their missingness flags, and, for train rows, the label. (2) Column and row attention alternate K times. Column attention passes through inducing points that read the train cells of each column. Four readout tokens ( [CLS] ) form the row embedding, which enters the ICL transformer as hi . (3) Test rows attend only to train rows. In Kumo Tabular-M and Kumo Tabular-L , they read two shared key/value heads (Test-GQA). Length-aware temperature keeps attention sharp as tables grow longer or wider.
Kumo Tabular-S
Kumo Tabular-M
Kumo Tabular-L
Parameters, classifier / regressor
27.46M / 28.47M
61.49M / 62.49M
213.67M / 215.68M
ICL share of parameters, classifier / regressor
93.8% / 94.1%
82.9% / 83.2%
95.3% / 95.4%
Table Encoder
Stages (column block → row block)
4
6
6
Cell width d
128
256
256
Heads × head width
4×32
4×64
4×64
Inducing points M
128
256
256
Table 1: The Kumo Tabular family. Model configurations of Kumo Tabular across model sizes and task types.
Figure 4: The Kumo prior combines causal dependencies with measurement noise and missing values. Building on TabICLv2 ( Qu et al., 2026 ) , steps 1–3 sample the table dimensions, variable types, causal graph, and node functions. Step 4 maps the resulting node representations to numerical or categorical features and targets. Our extensions to the readout and post-processing stages introduce noisy measurements, correlated features, varied feature and target distributions, and missingness patterns (steps 4–5). A small tree ensemble filters tables by target predictability (step 6). Retained tables are split into labeled train rows and test rows, whose targets supervise pretraining. Appendix A specifies the new and modified components.
Stage 1
Stage 2
Stage 3
Rows per table
1,024
400–10,240
200–60,000
Train rows (share)
30–90%
79–81%
79–81%
Features
2–100
2–100
2–100 (M/L: fewer above 32K/25K rows)
Realism augmentations
none
+relabeled targets
+heavy-tailed targets
+coarsened features
+high-cardinality codes
Table 2: Training recipe by stage , for the classifier and the regressor of every model size. Kumo Tabular-S , Kumo Tabular-M , and Kumo Tabular-L use 8, 32, and 64 NVIDIA H100 80 GB GPUs, respectively, in all stages.
Benchmark
Datasets (cls./reg.)
Rows
Columns
Splits
Metrics
Ranking; uncertainty
Entries
TabArena v0.1.9.8, 19 Oct
51 (38/13)
748–150,000
4–1,776
3-fold CV, 10 or 3 repeats; 816 splits
ROC AUC, log-loss, RMSE
Elo, RandomForest = 1000; bootstrap
97
TALENT scored 25 Sep
261 (172/89)
323–97,922
3–308
one fixed split
accuracy, RMSE
Elo, RandomForest = 1000; 200 bootstrap rounds
41
BeyondArena v0.1.9.8, 19 Oct
142 (98/44)
104–1,000,350
3–22,215
random (103), temporal (21), grouped (18); 2,000 splits
ROC AUC, log-loss, RMSE
Elo, XGBoost = 1000; bootstrap
40
ScoringBench 9 Oct ledger, 19 Oct
101 (0/101)
240– 107 ( ≤ 3,000)
3–1,024
5-fold CV
CRPS (51 metrics)
mean CRPS rank; Friedman and Nemenyi tests
62
STRABLE v0.1.6, 23 Oct
108 (32/76)
550–75,000
4–61, with strings
3-fold CV
ROC AUC, R2
median score over 324 table-folds; no intervals
94 pipelines
MulTaBench v1.9, 19 Oct
80 (40/40)
400–114,000 ( ≤ 10,000)
2–248, with text or images
3-fold CV, 2 repeats
1 − AUC, log-loss, RMSE
Elo, RandomForest = 1000; 1,000 bootstrap rounds
26
Table 3: The eight benchmarks. Under each name, the leaderboard version and the date of access in 2026 (TALENT and TabBench-Bio : the date we scored them). Datasets: classification/regression. Rows: per dataset (TALENT and BeyondArena: training rows), with the most a model sees in parentheses where the benchmark subsamples. Columns: features (BeyondArena: after preprocessing). Entries: each configuration of a method counted once.
Benchmark
Metric
Kumo
Strongest other entry
Head-to-head
S/M/L
Tabular-L
Name
Score
TabArena
Elo ( ↑ )
1962 [1862, 2124]
LimiX-2
1952 [1866, 2082]
49% win rate
5 / 3 / 1
TALENT
Elo ( ↑ )
1633 [1585, 1679]
LimiX-2
1576 [1536, 1614]
132–95 (0.02)
6 / 2 / 1
BeyondArena
Elo ( ↑ )
1426 [1382, 1468]
TabPFN-3.5
1376 [1341, 1416]
67% win rate
5 / 3 / 1
ScoringBench
CRPS rank ( ↓ )
1.84
LimiX-2
4.96
68–33 ( 6×10−4 )
4 / 2 / 1
STRABLE
Median score ( ↑ )
0.8325
Causilo
0.8190
95–13 ( <10−15 )
– / – / 1
Table 4: Benchmark results. Brackets show 95% uncertainty intervals where available. The strongest other entry reports the highest-ranked model outside the Kumo Tabular family. Head-to-head reports Kumo Tabular-L ’s wins–losses against that model, excluding ties, with exact two-sided sign-test p -values in parentheses. Comparisons are across datasets, except for STRABLE (tables) and TabBench-Bio (targets). TabArena and BeyondArena instead report win rates across splits, counting ties as half a win. Each dataset contributes equally to these win rates. For STRABLE, the reported score is the median over 324 table-folds, using R2 for regression and ROC AUC for classification. Both compared pipelines use the TILE encoder. S/M/L gives the leaderboard ranks of Kumo Tabular-S , Kumo Tabular-M , and Kumo Tabular-L variants. Dashes indicate unavailable results.
Figure 5: Benchmark scores by model size. Elo with 95% interval, or mean CRPS rank of each model size plotted against its number of model parameters.
Figure 6: Additive ablation from a TabICLv2 reimplementation to Kumo Tabular-L . Each row builds on the row above it and reports one minus the min-max normalized log loss on 36 TabArena classification datasets with fewer than 1,000 features (higher is better). On every outer split, the log losses of all models in the figure are rescaled to [0,1] between the best and the worst, and the result is averaged over splits and datasets. The light part of a bar is the score of the row it builds on, and the dark part is the change of the row, hatched if the change is negative. The first six rows are runs of 100K steps, with the mean and standard deviation over three pretraining seeds. The remaining rows are single checkpoints from our pretraining runs. All models are evaluated on all TabArena outer splits (576 in total). The inference rows change only the inference recipe of Kumo Tabular-L . Pretrain (h) is the wall-clock pretraining time on eight RTX PRO 6000 GPUs.
Figure 7: Benchmark scores after each pretraining stage. Elo with 95% intervals on (a) TabArena and (b) BeyondArena. (c) Rank on each BeyondArena dataset by number of training rows. Dotted lines report the mean ranks with 95% intervals shaded.
Figure 8: Accuracy, runtime, and memory across model sizes. (a) Average gap to the best entry on the BeyondArena datasets in four ranges of training rows, against the fit time of a table with 1,000, 5,000, 50K, or 200K training rows, respectively. For each dataset, the gap is 100(e−ebest)/e , where e and ebest are the errors of the evaluated model and the best entry. We report the mean gap over datasets. (b) Time to predict 1,000 test rows on one H100 with 80 GB VRAM and (c) peak GPU memory over fit and prediction with 100 features, including the model weights. The shaded area spans the memory for the three sizes with 500 features. Costs: binary classification with 100 features, recommended estimators, and FP16 autocast on one H100 80GB.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Mechanism
Difference to TabICLv2
Step
Section
Every stage
Distributional numerical converter
row-dependent noise, hidden subgroups, bounded values, and counts
4
A.2
Correlated feature blocks
groups of columns that share the same noise
5
A.3
Missing values
missing rows, feature-dependent missingness, values outside selected ranges, and random missing cells; stages 2 and 3 also add nearly empty columns and blocks controlled by a latent factor per row
5
A.5
Trivial examples
a test for nearly deterministic tables; rejected in stage 1, half kept in stages 2 and 3
6
A.6
Realism augmentations, stages 2 and 3
Appendix
Table A.1: Differences between the Kumo prior and the TabICLv2 prior. Step refers to the generation steps in Figure 4 .
Figure A.1: Numerical columns read from one node’s channels. Each panel converts the same 1,000 rows of three standard normal channels. (a) The pass-through converter of TabICLv2. (b–e) Individual components of the distributional converter: noise that grows with z1 ( w=0.7 , γ=0.5 ), a hidden subgroup ( w=0.9 , δ=2 ), the logistic link ( w=0.9 , a=1.5 , b=−0.5 , c=8 ), and the link with rounding.
Figure A.2: Correlated feature blocks. Absolute correlations between the 16 numerical features of one drawn table (a) before and (b) after three blocks with strengths ρ of 0.54, 0.44, and 0.45; (c) two features of the first block.
Figure A.3: Realism augmentations of stages 2 and 3. Each panel forces one augmentation on; gray: before, green: after. (a) Sampled and argmax classes of 1,500 rows from the same softmax logits, at their two-dimensional root points (u1,u2) ; 82% agree. (b) Classes before and after relabeling, against the score of the chosen features; circled: flipped labels. (c) One column coarsened to six levels in each mode. (d) Level sizes of three recoded columns. (e) Tails of a regression target under each warp.
Figure A.4: Missing-value patterns of the Kumo prior. Each panel applies one pattern, with the prior’s settings, to the same drawn regression table of 120 rows and 24 features, 16 numerical and then 8 categorical (dashed line). In (e), rows are sorted by the numerical column whose tail censors a categorical column; the other censored columns follow other sources, so their missing cells look scattered.
Figure B.1: Two signatures of training instability. Two BF16 classification runs from development, without QK-norm (a) and with QK-norm and additional learned scales (b). Top: parameter-based scale proxies relative to step 10K; middle: pre-clip gradient norm; bottom: training loss. (a) Without QK-norm, the proxy is the product of the largest singular values of the query and key projections. (b) With QK-norm, it is constructed from the RMS values of the learned query and key gains and per-channel query-scale parameters. Gradient norms and losses are shown as rolling medians.
We introduce TabPFN-3.5, our new flagship Tabular Foundation Model. It significantly outperforms its predecessor, TabPFN-3, and all existing baselines across a broad range of tabular problems. TabPFN-3.5 sets a new state of the art on standard tabular prediction in TabArena, and extends it to the data practitioners encounter in practice: non-i.i.d. data with temporal or grouped splits, tables with strings, text and images, high-cardinality categorical features, and wide tables with many features. These gains carry over to our task-specific harnesses: state of the art on relational data and stronger time-series forecasting. For faster inference, our variant TabPFN-3.5-Fast runs up to 3x faster than TabPFN-3 while keeping most of the accuracy gains. In addition, we upgrade TabPFN-3.5-Plus, expanding our multimodal capabilities with advanced text and date handling alongside proprietary inference optimizations. Finally, we release a new version of our Thinking mode, TabPFN-3.5-Thinking, which scales inference-time computation to push the state of the art further. It benefits from our stronger base model and from inference-time improvements that make it up to 12x faster than TabPFN-3-Thinking.
Tabular foundation models, such as TabPFNv2 and TabICL, have recently dethroned gradient-boosted trees at the top of predictive benchmarks, demonstrating the value of in-context learning for tabular data. We introduce TabICLv2, a new state-of-the-art foundation model for regression and classification built on three pillars: (1) a novel synthetic data generation engine designed for high pretraining diversity; (2) various architectural innovations, including a new scalable softmax in attention improving generalization to larger datasets without prohibitive long-sequence pretraining; and (3) optimized pretraining protocols, notably replacing AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 without any tuning surpasses the performance of the current state of the art, RealTabPFN-2.5 (hyperparameter-tuned, ensembled, and fine-tuned on real data). With only moderate pretraining compute, TabICLv2 generalizes effectively to million-scale datasets under 50 GB GPU memory while being markedly faster than RealTabPFN-2.5. We provide extensive ablation studies to quantify these contributions and foster open research by releasing code for inference, pretraining, and synthetic data generation at https://github.com/soda-inria/tabicl.
Jingang Qu, David Holzmüller, Gaël Varoquaux +1
SODA Team, INRIA Saclay, Palaiseau, France · Probabl, France
EXAONE Tabular is a compact tabular foundation model family for classification and regression via in-context learning, producing predictions without dataset-specific gradient updates. Pretrained exclusively on a synthetic structural-causal-model (SCM) prior, its central contribution is an architecture-centered redesign of tabular in-context learning. Rather than compressing features into a fixed row embedding before a separate row-level learner, EXAONE Tabular interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens. Across four public benchmarks, EXAONE Tabular combines strong predictive performance with high efficiency. On TabArena, its 20.81M-parameter classification model ranks first overall, surpassing tuned ensembles and 4-hour AutoML pipelines, while regression reaches the performance regime of the 1.64B-parameter TabFM at roughly 1/11 the inference cost. On BCCO and TALENT, EXAONE Tabular ranks second in classification and first in regression. On ScoringBench, it achieves the best mean rank for both point-estimation and predictive-distribution quality, leading the R2, RMSE, and CRPS evaluations. Together, these results establish EXAONE Tabular as a state-of-the-art compact tabular foundation model family, combining strong predictive performance across classification, point regression, and probabilistic regression with an efficient model design.