OmniTabBench: Mapping the Empirical Frontiers of GBDTs, Neural Networks, and Foundation Models for Tabular Data at Scale

Dihong Jiang; Ruoqi Cao; Zhiyuan Dang; Li Huang; Qingsong Zhang; Zhiyu Wang; Shihao Piao; Shenggao Zhu

cs.LGApr 8, 2026

OmniTabBench: Mapping the Empirical Frontiers of GBDTs, Neural Networks, and Foundation Models for Tabular Data at Scale

Authors: Dihong Jiang, Ruoqi Cao, Zhiyuan Dang, Li Huang, Qingsong Zhang, Zhiyu Wang, Shihao Piao, Shenggao Zhu,

Organizations: Huawei Cloud · School of Intelligence Science and Technology, Peking University

Abstract

While traditional tree-based ensemble methods have long dominated tabular tasks, deep neural networks and emerging foundation models have challenged this primacy, yet no consensus exists on a universally superior paradigm. Existing benchmarks typically contain fewer than 100 datasets, raising concerns about evaluation sufficiency and potential selection biases. To address these limitations, we introduce OmniTabBench, the largest tabular benchmark to date, comprising 3030 datasets spanning diverse tasks that are comprehensively collected from diverse sources and categorized by industry using large language models. We conduct an unprecedented large-scale empirical evaluation of state-of-the-art models from all model families on OmniTabBench, confirming the absence of a dominant winner. Furthermore, through a decoupled metafeature analysis, which examines individual properties such as dataset size, feature types, feature and target skewness/kurtosis, we elucidate conditions favoring specific model categories, providing clearer, more actionable guidance than prior compound-metric studies.

Figures & tables

Figure 1: Workflow of constructing OmniTabBench

Benchmark	#Datasets	Total Sizes		Sources	Tasks	Categorize?
Benchmark	#Datasets	#Rows	#Cols	Sources	Tasks	Categorize?
TableShift ( Gardner et al., 2023 )	15	17.0M	15.6k	UCI, Kaggle, Public	BC	✗
TabArena ( Erickson et al., 2025 )	51	1.0M	6.0k	OML, UCI, Kaggle	C, R	✗
Grinsztajn et al. (2022) benchmark	45	10.9M	2.1k	OML	C, R	✗
TabReD ( Rubachev et al., 2025 )	8	2.2M	3.0k	Kaggle, New	C, R	✗
TabZilla ( McElfresh et al., 2023 )	36	2.4M	8.2k	OML	C	✗

Table 1: Comparison between OmniTabBench and existing benchmarks. In column “Sources”, “OML” represents OpenML, “New” means that the authors introduce new datasets from ML products at a company, and “Public” means that datasets are collected from other publicly available sources, such as Physionet and American Community Survey. In column “Tasks”, “BC”, “C”, and “R” denote binary classification, classification (including multi-class), and regression, respectively.

Panel of Figure 2 — Figure 2: Comparison between OmniTabBench and existing representative benchmarks. (a) We visualize the number of rows, columns, and the percentage of categorical columns per dataset in OmniTabBench, as well as their distributions. (b) TabZilla and TabArena contain notably fewer datasets than OmniTabBench.

Figure 3: Categorization of OmniTabBench by industries

Figure 4: Rank of different models with increasing the number of evaluation datasets. The zoom-in window takes a closer look at the rank variation with a limited number of datasets. We plot this figure from datasets that all eight models have results.

Rank-1 Models	Sizes			Feature Types		Feature Distribution		Target Distribution
Rank-1 Models	$\#R$	$\#C$	$\#R/\#C$	$\%Cat.$	$\%Mis.$	$\|\gamma\|$	$\kappa$	Entropy	$\|\gamma\|$	$\kappa$
GBDT	6143.5	40.3	349.7	0.437	0.011	0.796	1.510	0.800	0.780	0.431
NN	8259.7	39.1	399.4	0.574	0.016	0.645	1.241	0.754	0.951	1.433
TabPFN	2773.2	36.0	166.2	0.368	0.014	0.625	0.524	0.860	0.687	0.126
GBDT vs NN	0	0.841	0.265	0	0.052	0	0.764	0.009	0.108	0.083
TabPFN vs NN	0	0.527	0	0	0.442	0.644	0.343	0	0.007	0.019

Table 2: The mean metafeatures of datasets that each category of models excel. We use

\gamma,\kappa

to denote skewness and kurtosis, respectively. We compute p-values via Welch’s t-test in the bottom three rows, where the statistically significant (

p<0.05

) pairwise difference is highlighted in bold (p-values lower than 1e-3 are rounded to 0).

Figure 5: The distribution of performance gap between different pairs of models. Five columns represent five different metafeatures, and three rows denote three pairwise comparison. Performance gap on different datasets refers to the subtraction of the score between the former and latter models (for example, NN vs GBDT means subtracting score of GBDT from NN), which are quantified by red (positive/win) and blue (negative/loss) points, respectively. We also fit a PDF of the points along each varying metafeatures via kernel density estimation.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The distribution of performance gap between NNs and GBDT. Performance gap on different datasets refers to the subtraction of the score between the former and latter models (for example, NN vs GBDT means subtracting score of GBDT from NN), which are quantified by red (positive/win) and blue (negative/loss) points, respectively. We also fit a PDF of the points along each varying metafeatures via kernel density estimation.

Figure 7: The distribution of performance gap between TabPFN and NN.

Figure 8: The distribution of performance gap between TabPFN and GBDT.

Tabular ML

Tabular machine learning develops models and tools for learning from structured data arranged in rows and columns.

See all 114 papers

Beyond IID: How General Are Tabular Foundation Models, Really?
Jun 29, 2026Lennart Purucker, Andrej Tschalzev, Nick Erickson +7Tabular Foundation Models OOD Generalization
1Prior Labs · University of Mannheim · 4INRIA Saclay +5
Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks
Apr 23, 2026Liane Vogel, Kavitha Srinivas, Niharika D'Souza +3Tabular Foundation Models Representation Learning
Technical University of Darmstadt · IBM Research, USA
Explaining Tabular Foundation Model Differences Through Meta-Features
May 27, 2026Markus Herre, Andrej Tschalzev, Sascha Marton +1Tabular Foundation Models Tabular ML
Clausthal University of Technology, Clausthal-Zellerfeld, Germany · University of Mannheim, Mannheim, Germany

Jun 29, 2026J/K move · Enter open · S save

Beyond IID: How General Are Tabular Foundation Models, Really?

Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer+6

1Prior Labs · University of Mannheim · 4INRIA Saclay+5

Tabular Foundation Models OOD Generalization Tabular ML Tabular Prediction Cross-Dataset Generalization

Open paper AskPDF arXiv

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.

Abstract

Tabular ML

Explore similar work

Beyond IID: How General Are Tabular Foundation Models, Really?

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

Explaining Tabular Foundation Model Differences Through Meta-Features