TabOmics: A Small, Open Tabular Foundation Model for Wide Omics Data
Authors: Andrea López-García, Luis Bote-Curiel, Francisco Javier Gimeno-Blanes, José Luis Rojo-Álvarez, Sergio Muñoz-Romero
Organizations: Department of Signal Theory and Communications, Telematics and Computing, Universidad Rey Juan Carlos, Fuenlabrada, Spain · Department of Signal Theory and Communications, Universidad Miguel Hernández, Elche, Spain
Omics studies often produce tables with thousands to tens of thousands of molecular features and few samples. Tabular foundation models use diverse strategies to predict labels for such tables, with some relying on in-context learning, while approaches to wide tables may involve subsampling, grouping, or compressing features. We present TabOmics, a small, open family of tabular foundation models with 7.3M parameters, pretrained exclusively on synthetic tables. TabOmics processes all features retained after preprocessing jointly and derives predictions, attention-derived feature-importance scores, and test sample embeddings from the same forward pass. On 46 classification tasks from three benchmarks of high-dimensional, low-sample-size tables, TabOmics v2.5 with eight ensemble members achieves the highest mean area under the receiver operating characteristic curve (AUROC) among the evaluated models (0.943), outperforming TabPFN-3.5 with eight members on 32 tasks despite having about 30 times fewer parameters. With a single member, TabOmics v2.5 reaches a mean AUROC of 0.941, versus 0.942 for TabPFN-3.5 with eight members, with no significant paired difference under the reported task-level analysis. On synthetic tables, its feature-importance scores achieve feature recovery similar to that of TabPFN-Wide, and its predictions remain above chance at 30,000 features. On breast cancer tasks, the top ranked genes include PAM50 genes, which define the subtype labels, indicating concordance with the label definition rather than independent biological validation. The code, model weights, training recipes, and experimental artifacts are publicly available.
Figures & tables
Figure 1: Architecture of TabOmics. An input table is represented as a grid of feature and label tokens for N training rows and M test rows. Each of the L=12 Transformer blocks applies feature-wise attention within rows, row-wise attention within columns, and a token-wise two-layer multilayer perceptron (MLP). During row-wise attention, test tokens attend only to the training rows. The final test-row label tokens provide class probabilities and 192-dimensional sample embeddings, while label-to-feature attention provides one score per retained feature. Solid and pale tokens denote training and test rows, respectively. Colors identify columns.
Benchmark
Dataset / Task group
Samples
Features
Classes
Min. class
CV
HDLSS
ALLAML
72
7,129
2
25
10×3
BASEHOCK
1,993
4,862
2
994
10×3
CLL_SUB_111
111
11,340
3
11
10×3
GLI_85
85
22,283
2
26
10×3
PCMAC
1,943
3,289
2
961
10×3
Prostate_GE
102
5,966
2
50
10×3
Table 1: Summary of the 46 classification tasks used to evaluate TabOmics. Each HDLSS row represents one dataset, whereas each MLOmics and Shamir row groups the modality-specific tasks of one cancer type. The modalities included in each task group are listed below its name. The table reports the numbers of samples, input features, classes, the size of the smallest class, and the cross-validation scheme. For MLOmics and Shamir, feature counts are ranges across the listed modalities. Fold-specific preprocessing may further reduce the number of retained features. CV denotes the cross-validation scheme: 10×3 indicates 10 repetitions of stratified 3-fold cross-validation, whereas 5 indicates stratified 5-fold cross-validation. The four HDLSS development datasets and the excluded LGG methylation and combined tasks are not included.
Model
Stage
Features
Rows
Widened
Learning rate
Epochs
Steps
Tables
v2
1. Base
2–100
40–1,024
–
10−4
100
100k
0.80M
2. Longer tables
2–100
40–2,048
–
10−4
100
100k
0.80M
3. Widening
2–350
40–1,024
0.3
10−4
30
3.75k
0.03M
4. Low learning rate
2–350
40–1,024
0.3
3×10−5
30
3.75k
0.03M
Total
207.5k
1.66M
v2.5
1. Base
2–100
40–1,024
–
10−4
300
300k
2.40M
Table 2: Pretraining stages of TabOmics v2 and v2.5. Feature and row ranges refer to tables sampled before feature widening. Rows are sampled log-uniformly. The widening probability is the fraction of batches augmented with 200–3,000 additional features, as described in Section 3.2 . Steps denote optimizer updates, and table counts are obtained from the training logs and rounded as displayed.
Model
Parameters
Tables
Pretraining compute
TabOmics v2
7.3M
1.66M
2.1 A100 GPU-days
TabOmics v2.5
7.3M
2.46M
2.4 A100 GPU-days
TabPFNv2 ( Hollmann et al., 2025 )
7.2M a
130M
2 weeks on 8 RTX 2080 Ti GPUs
TabICLv2 ( Qu et al., 2026 )
27.6M a
35M
24.5 H100 GPU-days
TabPFN-3.5 ( Jäger et al., 2026 )
220M
Not reported
Not reported
Table 3: Model size and reported pretraining budget of TabOmics and other TFMs. Tables is the reported number of synthetic tables processed during pretraining, with the TabOmics values taken from Table 2 . For TabOmics, compute includes only the stages leading to each released checkpoint. Compute values are reported as provided by the original studies and are not normalized across hardware, software environments, table dimensions, or numerical precision.
Mean AUROC
TabOmics v2.5 × 1 vs.
TabOmics v2.5 × 8 vs.
Model
HDLSS
MLOmics
Shamir
All
Rank
W–L
p
W–L
p
TabOmics v2.5 × 8
0.946
0.938
0.947
0.943
2.76
–
–
–
–
TabPFN-3.5 × 8
0.948
0.938
0.945
0.942
3.89
20–26
0.410
32–14
0.034
TabOmics v2 × 8
0.944
0.936
0.947
0.941
4.13
–
–
–
–
TabOmics v2.5 × 1
0.945
0.935
0.947
0.941
4.60
–
–
–
–
TabOmics v2 × 1
0.943
0.934
0.945
0.939
5.35
–
–
–
–
Table 4: Predictive performance across the three benchmarks. Values are mean AUROC by benchmark and across all 46 tasks, together with the average within-task AUROC rank. W–L gives the number of tasks on which the indicated TabOmics v2.5 reference configuration obtains a strictly higher or lower mean AUROC than the comparison model, ties are omitted. p -values are from paired Wilcoxon signed-rank tests on task-level means and are Holm-adjusted within each comparison family. Models are ordered by overall mean AUROC. Bold indicates the highest displayed value in each column, including ties after rounding. Dashes denote comparisons not reported.
Seconds per fold
GPU memory (GB)
Model
AUROC
HDLSS
MLOmics
Shamir
All
Max
Median
Max
TabOmics v2.5 × 8
0.943
10.9
87.3
31.8
65.2
1,506
3.1
27.7
TabPFN-3.5 × 8
0.942
2.4
3.2
3.2
3.1
16
1.6
6.4
TabOmics v2 × 8
0.941
11.0
87.5
32.0
65.3
1,506
3.1
27.7
TabOmics v2.5 × 1
0.941
4.7
20.3
8.1
15.5
217
3.1
27.7
TabOmics v2 × 1
0.939
4.7
20.5
8.1
15.6
217
3.1
27.7
Table 5: Predictive performance and computational cost. The table reports mean AUROC across 46 tasks, median runtime per fold within each benchmark and across tasks, maximum runtime, and median and maximum peak GPU memory. Runtime includes model construction, loading locally stored weights, and predicting the test rows, and is the median of three timed runs on the first fold of each task. GPU models were evaluated on one GPU, with ensemble members run sequentially, using the hardware specified in the Hardware and software environment section. GPU memory is the peak allocation recorded by PyTorch and is not available for XGBoost or CPU-based methods. CPU–GPU runtime comparisons are specific to the reported hardware and are not hardware-normalized. Models are ordered as in Table 4 .
AUROC
Accuracy
TabOmics v2.5
Mean
W–T–L
p
Mean
W–T–L
p
Seconds
GPU (GB)
One member (reference)
0.9406
–
–
0.8277
–
–
15.5
3.1
Feature clip 100
0.9405
16–10–20
0.247
0.8276
7–29–10
1.000
–
–
Single precision
0.9407
24–4–18
0.837
0.8279
10–25–11
1.000
56.0
6.0
Two members
0.9419
34–1–11
< 0.001
0.8294
27–5–14
0.341
21.3
3.1
Four members
0.9421
34–1–11
< 0.001
0.8299
28–2–16
0.341
36.7
3.1
Table 6: Ablations of TabOmics v2.5 relative to its default one-member configuration. W–T–L gives the numbers of tasks with higher, equal, or lower performance than the default. p -values are from paired Wilcoxon signed-rank tests and are Holm-adjusted across ablations. Runtime and memory are measured as in Table 5 . The feature-clipping ablation was not retimed because it does not change the computational procedure. Mean AUROC and accuracy values are reported to four decimal places to show the small differences between configurations.
Figure 2: Feature recovery and predictive performance as table width increases. Left: AUROC for recovering the informative features from the feature-importance score rankings. Right: predictive AUROC on 200 held-out test rows. Points and error bars show the mean and standard error across 15 synthetic tables at each width, comprising five seeds for each setting with 3, 5, or 7 informative features. The horizontal dotted line indicates chance performance, and the vertical dotted line marks 5,000 features, the maximum width considered in the original TabPFN-Wide protocol.
AUROC of the top k genes
Stability
PAM50 in top 10
Method
k=10
k=50
k=100
k=500
Jaccard
Spearman
MLOmics
Shamir
TabOmics v2.5 × 1
0.926
0.955
0.954
0.962
0.34
0.75
4.8
4.2
TabPFN-Wide × 1
0.929
0.956
0.957
0.965
0.38
0.77
4.2
4.4
Random forest
0.920
0.949
0.953
0.959
0.14
0.08
2.0
1.6
ANOVA F
0.897
0.927
0.939
0.961
0.61
0.92
4.0
0.2
Random genes
0.782
0.884
0.911
0.945
–
–
–
–
Table 7: Predictive value, cross-fold stability, and PAM50 concordance of gene rankings on the eight mRNA tasks. Predictive value is the mean AUROC of logistic regression fitted using the top k genes from each outer-training ranking. The all-gene reference achieves 0.953. Stability is measured by the mean pairwise Jaccard index of the top 100 genes and the Spearman correlation between complete score vectors across folds. PAM50 concordance is the mean number of PAM50 genes among the top 10 in the two breast cancer tasks. The corresponding chance expectations are approximately 0.03 genes for MLOmics BRCA and 0.02 for Shamir breast. Bold indicates the highest displayed value among ranking methods. Dashes indicate that stability or marker concordance is not defined for random selection.
Logistic regression
Nearest neighbors
Direct
Representation
AUROC
W–L
p
AUROC
W–L
p
AUROC
TabOmics v2.5 × 1
0.967
–
–
0.952
–
–
0.972
TabOmics v2.5 × 8
0.967
3–5
0.742
0.951
5–2
0.297
0.973
TabPFN-Wide × 1
0.963
7–1
0.445
0.948
6–2
0.297
0.963
TabPFNv2 × 1
0.950
7–1
0.156
0.932
7–1
0.164
0.937
All genes
0.957
6–2
0.445
0.892
8–0
0.047
–
Table 8: Classification using sample representations on the eight mRNA tasks. Values are mean AUROC for logistic regression and nearest-neighbor classifiers fitted to each representation. W–L gives the number of tasks on which TabOmics v2.5 ×1 performs better or worse than the comparison representation, and p -values are Holm-adjusted Wilcoxon test results. For TabOmics v2.5 ×8, embeddings are averaged across members. Direct TFM prediction AUROC on the same folds is included as a reference and is not applicable to gene-expression, PCA, or PLS representations. Bold indicates the highest displayed value in each AUROC column, including ties after rounding.
Figure 3: Two-dimensional PCA projections of gene-expression features and TFM sample embeddings. The panels show the 352 training samples from the first outer fold of the Shamir breast task, colored by PAM50 subtype. PCA was fitted separately to each standardized representation, and the first two principal components are shown. TFM training samples were represented using out-of-fold embeddings, so each sample was embedded without its own label in the context. The figure illustrates one fold. Quantitative results across all folds of the eight mRNA tasks are reported in Table 8 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Mean accuracy
TabOmics v2.5 × 1 vs.
TabOmics v2.5 × 8 vs.
Model
HDLSS
MLOmics
Shamir
All
Rank
W–L
p
W–L
p
TabOmics v2.5 × 8
0.882
0.827
0.795
0.832
3.53
–
–
–
–
TabPFN-3.5 × 8
0.896
0.820
0.790
0.830
3.62
19–27
0.585
24–22
0.365
TabOmics v2 × 8
0.880
0.822
0.794
0.829
4.13
–
–
–
–
TabOmics v2.5 × 1
0.881
0.819
0.795
0.828
4.75
–
–
–
–
TabOmics v2 × 1
0.877
0.818
0.792
0.825
5.71
–
–
–
–
Appendix
Table 9: Mean accuracy per benchmark and over the 46 tasks, average rank, and pairwise comparisons, as in Table 4 for AUROC.
Task
TabOmics v2.5
TabOmics v2
TabPFN-3.5
TabPFN-Wide
× 1
× 8
× 1
× 8
× 1
× 8
× 1
× 8
HDLSS
ALLAML
0.998
0.998
0.998
0.998
0.989
0.996
0.996
0.997
BASEHOCK
0.998
0.999
0.996
0.998
0.974
0.999
0.999
0.999
CLL_SUB_111
0.897
0.898
0.893
0.896
0.821
0.891
0.875
0.883
GLI_85
0.944
0.945
0.939
0.945
0.944
0.951
0.945
0.947
Appendix
Table 10: AUROC of both TabOmics models, TabPFN-3.5, and TabPFN-Wide, with one and eight members, for each task, averaged across folds. Bold indicates the highest displayed AUROC among all 15 models for each task, including ties after rounding. The highest value may appear in this table, in Table 11 , or in both.
Task
TabPFNv2
TabICLv2
XGBoost
Random forest
Elastic-net LR
× 1
× 8
× 1
× 8
HDLSS
ALLAML
0.998
0.997
0.955
0.991
0.963
0.995
0.993
BASEHOCK
0.999
0.999
0.991
0.997
0.991
0.996
0.997
CLL_SUB_111
0.778
0.896
0.721
0.774
0.873
0.854
0.885
GLI_85
0.921
0.934
0.928
0.916
0.901
0.938
0.934
Appendix
Table 11: AUROC of TabPFNv2, TabICLv2, and the classical methods for each task, averaged across folds. Bold indicates the highest displayed AUROC among all 15 models for each task, including ties after rounding. The highest value may appear in this table, in Table 10 , or in both.
AUROC
TabOmics v2.5 × 1 minus model
Model
madelon
arcene
RELATHE
gisette
Mean
Development
Reported
TabPFN-3.5 × 8
0.970
0.959
0.962
0.999
0.972
−0.009
−0.001
TabPFN-Wide × 8
0.954
0.952
0.957
0.996
0.965
−0.005
+0.003
TabPFN-3.5 × 1
0.968
0.943
0.937
0.998
0.962
−0.001
+0.009
TabPFN-Wide × 1
0.939
0.949
0.959
0.996
0.960
−0.003
+0.013
TabPFNv2 × 8
0.961
0.913
0.958
0.998
0.957
+0.001
+0.009
Appendix
Table 12: AUROC on the four development datasets, averaged over the folds of the HDLSS benchmark, and median AUROC difference between TabOmics v2.5 with one member and each model on these datasets (Development) and on the 46 reported tasks (Reported). Models are sorted by their mean AUROC over the four datasets. The highest value in each AUROC column is in bold.
MLOmics BRCA
Shamir breast
Gene
Folds
Gene
Folds
FOXC1
5
MIA
4
GABRP
5
KLK8
4
SRSF12
4
PPP1R14C
4
KRT5
4
STAC2
3
FOXA1
3
NDC80
3
Appendix
Table 13: Genes most frequently ranked among the top 10 by TabOmics v2.5 ×1 in the breast cancer tasks. For each task, the table reports the 10 genes with the highest selection frequency across the five outer folds. Ties are resolved by lower mean rank. PAM50 genes are shown in bold.
Biomedical tables often combine thousands of measured variables with only tens or hundreds of labelled samples, a regime that is poorly represented in general-purpose tabular benchmarks. We introduce TabBench-Bio, a living and interactive benchmark of 43 biomedical datasets spanning multiple domains. Under a shared cross-validation protocol, we compare classical estimators, neural networks, and tabular foundation models across 28 feature-by-sample operating points. At the reference cell of 10,000 features and 100 training samples, RealTabPFN 2.5 has the highest point estimate, closely followed by TabPFN 3 and Logistic Regression. A paired bootstrap over the target pool separates RealTabPFN 2.5 from TabPFN 3 by 87 Elo (95% interval [43, 129]). Tabular foundation models generally occupy the leading ranks, while the strongest configuration depends on the operating point and biomedical modality. The AutoML framework AutoGluon, using its one-hour "extreme" preset, is configured as a separate resource-intensive reference and reported here at the reference and full cell. Fold-level predictions, run status, and deterministic aggregations make every reported result reproducible and reusable. The benchmark is open to contributions of new biomedical datasets. The interactive leaderboard is available at https://tabbench-bio.eu.
Jules Kreuer, Sofiane Ouaari, Julia Hellmig +2
Methods in Medical Informatics, University of Tübingen, Tübingen, Germany
While traditional tree-based ensemble methods have long dominated tabular tasks, deep neural networks and emerging foundation models have challenged this primacy, yet no consensus exists on a universally superior paradigm. Existing benchmarks typically contain fewer than 100 datasets, raising concerns about evaluation sufficiency and potential selection biases. To address these limitations, we introduce OmniTabBench, the largest tabular benchmark to date, comprising 3030 datasets spanning diverse tasks that are comprehensively collected from diverse sources and categorized by industry using large language models. We conduct an unprecedented large-scale empirical evaluation of state-of-the-art models from all model families on OmniTabBench, confirming the absence of a dominant winner. Furthermore, through a decoupled metafeature analysis, which examines individual properties such as dataset size, feature types, feature and target skewness/kurtosis, we elucidate conditions favoring specific model categories, providing clearer, more actionable guidance than prior compound-metric studies.
Dihong Jiang, Ruoqi Cao, Zhiyuan Dang +8
Huawei Cloud · School of Intelligence Science and Technology, Peking University
Tabular foundation models (TFMs) have recently emerged as a promising paradigm for machine learning on tabular data, offering the ability to generalize across datasets without task-specific training. Since many machine learning datasets can be represented as tables, this raises the question: does TFM capability extend beyond tasks traditionally regarded as tabular? We address this question by using TabPFN v3 on three non-tabular classification problems: handwritten digit recognition on MNIST, language identification of French and German words, and image classification on Tiny ImageNet. In each case, the original data are represented as rows of a table and classification is formulated as prediction of a missing label. We evaluate performance as a function of the number of context samples provided to the pretrained model, with no additional training or fine-tuning. Despite having no explicit access to the spatial or sequential structure characterizing the data, TabPFN v3 in some cases achieves accuracies comparable with that of models or methods geared specifically toward the corresponding tasks.
Goran Nakerst, John Brennan, Wouter Beugeling +1
Institut für Theoretische Physik, Technische Universität Dresden, 01062 Dresden, Germany · Department of Physics, Maynooth University, Ireland · Physikalisches Institut (EP3) and Institute for Topological Insulators, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany