TabOmics: A Small, Open Tabular Foundation Model for Wide Omics Data
Organizations: Department of Signal Theory and Communications, Telematics and Computing, Universidad Rey Juan Carlos, Fuenlabrada, Spain · Department of Signal Theory and Communications, Universidad Miguel Hernández, Elche, Spain
Abstract
Omics studies often produce tables with thousands to tens of thousands of molecular features and few samples. Tabular foundation models use diverse strategies to predict labels for such tables, with some relying on in-context learning, while approaches to wide tables may involve subsampling, grouping, or compressing features. We present TabOmics, a small, open family of tabular foundation models with 7.3M parameters, pretrained exclusively on synthetic tables. TabOmics processes all features retained after preprocessing jointly and derives predictions, attention-derived feature-importance scores, and test sample embeddings from the same forward pass. On 46 classification tasks from three benchmarks of high-dimensional, low-sample-size tables, TabOmics v2.5 with eight ensemble members achieves the highest mean area under the receiver operating characteristic curve (AUROC) among the evaluated models (0.943), outperforming TabPFN-3.5 with eight members on 32 tasks despite having about 30 times fewer parameters. With a single member, TabOmics v2.5 reaches a mean AUROC of 0.941, versus 0.942 for TabPFN-3.5 with eight members, with no significant paired difference under the reported task-level analysis. On synthetic tables, its feature-importance scores achieve feature recovery similar to that of TabPFN-Wide, and its predictions remain above chance at 30,000 features. On breast cancer tasks, the top ranked genes include PAM50 genes, which define the subtype labels, indicating concordance with the label definition rather than independent biological validation. The code, model weights, training recipes, and experimental artifacts are publicly available.
Figures & tables
| Benchmark | Dataset / Task group | Samples | Features | Classes | Min. class | CV |
|---|---|---|---|---|---|---|
| HDLSS | ALLAML | 72 | 7,129 | 2 | 25 | |
| BASEHOCK | 1,993 | 4,862 | 2 | 994 | ||
| CLL_SUB_111 | 111 | 11,340 | 3 | 11 | ||
| GLI_85 | 85 | 22,283 | 2 | 26 | ||
| PCMAC | 1,943 | 3,289 | 2 | 961 | ||
| Prostate_GE | 102 | 5,966 | 2 | 50 |
| Model | Stage | Features | Rows | Widened | Learning rate | Epochs | Steps | Tables |
|---|---|---|---|---|---|---|---|---|
| v2 | 1. Base | 2–100 | 40–1,024 | – | 100 | 100k | 0.80M | |
| 2. Longer tables | 2–100 | 40–2,048 | – | 100 | 100k | 0.80M | ||
| 3. Widening | 2–350 | 40–1,024 | 0.3 | 30 | 3.75k | 0.03M | ||
| 4. Low learning rate | 2–350 | 40–1,024 | 0.3 | 30 | 3.75k | 0.03M | ||
| Total | 207.5k | 1.66M | ||||||
| v2.5 | 1. Base | 2–100 | 40–1,024 | – | 300 | 300k | 2.40M |
| Model | Parameters | Tables | Pretraining compute |
|---|---|---|---|
| TabOmics v2 | 7.3M | 1.66M | 2.1 A100 GPU-days |
| TabOmics v2.5 | 7.3M | 2.46M | 2.4 A100 GPU-days |
| TabPFNv2 ( Hollmann et al., 2025 ) | 7.2M a | 130M | 2 weeks on 8 RTX 2080 Ti GPUs |
| TabICLv2 ( Qu et al., 2026 ) | 27.6M a | 35M | 24.5 H100 GPU-days |
| TabPFN-3.5 ( Jäger et al., 2026 ) | 220M | Not reported | Not reported |
| Mean AUROC | TabOmics v2.5 1 vs. | TabOmics v2.5 8 vs. | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | HDLSS | MLOmics | Shamir | All | Rank | W–L | W–L | ||
| TabOmics v2.5 8 | 0.946 | 0.938 | 0.947 | 0.943 | 2.76 | – | – | – | – |
| TabPFN-3.5 8 | 0.948 | 0.938 | 0.945 | 0.942 | 3.89 | 20–26 | 0.410 | 32–14 | 0.034 |
| TabOmics v2 8 | 0.944 | 0.936 | 0.947 | 0.941 | 4.13 | – | – | – | – |
| TabOmics v2.5 1 | 0.945 | 0.935 | 0.947 | 0.941 | 4.60 | – | – | – | – |
| TabOmics v2 1 | 0.943 | 0.934 | 0.945 | 0.939 | 5.35 | – | – | – | – |
| Seconds per fold | GPU memory (GB) | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | AUROC | HDLSS | MLOmics | Shamir | All | Max | Median | Max |
| TabOmics v2.5 8 | 0.943 | 10.9 | 87.3 | 31.8 | 65.2 | 1,506 | 3.1 | 27.7 |
| TabPFN-3.5 8 | 0.942 | 2.4 | 3.2 | 3.2 | 3.1 | 16 | 1.6 | 6.4 |
| TabOmics v2 8 | 0.941 | 11.0 | 87.5 | 32.0 | 65.3 | 1,506 | 3.1 | 27.7 |
| TabOmics v2.5 1 | 0.941 | 4.7 | 20.3 | 8.1 | 15.5 | 217 | 3.1 | 27.7 |
| TabOmics v2 1 | 0.939 | 4.7 | 20.5 | 8.1 | 15.6 | 217 | 3.1 | 27.7 |
| AUROC | Accuracy | |||||||
|---|---|---|---|---|---|---|---|---|
| TabOmics v2.5 | Mean | W–T–L | Mean | W–T–L | Seconds | GPU (GB) | ||
| One member (reference) | 0.9406 | – | – | 0.8277 | – | – | 15.5 | 3.1 |
| Feature clip 100 | 0.9405 | 16–10–20 | 0.247 | 0.8276 | 7–29–10 | 1.000 | – | – |
| Single precision | 0.9407 | 24–4–18 | 0.837 | 0.8279 | 10–25–11 | 1.000 | 56.0 | 6.0 |
| Two members | 0.9419 | 34–1–11 | 0.001 | 0.8294 | 27–5–14 | 0.341 | 21.3 | 3.1 |
| Four members | 0.9421 | 34–1–11 | 0.001 | 0.8299 | 28–2–16 | 0.341 | 36.7 | 3.1 |
| AUROC of the top genes | Stability | PAM50 in top 10 | ||||||
|---|---|---|---|---|---|---|---|---|
| Method | Jaccard | Spearman | MLOmics | Shamir | ||||
| TabOmics v2.5 1 | 0.926 | 0.955 | 0.954 | 0.962 | 0.34 | 0.75 | 4.8 | 4.2 |
| TabPFN-Wide 1 | 0.929 | 0.956 | 0.957 | 0.965 | 0.38 | 0.77 | 4.2 | 4.4 |
| Random forest | 0.920 | 0.949 | 0.953 | 0.959 | 0.14 | 0.08 | 2.0 | 1.6 |
| ANOVA F | 0.897 | 0.927 | 0.939 | 0.961 | 0.61 | 0.92 | 4.0 | 0.2 |
| Random genes | 0.782 | 0.884 | 0.911 | 0.945 | – | – | – | – |
| Logistic regression | Nearest neighbors | Direct | |||||
|---|---|---|---|---|---|---|---|
| Representation | AUROC | W–L | AUROC | W–L | AUROC | ||
| TabOmics v2.5 1 | 0.967 | – | – | 0.952 | – | – | 0.972 |
| TabOmics v2.5 8 | 0.967 | 3–5 | 0.742 | 0.951 | 5–2 | 0.297 | 0.973 |
| TabPFN-Wide 1 | 0.963 | 7–1 | 0.445 | 0.948 | 6–2 | 0.297 | 0.963 |
| TabPFNv2 1 | 0.950 | 7–1 | 0.156 | 0.932 | 7–1 | 0.164 | 0.937 |
| All genes | 0.957 | 6–2 | 0.445 | 0.892 | 8–0 | 0.047 | – |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Mean accuracy | TabOmics v2.5 1 vs. | TabOmics v2.5 8 vs. | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | HDLSS | MLOmics | Shamir | All | Rank | W–L | W–L | ||
| TabOmics v2.5 8 | 0.882 | 0.827 | 0.795 | 0.832 | 3.53 | – | – | – | – |
| TabPFN-3.5 8 | 0.896 | 0.820 | 0.790 | 0.830 | 3.62 | 19–27 | 0.585 | 24–22 | 0.365 |
| TabOmics v2 8 | 0.880 | 0.822 | 0.794 | 0.829 | 4.13 | – | – | – | – |
| TabOmics v2.5 1 | 0.881 | 0.819 | 0.795 | 0.828 | 4.75 | – | – | – | – |
| TabOmics v2 1 | 0.877 | 0.818 | 0.792 | 0.825 | 5.71 | – | – | – | – |
| Task | TabOmics v2.5 | TabOmics v2 | TabPFN-3.5 | TabPFN-Wide | ||||
|---|---|---|---|---|---|---|---|---|
| 1 | 8 | 1 | 8 | 1 | 8 | 1 | 8 | |
| HDLSS | ||||||||
| ALLAML | 0.998 | 0.998 | 0.998 | 0.998 | 0.989 | 0.996 | 0.996 | 0.997 |
| BASEHOCK | 0.998 | 0.999 | 0.996 | 0.998 | 0.974 | 0.999 | 0.999 | 0.999 |
| CLL_SUB_111 | 0.897 | 0.898 | 0.893 | 0.896 | 0.821 | 0.891 | 0.875 | 0.883 |
| GLI_85 | 0.944 | 0.945 | 0.939 | 0.945 | 0.944 | 0.951 | 0.945 | 0.947 |
| Task | TabPFNv2 | TabICLv2 | XGBoost | Random forest | Elastic-net LR | ||
|---|---|---|---|---|---|---|---|
| 1 | 8 | 1 | 8 | ||||
| HDLSS | |||||||
| ALLAML | 0.998 | 0.997 | 0.955 | 0.991 | 0.963 | 0.995 | 0.993 |
| BASEHOCK | 0.999 | 0.999 | 0.991 | 0.997 | 0.991 | 0.996 | 0.997 |
| CLL_SUB_111 | 0.778 | 0.896 | 0.721 | 0.774 | 0.873 | 0.854 | 0.885 |
| GLI_85 | 0.921 | 0.934 | 0.928 | 0.916 | 0.901 | 0.938 | 0.934 |
| AUROC | TabOmics v2.5 1 minus model | ||||||
|---|---|---|---|---|---|---|---|
| Model | madelon | arcene | RELATHE | gisette | Mean | Development | Reported |
| TabPFN-3.5 8 | 0.970 | 0.959 | 0.962 | 0.999 | 0.972 | ||
| TabPFN-Wide 8 | 0.954 | 0.952 | 0.957 | 0.996 | 0.965 | ||
| TabPFN-3.5 1 | 0.968 | 0.943 | 0.937 | 0.998 | 0.962 | ||
| TabPFN-Wide 1 | 0.939 | 0.949 | 0.959 | 0.996 | 0.960 | ||
| TabPFNv2 8 | 0.961 | 0.913 | 0.958 | 0.998 | 0.957 | ||
| MLOmics BRCA | Shamir breast | ||
|---|---|---|---|
| Gene | Folds | Gene | Folds |
| FOXC1 | 5 | MIA | 4 |
| GABRP | 5 | KLK8 | 4 |
| SRSF12 | 4 | PPP1R14C | 4 |
| KRT5 | 4 | STAC2 | 3 |
| FOXA1 | 3 | NDC80 | 3 |
Tabular Foundation Models
Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.
See all 209 papers