Tabular Foundation Models Can Learn Association Rules
Authors: Erkan Karabulut, Daniel Daza, Paul Groth, Martijn C. Schut, Victoria Degeler
Organizations: University of Amsterdam, The Netherlands · Vrije Universiteit Amsterdam, The Netherlands · Amsterdam UMC, location VUmc, Vrije Universiteit Amsterdam, the Netherlands
Association Rule Mining (ARM) is a fundamental task for knowledge discovery in tabular data and is widely used in high-stakes decision-making. Classical ARM methods rely on frequent itemset mining, leading to rule explosion and poor scalability, while recent neural approaches mitigate these issues but suffer from degraded performance in low-data regimes. Tabular foundation models (TFMs), pretrained on diverse tabular data with strong in-context generalization, provide a basis for addressing these limitations. We introduce a model-agnostic association rule learning framework that extracts association rules from any conditional probabilistic model over tabular data, enabling us to leverage TFMs. We then introduce TabProbe, an instantiation of our framework that utilizes TFMs as conditional probability estimators to learn association rules out-of-the-box without frequent itemset mining. We evaluate our approach on tabular datasets of varying sizes based on standard ARM rule quality metrics and downstream classification performance. The results show that TFMs consistently produce concise, high-quality association rules with strong predictive performance and remain robust in low-data settings without task-specific training. Source code is available at https://github.com/DiTEC-project/tabprobe.
Figures & tables
Age
Sex
Steroid
Antiviral
⋯
Class
30
female
no
yes
⋯
LIVE
50
male
no
yes
⋯
LIVE
78
male
yes
yes
⋯
LIVE
39
male
yes
no
⋯
DIE
Rule 1
Antiviral = yes ∧ Malaise = yes → Class = LIVE
Rule 2
Malaise = yes ∧ Anorexia = yes → Class = LIVE
Table 1: Sample clinical dataset and association rules. The table shows sample patient records from a hepatitis dataset with a target outcome. Listed association rules are extracted from TabPFN using our approach and capture interpretable patterns relating symptoms and treatments to patient survival.
Figure 1: Example association rule learning from tabular foundation models using Algorithm 2 : (1) split table into features (X) and labels (Y), (2) fit TFM on X and Y, (3) construct probing matrix with marked items (probability 1) and uniform priors for other items, (4) predict per feature, (5) aggregate predictions into reconstruction matrix, (6) apply thresholding to extract association rules.
Algorithm
# Rules
Support
Confidence
Zhang’s metric
Interestingness
Coverage
Small Tabular Datasets
TabICL
378
38
84.3
31.8
42.6
98.1
TabPFNv2.5
286
38.9
86.3
30.3
43.2
98.1
TabDPT
294
36.6
82.9
26.2
40.9
1
Aerial+
313
28.7
89.3
23.5
34.4
88.1
FP-G (0.3)
430
42.4
89.8
23.6
48.6
99.7
Table 2: Averages of rule quality metrics in percentage across 5 small and 5 larger datasets. Tabular foundation models can learn a concise number of rules with higher association strength, interestingness on average, and full data coverage, with slightly lower confidence scores (FP-G (x): refers to FP-Growth with x being min. support threshold).
Method
# Rules
Accuracy
F1 score
Precision
Recall
Small Tabular Datasets
TabICL
616
82.93
81.07
81.13
82.93
TabDPT
641
82.22
80.35
80.58
82.22
TabPFNv2.5
657
84.47
82.75
83.02
84.47
Aerial+
579
80.69
77.35
76.51
80.69
FP-G (0.3)
266
79.72
76.45
75.84
79.72
Table 3: Predictive performance in percentage and rule (CBA) or itemset (CORELS) size given in column two. TFMs achieve better predictive performance with a concise number of rules in small tabular data, while being behind FP-G by only -1.5% despite FP-G’s rule explosion (FP-G (x) denotes FP-Growth with minimum support threshold x ).
Figure 2: Mean execution time on an 8,124-rows dataset (mushroom) as itemset universe size increases. TFMs learn association rules within a minute on modest hardware for small-to-medium tables, whereas only Aerial+ scales to larger sizes (FP-Growth (x): x is min. support threshold).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Columns
Items
Rows
Small Tabular Datasets
Acute Inflammations
6
22
120
Hepatitis
19
89
155
Cervical Cancer
19
197
72
ASD Screening (Adolescent)
21
95
104
Fertility
9
36
100
Appendix
Table 4: Small and larger tabular datasets from the UCI ML repository used in all the experimental settings.
Seeds
Values
Seeds 1–4
42, 1608637542, 1273642419, 1935803228
Seeds 5–7
787846414, 996406378, 1201263687
Seeds 8–10
423734972, 415968276, 670094950
Appendix
Table 5: Fixed execution seeds used in all experimental settings for reproducibility of the results.
Table 6: Parameter settings for TFMs and baselines across experimental settings. a is antecedent length. † τa=0.5 for Setting 1, τa=0.3 for Setting 2. ‡ min_supp ∈{0.3,0.2,0.1} for Setting 1, ∈{0.3,0.2,0.1,0.05,0.01} for Setting 2.
Dataset
Batches
Layer Dimensions
Epochs
Small Tabular Datasets
Acute Inflammations
2
4
10
Hepatitis
2
4
10
Cervical Cancer
18
2
20
ASD Screen. (Adolescent)
2
4
10
Fertility
2
4
10
Appendix
Table 7: Optimized Aerial+ architecture and training parameters per dataset; all other parameters kept at default values.
Algorithm
# Rules
Support
Confidence
Zhang’s metric
Interestingness
Coverage
Acute Inflammations
TabICL
46.1
32.9
80.7
57.8
47.3
1
TabPFNv2.5
51.9
31.8
81.2
60.9
46.7
1
TabDPT
52.5
32.4
80.8
60.8
47.5
1
Aerial+
138.2
25.1
87.7
46.3
39.4
1
FP-G(0.3)
46
37.7
92.4
62.1
58.8
1
Appendix
Table 8: Rule quality evaluation of TFMs and baselines on small tabular datasets. In small tabular data, algorithmic ARM (FP-G) still leads to rule explosion. TFMs and Aerial+ can address rule explosion while TFMs find rules of higher statistical significance (Zhang and Interestingness). FP-G(x) refers to FP-Growth with x being min support threshold.
Algorithm
# Rules
Support
Confidence
Zhang’s metric
Interestingness
Coverage
Breast Cancer
TabICL
24.8
49.5
88.7
60.4
58.2
91.2
TabPFNv2.5
27.9
47.4
87.9
57.4
55.6
95.2
TabDPT
27
47.5
88.2
58.5
55.8
91.7
Aerial+
15.7
43.9
88.1
36.4
52.4
90.2
FP-G(0.3)
51
44.4
87.8
32.1
51.4
94.4
Appendix
Table 9: Rule quality evaluation of TFMs and baselines on larger tabular datasets. TFMs can reach full data coverage with up to 12 times fewer rules (Spambase), addressing rule explosion, and lead to rules of higher statistical significance (Zhang and Interestingness). FP-G(x) refers to FP-Growth with x being min support threshold.
Figure 3: Experimental Setting 2 for downstream task evaluation of TFMs and baselines. (1) tabular data is split into training as test folds for cross validation, (2) TFMs and baselines are run on the training fold to learn rules (CBA) or frequent itemsets (CORELS work with frequent itemsets and not rules), (3) learned rules and itemsets are passed to CBA and CORELS to build a rule-based classifier, (4) rule-based classifiers are evaluated on the test fold. This process is repeated 5-fold across 10 random seeds.
Figure 4: Classification tasks results with CORELS on small tabular data. TFMs have better or compatible ( ±1% ) predictive performance on 4 out of 5 small datasets, except the Cervical Cancer dataset. This shows that the small number of rules that TFMs learn is indeed more generalizable than the baselines on unseen data.
Figure 5: Classification task results with CORELS on large tabular data. On large datasets, TFMs have achieved better or comparable performance on all datasets compared to the baselines.
Figure 6: Classification tasks results with CBA on small tabular data. TFMs have better or compatible predictive performance than the baselines in 4 out of 5 datasets, except the Autism Screening dataset, while utilizing a small number of rules.
Figure 7: Classification tasks results with CBA on large tabular data. TFMs resulted in better or compatible predictive performance in only 2 out of the 5 large datasets with CBA. Note that CBA prioritizes rules of higher support and confidence rather than rules of higher association strength and generalizability.
Figure 8: Hyperparameter analysis of Algorithm 2 over all 10 datasets. The left y-axis specifies rule counts, and the right y-axis specifies average rule quality scores. The first row shows that increasing τa (while keeping τc=0.8 fixed) leads to a smaller number of rules, higher support and interestingness, while the association strength (Zhang) and confidence scores do not change significantly. The second row shows that increasing τc (while keeping τa=0.5 fixed) leads to higher support, confidence, association strength, and interestingness scores and reduces the number of rules learned.
Tabular foundation models (TFMs) have recently emerged as a promising paradigm for machine learning on tabular data, offering the ability to generalize across datasets without task-specific training. Since many machine learning datasets can be represented as tables, this raises the question: does TFM capability extend beyond tasks traditionally regarded as tabular? We address this question by using TabPFN v3 on three non-tabular classification problems: handwritten digit recognition on MNIST, language identification of French and German words, and image classification on Tiny ImageNet. In each case, the original data are represented as rows of a table and classification is formulated as prediction of a missing label. We evaluate performance as a function of the number of context samples provided to the pretrained model, with no additional training or fine-tuning. Despite having no explicit access to the spatial or sequential structure characterizing the data, TabPFN v3 in some cases achieves accuracies comparable with that of models or methods geared specifically toward the corresponding tasks.
Goran Nakerst, John Brennan, Wouter Beugeling +1
Institut für Theoretische Physik, Technische Universität Dresden, 01062 Dresden, Germany · Department of Physics, Maynooth University, Ireland · Physikalisches Institut (EP3) and Institute for Topological Insulators, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany
Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).
Tabular foundation models (TFMs) are becoming a powerful alternative to conventional tabular learning, adapting to new datasets in context without task-specific training. Current architectures, however, face a trade-off: models that preserve cell-level representations are expensive because they repeatedly attend across rows and columns, while faster models collapse each row into a fixed embedding and lose this fine-grained state. We introduce Factorized Tabular Foundation Models (FTFM), which preserve cell representations while factorizing their computation through one contextual latent per row and one per column. Cells no longer attend to one another directly; instead, each cell is updated through the interaction of its row and column latents, reducing repeated attention while retaining feature-specific information. FTFM achieves competitive predictive performance in tabular classification on major benchmarks such as TabArena and BeyondArena.