We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the p(y∣x,Dcontext) objective of conventional tabular PFNs, it is designed around learning p(x,y∣Dcontext), a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
Figures & tables
Figure 1: Performance overview on evaluated benchmarks. LimiX-2 achieves Elo scores of 1935 , 1506 , and 1432 on TabArena ( Erickson et al., 2025 ) , TALENT ( Ye et al., 2024 ; Liu et al., 2024 ) , and BCCO ( LimiX Team, 2025 ) , respectively, outperforming all compared foundation models and AutoGluon 1.6 ( Erickson et al., 2020 ) . For TabArena, only the best AutoGluon setting is shown. Bar ends and labels indicate Elo point estimates, while shaded regions with terminal markers show 95% bootstrap confidence intervals.
Figure 2: Overall structure of LimiX-2. Features are encoded by a MLP-based encoder, and missing cell is imputed by a shared learnable vector. Targets are encoded into K=4 embedding slots. Each block successively applies sample-axis attention and SwiGLU on the feature and target representations, followed by asymmetric feature-axis attention. Feature embeddings produced at the shallow depth are used for masked-feature reconstruction. Final-layer task embeddings are used for both classification and regression tasks.
Figure 3: Schematic overview of the synthetic data generation process for pretraining. The pipeline consists of five phases, that are hyperparameter sampling, directed acyclic graph (DAG) generation, SCM propagation, data sampling and task adaptation respectively.
Benchmark
# Datasets
Task Type
# Tasks
Main Metrics
TabArena
51
Binary CLS
30
Elo, Rank, Win Rate
Multiclass CLS
8
REG
13
TALENT
288
Binary CLS
120
Multiclass CLS
68
REG
100
Table 1: Task composition and evaluation metrics of the 3 tabular benchmarks. Twelve TALENT datasets with more than 10 target classes are excluded.
Figure 4: Performance on the TabArena benchmark. Baseline results are reported under the default, tuned, and tuned-plus-ensembled configurations. LimiX-2 with default configuration achieves an Elo score of 1935 , outperforming all compared foundation models, including TabFM, TabPFN-3, and EXAONE Tabular. LimiX-2 also surpasses AutoGluon under its noncommercial 4h configuration.
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
LimiX-2 (D)
1935 -77,+111
3.3%
5.5
18.9
TabM (D)
1283 -42,+43
18.9%
47.7
0.1
TabFM+
1818 -84,+109
6.2%
9.0
5.3
iLTM (T)
1280 -32,+35
19.0%
47.9
0.1
Causilo (D)
1790 -57,+92
8.9%
10.1
1.7
BetaTabPFN (D)
1272 -57,+58
20.3%
48.8
0.0
AutoGluon 1.6 (NC, 4h)
1789 -60,+99
8.6%
10.1
1.2
TabPFNv2 (T)
1272 -57,+57
20.0%
48.8
0.1
TabFM (D)
1774 -98,+101
6.5%
10.7
5.9
SAP-RPT-OSS (D)
1268 -58,+57
20.2%
49.1
0.6
Mitra-v2 (D)
1769 -74,+102
8.3%
10.9
3.2
TorchMLP (T+E)
1268 -45,+46
19.0%
49.1
0.0
Table 2: Performance on the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 1935 (117.4 points above TabFM+ before rounding). Compared with TabFM+, its improvability is 3.3% versus 6.2%, its average rank is 5.5 versus 9.0, and its aggregated win count is 18.9 versus 5.3.
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
LimiX-2 (D)
1917 -77,+127
4.3%
6.0
10.6
TabPFNv2 (T)
1304 -77,+69
22.9%
46.7
0.1
TabFM+
1796 -86,+121
7.5%
9.9
4.7
TabM (D)
1304 -43,+52
21.5%
46.7
0.1
Causilo (D)
1768 -70,+90
11.0%
11.0
1.3
EBM (T+E)
1301 -36,+38
22.0%
47.1
0.0
AutoGluon 1.6 (NC, 4h)
1761 -53,+88
10.5%
11.4
1.1
TorchMLP (T+E)
1297 -39,+41
21.2%
47.5
0.0
TabFM (D)
1760 -114,+125
7.5%
11.4
5.6
SAP-RPT-OSS (D)
1291 -71,+53
22.8%
48.1
0.5
Mitra-v2 (D)
1754 -85,+126
10.1%
11.6
2.9
iLTM (T)
1285 -38,+42
21.7%
48.7
0.1
Table 3: Performance on classification tasks in the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 1917 (121.1 points above TabFM+ before rounding). Its improvability is 4.3%, approximately 42.7% lower than that of TabFM+ (7.5%). Its average rank is 6.0 versus 9.9, and its aggregated win count is 10.6 versus 4.7.
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
Model
Elo ↑
Improv- ability ↓
Avg. rank ↓
#wins ↑
LimiX-2 (D)
2206 -188,+318
0.6%
3.8
8.3
TabM (D)
1274 -94,+114
11.3%
48.7
0.0
TabFM+
2063 -139,+212
2.6%
6.4
0.5
ModernNCA (D)
1268 -73,+81
12.5%
49.2
0.0
AutoGluon 1.6 (NC, 4h)
2060 -141,+193
3.1%
6.5
0.1
Mitra (D)
1264 -112,+127
12.1%
49.5
0.0
Causilo (D)
2027 -131,+237
2.9%
7.3
0.3
SAP-RPT-OSS (D)
1260 -147,+158
12.6%
49.8
0.1
AutoGluon 1.6 (EX, 4h)
2002 -105,+165
3.5%
7.9
0.4
LimiX-16M (D)
1247 -156,+177
12.2%
50.8
0.0
TabFM (D)
1976 -96,+176
3.7%
8.6
0.4
TabPFNv2 (T)
1233 -129,+146
11.6%
51.9
0.0
Table 4: Performance on regression tasks in the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 2206, exceeding TabFM+ and AutoGluon 1.6 (noncommercial, 4h) by 143.2 and 146.3 points, respectively, before rounding. Compared with those two methods, respectively, its improvability is 0.6% versus 2.6% and 3.1%, its average rank is 3.8 versus 6.4 and 6.5, and its aggregated win count is 8.3 versus 0.5 and 0.1.
Figure 5: Pairwise win rates on the TabArena benchmark. Entry (i,j) is the pairwise win rate of method i against method j , computed within each dataset and then averaged equally across datasets, with ties counted as half a win. All off-diagonal entries in the row corresponding to LimiX-2 exceed 60%.
Figure 6: Average-rank comparison on the TALENT benchmark across binary classification, multiclass classification, and regression tasks. Each bar represents a method’s mean rank over datasets in the corresponding task, using accuracy for classification and RMSE for regression. Error bars denote 95% bootstrap CIs from 1000 dataset resamples. For consistency with Table 5 , methods are displayed in the same order across all panels. LimiX-2 is highlighted in pink, while baseline models are shown in dark blue. LimiX-2 achieves the lowest average rank in all three task categories, with ranks of 4.62, 3.91, and 3.88 for binary classification, multiclass classification, and regression, respectively.
Figure 7: Pairwise win rates on the TALENT benchmark. Entry (i,j) is the percentage of evaluation tasks on which method i beats method j . Cells range from purple (row loses) to white (tie) to green (row wins). LimiX-2 achieves a win rate above 50% against every competing method, ranging from 57% against TabFM to 96% against TabM.
Figure 8: Model rankings across dataset meta-features on TALENT. Lines show median piecewise-linear fits over 300 bootstrap rounds, with 10th–90th percentile bands ( Qu et al., 2026 ) . LimiX-2 achieves among the lowest fitted ranks across most sample-size and feature-count ranges, with particularly strong performance on medium-to-large datasets. These results show that its strong aggregate performance remains robust across datasets with varying scales and dimensionalities.
Figure 9: Average-rank comparison on the BCCO benchmark across binary classification, multiclass classification, and regression tasks. Each bar represents a method’s mean rank over datasets in the corresponding task, using accuracy for classification and RMSE for regression. Error bars denote 95% bootstrap CIs from 1000 dataset resamples. For consistency with Table 6 , methods are shown in the same order across all panels. LimiX-2 is highlighted in pink, and baseline models are shown in dark blue. LimiX-2 achieves a low average rank in all three task categories, with ranks of 5.45, 5.17, and 3.10 for binary classification, multiclass classification, and regression, respectively.
Figure 10: Pairwise win rates on the BCCO benchmark. Entry (i,j) is the percentage of evaluation tasks on which method i beats method j . Cells range from purple (row loses) to white (tie) to green (row wins). LimiX-2 wins the majority of tasks against every compared method, achieving win rates of 65% against TabFM and 67% against EXAONE Tabular. Its win rates reach 73–79% against the remaining tabular foundation models, including 79% against LimiX-16M, and 81–94% against tree-based and neural-network baselines, demonstrating broad predictive advantages across the benchmark.
Figure 11: Model rankings as a function of meta-features on the BCCO benchmark. Each curve is a piecewise-linear fit. The line is the median over 300 bootstrap rounds and the band is the 10th–90th percentile ( Qu et al., 2026 ) . Lower is better.
Evaluation task
α
β
R2
RMSE
TabArena
1863.88
34.68
0.9808
8.31
TALENT classification
1427.25
22.16
0.9792
5.53
TALENT regression
1545.12
18.26
0.9680
5.69
BCCO classification
1295.86
11.24
0.9617
3.84
BCCO regression
1795.89
30.06
0.9702
9.03
Table 8: Log-linear scaling fits. α is the fitted Elo at 100M parameters, and β is the Elo gained per parameter doubling.
Figure 12: Parameter scaling on TabArena. The solid line connects the observed LimiX-2 scale points, while the dashed line shows the log-linear OLS fit extrapolated to 2B parameters.
Figure 13: Parameter scaling on TALENT classification and regression.
Figure 14: Parameter scaling on BCCO classification and regression.
We introduce TabPFN-CFM, a causal foundation model that can handle multiple causal problems. TabPFN-CFM predicts both causal structure and outcomes from observational data, supports queries on all three levels of Pearl's Causal Hierarchy and uses known graph structure when available to improve predictions. TabPFN-CFM is trained on synthetic datasets, and generalises to real datasets, demonstrating improved performance over both structural and outcome prediction baselines.
Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.
Christopher Stith, Hossein Rahmani, Jesse C. Cresswell
Causal discovery aims to recover directed causal relations from observational and interventional data, providing a basis for mechanistic understanding and reliable decision-making. Causal discovery foundation models (CDFMs) seek to amortize this problem by mapping a dataset directly to a causal graph in a single forward pass, avoiding per-dataset testing, search, or optimization. However, existing CDFMs remain limited, often failing to consistently match strong classical methods, and we find that a key bottleneck is how causal pretraining tasks are constructed. Based on this observation, we propose TabCausal, a data-driven CDFM trained with broad causal pretraining over diverse graph priors, structural mechanisms, noise models, dimensions, sample sizes, and intervention regimes. A dynamic task construction strategy composes these causal environments into varied discovery tasks, enabling more transferable structural learning from observational and mixed-interventional data. On large-scale synthetic benchmarks, TabCausal achieves better macro-averaged performance than a diverse set of causal discovery baselines. To further bridge abstract synthetic generators and realistic causal reasoning scenarios, we introduce a protocol-guided and LLM-audited semantic causal environment benchmark, where domain-grounded SCMs generate interpretable observational and interventional datasets for out-of-distribution analysis. Across both synthetic and semantic environments, TabCausal demonstrates robust structure recovery, especially under interventional evidence, highlighting broad causal pretraining as a key ingredient for transferable amortized causal discovery.