Causilo Technical Report

Minyong Cho; Minho Jeong; Dooho Lee; Jinmo Lee; Jaemin Yoo

cs.LGSep 19, 2026

Causilo Technical Report

Authors: Minyong Cho, Minho Jeong, Dooho Lee, Jinmo Lee, Jaemin Yoo

Abstract

We introduce Causilo, a tabular foundation model (TFM) that combines frontier predictive performance with exceptionally fast inference. On TabArena, Causilo achieves 1785.4 Elo, at a median inference time of 0.10 seconds per 1K test samples. It outperforms TabPFN-3.5-Fast with 31.6% less inference time, placing it on the performance--efficiency Pareto frontier. Causilo follows TabICL's column-then-row architecture but introduces another row-refinement module before row compression. This module exchanges information among cell representations within each row after column encoding. The refined cells then visit the context set again through an additional column stage before being compressed into row embeddings. For inference efficiency, both row stages use cross-attention through a fixed number of summary tokens, keeping their attention cost linear in the number of features. Pretrained on approximately 36M synthetic tables, Causilo delivers strong benchmark results across TabArena, BeyondArena, and ScoringBench, achieving frontier-level performance with substantially faster inference.

Figures & tables

Figure 1: TabArena performance–efficiency frontier. The horizontal axis shows median inference time per 1K test samples on a logarithmic scale. Causilo achieves 1785 Elo at 0.1043 seconds, placing it on the Pareto frontier. Its classification and regression models contain 36.08M and 37.08M parameters, roughly one-sixth as many as TabPFN-3.5 (218.98M) and one-eleventh as many as LimiX-2 (406.2M), which also lie on the frontier.

Figure 2: Architecture of Causilo . For a single input view, column encoding, row refinement, and column refinement update cell tokens while retaining the table’s row and feature-group axes. Row compression then produces a fixed-width representation for each row, followed by dataset-level in-context prediction. The lower panels expose the architectural separation at the center of Causilo : temporary summaries mediate cell representation refinement, whereas an independent set of summaries performs the final readout.

Figure 3: Feature-wise forward-time scaling of tabular foundation models. Markers show the median of 50 runs, normalized by the measured time at 16 features. Lines show fits of

T(F)=c+aF^{\beta}

, normalized by the fitted time at 16 features; the annotated

\beta

values are estimated over the displayed feature range.

Operation	Complexity
Full self-attention over cell and summary tokens	$\mathcal{O}((G+K)^{2})$
Causilo : one refinement round	$\mathcal{O}(K(G+K))$
Causilo : final row-refinement broadcast	$\mathcal{O}(GK)$
Causilo : one compression layer	$\mathcal{O}(K(G+K))$

Table 1: Attention complexity per row and per head for

G

cell tokens and

K

summary tokens. With fixed

K

, all three operations in Causilo scale linearly with

G

Figure 4: TabArena classification performance–efficiency frontier. Higher Elo and lower median inference time per 1K test samples are better; the time axis is logarithmic. The dashed line traces the Pareto frontier among the plotted configurations. Causilo achieves 1766.3 Elo at 0.1096 seconds per 1K test samples.

Figure 5: TabArena regression performance–efficiency frontier. Higher Elo and lower median inference time per 1K test samples are better; the time axis is logarithmic. The dashed line traces the Pareto frontier among the plotted configurations. Causilo achieves 2019.3 Elo at 0.0930 seconds per 1K test samples.

Panel of Figure 6 — Figure 6: BeyondArena Elo of the top 12 models for classification and regression, excluding system submissions and retaining only the highest-rated variant from each model family within each comparison. Higher is better. Black denotes Causilo , gold denotes other foundation models, and blue denotes supervised methods.

Model	Elo $\uparrow$	Improv- ability $\downarrow$	Avg. rank $\downarrow$	#wins $\uparrow$	Model	Elo $\uparrow$	Improv- ability $\downarrow$	Avg. rank $\downarrow$	#wins $\uparrow$
Causilo	$\mathbf{1359}_{-40,+45}$	6.4%	6.2	47.3	TabM (D)	$1100_{-27,+26}$	15.5%	14.4	1.8
RealMLP (T+E)	$1274_{-26,+30}$	11.9%	8.5	7.1	RealMLP (D)	$1053_{-27,+22}$	17.4%	16.1	3.0
TabPFN-3 (D)	$1270_{-41,+43}$	10.0%	8.6	12.5	TabDPT (D)	$1047_{-47,+37}$	19.0%	16.3	5.5
TabPFN-2.6 (D)	$1216_{-37,+40}$	12.4%	10.4	9.4	ExtraTrees (T+E)	$1011_{-30,+22}$	19.6%	17.5	1.0
TabM (T+E)	$1211_{-27,+29}$	13.2%	10.5	3.2	XGBoost (D)	$1000_{-26,+31}$	19.0%	17.9	1.0
CatBoost (T+E)	$1205_{-22,+27}$	13.6%	10.7	4.8	LightGBM (D)	$988_{-21,+26}$	19.1%	18.3	0.8

Table 2: Combined BeyondArena results for 29 evaluated configurations, sorted by Elo. D denotes default, T denotes tuned, and T+E denotes tuned and ensembled configurations. Elo subscripts give the reported lower and upper confidence deviations. Wins are aggregated first-place counts and may be fractional. Best values are bold, and the Causilo row is shaded.

Figure 7: BeyondArena classification Elo by subgroup. Each panel shows the top five model families, retaining the highest-rated variant within each family. Dataset counts appear below the subgroup names. Black denotes Causilo , gold other TFMs, and blue conventional supervised models. Higher Elo is better.

Figure 8: BeyondArena regression Elo by subgroup, using the same model-family selection and colors as Figure 7 . Causilo leads five of the six displayed subgroups and is the highest-ranked TFM on grouped regression. Higher Elo is better.

Panel of Figure 9 — Figure 9: ScoringBench performance–efficiency frontiers for point prediction (RMSE, $R^{2}$ , and MAE) and distributional prediction (CRPS). Each panel plots metric-specific Elo against median inference time per test fold in seconds. Dashed lines trace the empirical Pareto frontier; breaks in the time axis separate the much slower Mitra-v2 configurations. Causilo lies on the frontier in all four panels.

Tabular Foundation Models

Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.

See all 188 papers

TabPFN-3.5: Technical Report
Sep 15, 2026Benjamin Jäger, Nick Erickson, Léo Grinsztajn +44Tabular Foundation Models Inference-Time Optimization
TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction
Aug 2, 2026Rasa Hosseinzadeh, Alex Labach, Zexin Xue +3Tabular Foundation Models Tabular ICL
Layer 6 AI, Toronto, Canada · Cohere, Toronto, Canada
TabPFN-3: Technical Report
May 13, 2026Léo Grinsztajn, Klemens Flöge, Oscar Key +38Tabular Foundation Models Efficient Neural Network Inference

Sep 15, 2026J/K move · Enter open · S save

TabPFN-3.5: Technical Report

Benjamin Jäger, Nick Erickson, Léo Grinsztajn, Felix Birkel+43

Tabular Foundation Models Inference-Time Optimization Tabular ICL Tabular Prediction

Open paper AskPDF arXiv

We introduce TabPFN-3.5, our new flagship Tabular Foundation Model. It significantly outperforms its predecessor, TabPFN-3, and all existing baselines across a broad range of tabular problems. TabPFN-3.5 sets a new state of the art on standard tabular prediction in TabArena, and extends it to the data practitioners encounter in practice: non-i.i.d. data with temporal or grouped splits, tables with strings, text and images, high-cardinality categorical features, and wide tables with many features. These gains carry over to our task-specific harnesses: state of the art on relational data and stronger time-series forecasting. For faster inference, our variant TabPFN-3.5-Fast runs up to 3x faster than TabPFN-3 while keeping most of the accuracy gains. In addition, we upgrade TabPFN-3.5-Plus, expanding our multimodal capabilities with advanced text and date handling alongside proprietary inference optimizations. Finally, we release a new version of our Thinking mode, TabPFN-3.5-Thinking, which scales inference-time computation to push the state of the art further. It benefits from our stronger base model and from inference-time improvements that make it up to 12x faster than TabPFN-3-Thinking.

Abstract

Tabular Foundation Models

Explore similar work

TabPFN-3.5: Technical Report

TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction

TabPFN-3: Technical Report