EXAONE Tabular 1.0 : Technical Report

Moonjung Eo; Min-Kook Suh; Hye-Seung Cho; Jiwon Kim; Seoyoon Kim; Sangjun Nam; Soonyoung Lee

cs.LGAug 26, 2026

EXAONE Tabular 1.0 : Technical Report

Authors: Moonjung Eo, Min-Kook Suh, Hye-Seung Cho, Jiwon Kim, Seoyoon Kim, Sangjun Nam, Soonyoung Lee

Organizations: LG AI Research

Abstract

EXAONE Tabular is a compact tabular foundation model family for classification and regression via in-context learning, producing predictions without dataset-specific gradient updates. Pretrained exclusively on a synthetic structural-causal-model (SCM) prior, its central contribution is an architecture-centered redesign of tabular in-context learning. Rather than compressing features into a fixed row embedding before a separate row-level learner, EXAONE Tabular interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens. Across four public benchmarks, EXAONE Tabular combines strong predictive performance with high efficiency. On TabArena, its 20.81M-parameter classification model ranks first overall, surpassing tuned ensembles and 4-hour AutoML pipelines, while regression reaches the performance regime of the 1.64B-parameter TabFM at roughly 1/11 the inference cost. On BCCO and TALENT, EXAONE Tabular ranks second in classification and first in regression. On ScoringBench, it achieves the best mean rank for both point-estimation and predictive-distribution quality, leading the $R^2$ , RMSE, and CRPS evaluations. Together, these results establish EXAONE Tabular as a state-of-the-art compact tabular foundation model family, combining strong predictive performance across classification, point regression, and probabilistic regression with an efficient model design.

Figures & tables

Figure 1: Performance–size Pareto front on the full TabArena benchmark (classification and regression). EXAONE Tabular lies on the Pareto front in its default setting and is the strongest compact tabular foundation model (

\leq

100M parameters), outperforming every model shown except TabFM — with which it is statistically tied at

\sim

1.3% of the parameters — by 113 to 488 Elo. Sizes count the weights stored in the evaluated checkpoints.

Component	Role
Cross-Axis Summary Transformer (CAST)	Interleaves feature-axis processing within items with support-conditioned item-axis processing within features; item-summary tokens join feature-axis sequences, while feature-summary tokens join support item-axis sequences.
Synthetic SCM prior	Generates causal, categorical, nonlinear, noisy, and missing-data prediction tasks without using real pretraining tables.
Task-specific prediction heads	Uses separate classification and regression heads for the independently trained task-specific checkpoints.
Training and inference pipeline	Applies task-specific optimization and inference procedures, including preprocessing, test-time ensembling, and chunked support-conditioned inference.

Table 1: Core components of EXAONE Tabular.

Figure 2: Overall framework of EXAONE Tabular. The model combines an SCM-based synthetic prior, a Cross-Axis Summary Transformer with interleaved feature-axis and item-axis attention, task-specific heads for classification and regression, and in-context inference on support and query sets. Item-summary tokens are concatenated with cell states along the feature axis, while feature-summary tokens are concatenated with support-item states along the item axis.

Configuration	Value
Embedding dimension	192
Attention heads	6
Transformer layers	12
Feed-forward expansion	$4\times$
Feature-attention operations per layer	2
Item-summary tokens per item	3

Table 2: Core model configuration.

Panel of Figure 3 — Figure 3: Accuracy–latency Pareto fronts on TabArena for (a) classification and (b) regression tasks. Elo is plotted against median inference time per 1,000 samples (log scale); dashed staircases trace the Pareto fronts, and horizontal lines mark the 4-hour AutoGluon reference pipelines [ 7 ] . Using its default single-forward-pass inference at 0.605 s per 1,000 samples, EXAONE Tabular lies on the Pareto front of both task types. On classification it attains the highest Elo on the entire leaderboard, exceeding TabPFN-3 by $\sim$ 125 Elo at comparable latency. On regression it reaches the accuracy regime of TabFM — within overlapping 95% confidence intervals — at roughly $1/11$ the inference cost, and exceeds the regression-specialized Nori models [ 32 ] by $\sim$ 140 Elo.

Panel of Figure 4 — Figure 4: Default vs. tuned and post-hoc ensembled TabArena Elo for (a) classification and (b) regression. Bars show each method’s default configuration, with additional segments marking the gains from per-task tuning and post-hoc ensembling where those variants exist. Conventional GBDT and neural baselines depend heavily on both, worth $\sim$ 230 Elo for LightGBM [ 17 ] and RealMLP [ 16 ] on classification and over 420 Elo for RealMLP on regression alone. EXAONE Tabular (navy), like most tabular foundation models, is evaluated in a single untuned default configuration, and ranks first on classification and, on regression, second only to TabFM — a model $\sim$ 78 $\times$ its size. Dashed lines mark the 4-hour AutoGluon 1.4/1.5 pipelines, both of which it clears on both task types.

Figure 5: Pairwise win rates on the full TabArena benchmark (classification and regression). Each entry $(A,B)$ reports the percentage of evaluation splits on which method $A$ attains a lower test error than method $B$ ; cells are shaded from purple (row loses) through white (parity) to green (row wins). Method labels are colored by family — purple: tabular foundation models; blue: neural networks; green: GBDTs. With a single fixed default configuration, EXAONE Tabular wins the majority of splits against the strongest variant of every competing method except TabFM: 65–92% against the other tabular foundation models, and at least 89% against every tuned-and-ensembled GBDT and neural-network baseline. The advantage is therefore consistent at the split level rather than being driven by a small subset of datasets.

Panel of Figure 6 — Figure 6: Performance comparison on the BCCO benchmark. EXAONE Tabular achieves the second-highest mean accuracy on classification and the highest mean $R^{2}$ on regression, demonstrating competitive performance across both tasks. Error bars represent 95% bootstrap confidence intervals.

Panel of Figure 7 — Figure 7: Performance comparison on the TALENT benchmark. EXAONE Tabular achieves the second-highest mean accuracy on classification and the highest mean $R^{2}$ on regression, demonstrating consistently competitive performance across both tasks. Error bars represent 95% bootstrap confidence intervals.

Panel of Figure 8 — Figure 8: Joint point-estimation and predictive-distribution rankings on ScoringBench. Each point represents a model with valid results for both metrics and is positioned by its official Autorank mean ranks over 102 regression datasets; lower ranks are better on both axes. Fine-tuned and HPO variants are omitted for readability, leaving 28 models. The dashed diagonal indicates identical rank under $R^{2}$ and CRPS, while the highlighted Pareto-optimal point identifies EXAONE Tabular, which achieves the best mean rank for both metrics. Name tags are restricted to selected representative models.

Tabular Foundation Models

Tabular foundation models learn reusable representations or in-context prediction skills for structured data such as spreadsheets and databases.

See all 188 papers

OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning
Oct 14, 2026Mohamed Bouadi, Nassim Bouarour, Aditya Tanna +1Tabular Foundation Models Tabular ICL
Lexsi Labs
Xiaomi-TabLDM: A Tabular Foundation Model Technical Report
Sep 3, 2026Xiaomi-TabLDM Team, :, Penghui Wang +13Synthetic Data Pretraining Tabular Foundation Models
Xiaomi
TabICLv2: A better, faster, scalable, and open tabular foundation model
Feb 11, 2026Jingang Qu, David Holzmüller, Gaël Varoquaux +1Synthetic Data Pretraining Tabular Foundation Models
SODA Team, INRIA Saclay, Palaiseau, France · Probabl, France

Oct 14, 2026J/K move · Enter open · S save

OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning

Mohamed Bouadi, Nassim Bouarour, Aditya Tanna, Vinay Kumar Sankarapu

Lexsi Labs

Tabular Foundation Models Tabular ICL Tabular Prediction Mixture-of-Experts Models

Open paper AskPDF arXiv

Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter updates. Despite recent advances, most existing approaches rely on dense architectures that apply the same transformations across features and datasets. We introduce OrionMoE-Small-v1.0, a compact tabular foundation model that combines distribution-aware representations with dataset-conditioned sparse mixture-of-experts computation. Built on a dual-axis transformer architecture, OrionMoE-Small selectively activates experts with routing decisions informed by both individual cell representations and the broader dataset context. This design enables adaptive allocation of computation across heterogeneous tabular problems while retaining a shared computational pathway across tasks. The model is pretrained entirely on synthetic classification tasks generated from a structured causal prior and performs downstream prediction without gradient-based adaptation. Experiments across multiple tabular benchmarks demonstrate competitive performance against substantially larger foundation models, with particularly strong results on small-context and high-dimensional datasets. Further analysis reveals favorable relative performance under class imbalance and highlights the importance of feature composition.

Abstract

Tabular Foundation Models

Explore similar work

OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning

Xiaomi-TabLDM: A Tabular Foundation Model Technical Report

TabICLv2: A better, faster, scalable, and open tabular foundation model