Tabular ICL

ICL: In-Context Learning

Tabular in-context learning studies models that infer a table’s prediction rule from labeled examples supplied at inference time, often using foundation models pretrained on synthetic or real tabular tasks. Common approaches include prior-fitted transformers, retrieval and context-selection methods, and test-time adaptation; performance is often compared on suites such as TabArena, TALENT, and OpenML. Work now extends beyond classification and regression to anomaly detection, imputation, generation, survival analysis, and streaming settings. A central tension is how to scale context and broaden task coverage while controlling inference cost and ensuring robustness to distribution shift, noisy examples, and privacy risks.

Latest papers

108 papers28 papers in the last four weeks, up 211% on the four weeks before. 0.2% of all new papers.

Recall or Relearn? A Mixture of In-Context Experts for Recurring Concept Drift
Oct 27, 2026Zhangkai Wu, Xu Zhang, Zhihong Cui +2Concept Drift Online Continual Learning
University of Sydney, Sydney, Australia · Macquarie University, Sydney, Australia · Department of Informatics, University of Oslo, Oslo, Norway +2
Beyond the Vote: When Do LLMs Add Value to Retrieval-Based Tabular Classification?
Oct 21, 2026Ziquan Wang, Yuxuan Nie, Jian Ma +1Tabular ICL Retrieval-Augmented ICL
Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · Peking University, Beijing, China
Two Is Enough: Binary Classification Priors Suffice for Multi-Class In-Context Tabular Prediction
Oct 20, 2026Zhixiong Cheng, Shiyuan Han, Jingjing Suo +1Tabular Foundation Models Tabular ICL
Bairong Inc.AI, Beijing, China
On Data Scaling in Tabular ML: More Data, Better Generalization?
Oct 19, 2026Ivan Rubachev, Nikita Lazarev, Artem BabenkoTabular ML Tabular Foundation Models
Yandex · HSE University
FTFM: Factorized Tabular Foundation Models
Oct 18, 2026Josif GrabockaTabular Foundation Models Tabular ICL
University of Technology Nuremberg
OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning
Oct 14, 2026Mohamed Bouadi, Nassim Bouarour, Aditya Tanna +1Tabular Foundation Models Tabular ICL
Lexsi Labs
When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models
Oct 14, 2026Wenwen Gong, Haotian Gu, Zhen Yang +6Tabular Foundation Models Transformer Inference
China Agricultural University · Tsinghua University · Nankai University
TabGift: A Context-Efficient Foundation Model for Tabular Data Generation
Oct 12, 2026Yu-Sheng Li, Jun-Peng Jiang, Han-Jia YeSynthetic Tabular Data Generation Generative Modeling
School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China
When Repeated Measurements Hurt Tabular Foundation Models
Oct 12, 2026Yixiang Liu, Ruizhe Deng, Bibhas Chakraborty +2Tabular Foundation Models Tabular Prediction
National University of Singapore · Duke-NUS Medical School · Mohamed bin Zayed University of Artificial Intelligence
Relational Learning as World Modeling
Oct 12, 2026Zheyu Zhang, Gjergji KasneciAction-Conditioned World Models World Model-Based Planning
Technical University of Munich Munich Center for Machine Learning (MCML)
TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks
Oct 11, 2026Jayoung Kim, Sanghyun Hong, Noseong ParkSynthetic Tabular Data Generation Tabular Diffusion Models
KAIST · Oregon State University
Recurrent In-Context Learning for Tabular Foundation Models
Oct 10, 2026Tejas Vaidhya, Gaurav Sahu, Irina RishTabular Foundation Models Tabular ICL
Mila – Quebec AI Institute · Nolano AI · Unified Sciences
Tabular Foundation Models Are Effectively Shallow
Oct 9, 2026Irene Cannistraci, Julia E. VogtTransformer Pruning Tabular Foundation Models
Department of Computer Science ETH Zurich Switzerland
Test-Time Compute for Tabular Foundation Models: Mechanisms, Gains, and Limits
Oct 8, 2026Kanghui Ning, Marin Biloš, James T. Wilson +5Tabular Foundation Models Inference-Time Scaling
School of Computing, University of Connecticut, Storrs, USA. · Department of Machine Learning Research, Morgan Stanley, New York, USA.
Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learning
Oct 8, 2026Xu Zhao, Jiaming Zhao, Bin Zhao +1Context Compression Tabular ICL
Thinking in Depth: Retrospective Inference for Tabular Foundation Models
Oct 7, 2026Hao-Run Cai, Si-Yang Liu, Zi-Jian Cheng +8Tabular Foundation Models Tabular ICL
School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University
GeneICL: A Tabular Foundation Model for Bulk Transcriptomics
Oct 6, 2026Michael Bohl, Alexander Theus, David Wissel +1Tabular Foundation Models Clinical Outcome Prediction
ETH Zurich · Max Planck Institute for Intelligent Systems · Swiss Institute of Bioinformatics +1
The Standardization Trap: Certifying Joint Label Processing in Tabular Foundation Models
Oct 6, 2026Duong Nguyen, Nicolas Chesneau, Milan BhanTabular Foundation Models Transformer Interpretability
Ekimetrics
TICDA: Tabular In-Context Data Attribution
Oct 6, 2026Yacine Benihaddadene, Milan Bhan, Eliot Dugelay +4Tabular Foundation Models Tabular ICL
Ekimetrics · ETH Zurich
TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity
Oct 6, 2026Zijian Li, Xiangchen Song, Gongxu Luo +7Synthetic Data Pretraining Tabular Foundation Models
Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence · Guangdong University of Technology +1
Closing the Context Gap: Activation Alignment for Tabular In-Context Learning
Oct 5, 2026Yoel ZeldesTabular Foundation Models Tabular ICL
Independent Researcher
Rethinking Tabular Foundation Models On Data Streams
Oct 4, 2026Nilesh Verma, Daniel Nowak-Assis, Afonso Lourenço +4Tabular Foundation Models Tabular ICL
AI Institute, University of Waikato, New Zealand · LIP6, CNRS, Sorbonne Universit´e, France · GECAD, Polytechnic of Porto, Portugal
Distillation of Tabular Foundation Models into Efficient Predictors
Oct 1, 2026Minho Jeong, Dooho Lee, Jinmo Lee +1Tabular Foundation Models Efficient Neural Network Inference
Nums AI
Molecular Property Prediction under Structural Shift with Tabular Foundation Models
Sep 30, 2026Jinmo Lee, Dooho Lee, Minho Jeong +1Tabular Foundation Models Molecular Property Prediction
Nums AI
TabFM: A Zero-Shot Foundation Model for Tabular Data
Sep 29, 2026Weihao Kong, Erez Louidor Ilan, Shuxin Nie +6Tabular Foundation Models Tabular Prediction
Google Research
TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Tasks
Sep 29, 2026Doyun Choi, Dooho Lee, Jaemin YooTabular Anomaly Detection Zero-Shot Anomaly Detection
Seoul National University · KAIST · Nums AI Inc.
Architecture Alignment With Sparse Priors in Tabular Foundation Models
Sep 29, 2026Tianqi Zhao, Tianyi Zhuang, Shuo Duan +3Tabular Foundation Models Transformer
National University of Singapore · Rutgers University
Adapting Linear-Time Architectures for Tabular In-Context Learning
Sep 28, 2026David Schnurr, Felix Sarnthein, Thomas Hofmann +1Tabular ICL Tabular Foundation Models
ETH Zürich · ELLIS Institute Tübingen · MPI-IS +1
LoopICL: Looping a single transformer block to solve tabular tasks
Sep 28, 2026Amir Rezaei Balef, Katharina EggenspergerTabular Foundation Models Cost-Aware Inference
TU Dortmund University, Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Dortmund, Germany · University of Tübingen, Tübingen, Germany
Benchmarking Attention for Tabular Foundation Models
Sep 25, 2026Maximilian Schambach, Clemens Biehl, Sam ThelinTabular Foundation Models Efficient Attention
SAP SE, Germany

Oct 27, 2026J/K move · Enter open · S save

Recall or Relearn? A Mixture of In-Context Experts for Recurring Concept Drift

Zhangkai Wu, Xu Zhang, Zhihong Cui, Yiqi Wang+1

University of Sydney, Sydney, Australia · Macquarie University, Sydney, Australia · Department of Informatics, University of Oslo, Oslo, Norway+2

Concept Drift Online Continual Learning Tabular ICL Ensemble Learning

Open paper AskPDF arXiv

Tabular foundation models (TFMs) are being applied to Web data streams such as network intrusion detection, where they adapt by updating the labelled rows of their context without retraining. When a past concept returns after leaving the context, a TFM faces a recall-or-relearn decision between restoring stored rows of the concept and relearning it from new labels. We propose MICE (Mixture of In-Context Experts), a training-free learner that cuts the stream into segments at detected changes. It merges a closed segment into a stored context when the TFM's estimated additional error on the segment is within a tolerance, under a capacity limit on stored contexts. A fast level weights the stored contexts and first-in-first-out (FIFO) windows by the log-loss of their predictions on the last labelled batch. We show that, with pure stored contexts and a margin condition, this fast level gains over relearning, in expected accuracy, the normalised positive area between the stored context's accuracy and the TFM's learning curve, up to a first-batch term. A slow level mixes the fast level's prediction with that of the FIFO window of the last labelled rows and bounds the discounted half-Brier regret of MICE against this window at every time. Learning curves measured beforehand predict the ordering of the observed recall gains. MICE outperforms reset, which empties the context at detected drifts, on grid streams with random boundary phases. Merging keeps fewer experts than pools that store every segment, and with fixed segmentation a time-matched snapshot pool matches MICE on the shifted grid. On real streams, MICE has the highest mean accuracy of all policies we compare.