TabCUQ: Context-Uncertainty Quantification for Tabular Foundation Models
Authors: Yuebei Shi, Molei Liu, Zijian Guo
Organizations: School of Mathematics, Zhejiang University · Center for Data Science, Zhejiang University · Department of Biostatistics, Peking University Health Science Center · Beijing International Center for Mathematical Research, Peking University
Tabular foundation models use in-context learning to estimate conditional expected outcomes and average treatment effects for specific groups of individuals. Using these estimates to guide scientific decisions requires reliable uncertainty quantification. The challenge is to account simultaneously for two sources of uncertainty: limited-context bias and context variation arising from the small labeled sample supplied at inference. Predictive spread does not directly quantify both sources of uncertainty, and we observe that the ordinary bootstrap may substantially overestimate context variation, whereas subsampling may underestimate it. We introduce TabCUQ, a perturbation-based framework that injects noise into observed outcomes and applies the foundation model to the perturbed contexts. TabCUQ estimates context variation using wild-bootstrap dispersion and accounts for limited-context bias using the maximum deviation between perturbed and original estimates. For a kernel-regression analogue whose estimation error admits an exact decomposition into these two components, we show that our framework provides asymptotically valid pointwise coverage. In simulations, TabCUQ substantially improves coverage, with average interval widths typically about twice those of an oracle bias-aware benchmark. Across seven real-world tasks, it also improves coverage of reference functions learned from much larger samples.
Figures & tables
Figure 1: In-context prediction with a tabular foundation model (TFM) pretrained on many synthetic datasets. The context contains observed feature–outcome pairs, while the query supplies feature values with its outcome Y left unknown. With its pretrained weights fixed, the model processes both inputs to produce a predictive distribution for Y .
Figure 2: TabCUQ quantifies context uncertainty. (A) With the pretrained TFM and query fixed, independently sampled contexts yield different estimates. Their spread represents context variation, while the gap between their across-context mean and truth represents limited-context bias. (B) Wild-bootstrap outcome perturbations provide an estimate of context variation and an empirical bias cushion. The blue central segment represents the contribution of estimated variation, while the orange extensions represent the empirical bias cushion.
Figure 3: Context uncertainty in two real-world examples. (a) Estimated conditional-mean stability change for the N139V mutation in PDB 1G3P. (b) Estimated CATE of job-training assignment on pretreatment earnings in the National Supported Work experiment, where the true effect is zero. Pink and teal indicate positive and negative estimates the solid orange line in (a) indicates the pseudo-truth from a LightGBM reference model fitted to the full protein data observations, approximately −0.091 kcal/mol. The gray dashed line in (b) marks the known zero CATE.
Figure 4: Context variation and limited-context bias of TabPFN 2.6 in the simulation setting with Y=sin(πX)+ε , where independent X,ε∼N(0,1) and the context size n=300 . (A) At x=−1.737 , TabPFN estimates from 100 independent contexts (gray) and 100 bootstrap estimates conditional on one context (blue). (B) Ratios of mean resampling variance to the variance of TabPFN estimates across independent contexts, averaged over 20 queries in [−3,3] . We use the same 100 contexts to estimate both variance, with 100 resamples per method and context. Subsampling draws m=30 without replacement and scales variance by m/(n−m) .
ri←Yi−μ(Xi),r←n1∑j=1nrj,ei←ri−r.
Algorithm 1 TabCUQ for a conditional mean
τ(x)
←μ1(x)−μ0(x),
τb(x)
←μ1,b(x)−μ0,b(x),b=1,…,B.
Algorithm 2 TabCUQ for a CATE using a T-learner
A. Conditional means: Y=μ(X)+ε(X)
Univariate: X∼N(0,1) . Multivariate: X∼N(0,Σρ) , with p∈{5,10} and ρ=0 for M1–M8. M1 also uses ρ=0.5 ; M9 uses p=2 and ρ∈{0,0.5} .
Design
Univariate mean
Multivariate mean
Outcome noise
Linear (U1 / M1)
x
∑j=1pxj
Z
Quadratic (U2 / M2)
x2
∑j=1pxj2
Z
Bimodal (U3 / M3)
x
∑j=1pxj
1.5U+Z
Heteroskedastic linear (U4 / M4)
x
∑j=1pxj
1D: (1+∣x∣)Z Multi: (1+p∣∑jxj∣)Z
Table 1: Synthetic data-generating processes for conditional means and CATE. U and M stand for paired univariate and multivariate designs, respectively; C stands for the CATE designs.
Figure 5: Univariate conditional-mean interval performance for TabPFN 2.6 across eight designs, with S=300 , n=300 , and B=300 . (a) Coverage averaged over the eight edge queries satisfying ∣x∣≥2 . (b) Mean pointwise interval-width/OBA-width ratio over the same queries. Dashed reference lines mark 0.95 coverage and a width ratio of one.
Figure 6: Multivariate conditional-mean interval performance for TabPFN 2.6 across 20 settings, with S=100 independent contexts of size n=300 and B=100 . (a) Coverage averaged over 20 fixed queries in the 90 – 99% Mahalanobis stratum. (b) Mean pointwise interval-width/OBA-width ratio over the same queries. Dashed reference lines mark 0.95 coverage and a width ratio of one.
Figure 7: Pointwise conditional-mean results in the quadratic design for TabPFN 2.6, with S=300 , n=300 , and B=300 . (a) Absolute Monte Carlo limited-context bias (orange dashed) and mean empirical bias cushion (blue solid). (b) Coverage of wild-bootstrap and TabCUQ intervals; the horizontal dashed line marks 0.95 . (c) Mean interval widths for wild bootstrap, TabCUQ, and OBA. The cushion in (a) is the added component of the TabCUQ half-width.
Figure 8: CATE interval performance for TabPFN 2.6 across nine designs, with S=100 , total n=600 , and B=100 . (a) Coverage averaged over the eight edge queries satisfying ∣x∣≥2 . (b) Mean pointwise interval-width/OBA-width ratio over the same queries. Dashed reference lines in (a,b) mark 0.95 coverage and a width ratio of one.
Model
Designs
Wild bootstrap coverage
TabCUQ coverage
TabCUQ width / OBA
A. Univariate edge queries
TabPFN 2.6
4
0.738 – 0.910
0.966 – 1.000
1.773 – 2.082
TabPFN 3
4
0.854 – 0.936
0.986 – 1.000
1.967 – 2.419
TabFM
4
0.784 – 0.900
0.971 – 0.998
1.722 – 2.164
B. Multivariate tail queries
TabPFN 2.6
9
0.633 – 0.851
0.981 – 0.998
1.651 – 2.072
Table 2: Sensitivity to the choice of foundation models. Entries give ranges across all the simulation settings. Panel A concerns the eight univariate edge queries in U1–U4; Panel B concerns the 20 queries in the 90 – 99% Mahalanobis stratum. Relative width is the mean pointwise TabCUQ/OBA width ratio, using each model’s own Monte Carlo bias and standard deviation to construct its OBA reference. All three models are evaluated on the same nine multivariate settings: M1 at p∈{5,10} and ρ∈{0,0.5} , M2–M4 at p=5 and ρ=0 , and M9 at p=2 and ρ∈{0,0.5} .
Dataset
Pool
Evaluation
Input p
Outcome
Census
11,800
200
2
Income
Tree cover
11,800
200
4
Canopy cover
Job Corps
10,014
200
23
Earnings
Jester Jokes
48,245
200
110
Rating
Protein engineering ΔΔG
8,754
461
145
Gibbs free energy changes ( ΔΔG )
Protein engineering ΔTm
4,346
571
155
Melting temperature changes ( ΔTm )
Table 3: Basic information of the real-world datasets included in our studies.
Figure 9: Pointwise reference coverage and relative interval width across six real-world tasks, with context size n=1,000 . For each task, the left panel shows the distribution across evaluation queries of reference coverage. The right panel shows the distribution of mean interval width over those contexts divided by the OBA width at the same query. Wild bootstrap uses the estimated standard error without the empirical bias cushion; TabCUQ adds the empirical bias cushion from the same wild-bootstrap perturbations. Linear model denotes ordinary least-squares intervals, included only for the Census and Tree cover examples. OBA stands for the oracle benchmark. Inner boxes indicate medians and interquartile ranges; horizontal bars indicate identical values across queries. Dashed lines mark 0.95 coverage and a width ratio of one.
Recent Tabular Foundation Models (TFMs) have demonstrated state-of-the-art predictive performance, often surpassing Gradient-Boosted Decision Trees (GBDTs). However, the trustworthiness of these models, particularly their uncertainty quantification, has been largely overlooked. We investigate this gap through an extensive study comparing TFMs, GBDTs, and classical baselines on the 112 datasets of the TALENT benchmark. Our results reveal a performance-uncertainty trade-off: although TFMs achieve the highest predictive performance, measured by AUC, they exhibit lower conditional coverage under conformal prediction, measured by SSCS, compared to GBDTs. Complementary experiments on synthetic datasets further characterize the regimes in which this effect intensifies. We conclude that while TFMs advance predictive frontiers, achieving well-calibrated uncertainty remains a major open challenge for their reliable adoption. Code is available at: https://github.com/jose-melo/high-performance-low-reliability
José Lucas De Melo Costa, Fabrice Popineau, Arpad Rimmel +1
CentraleSup´elec, Universit´e Paris-Saclay Gif-sur-Yvette - France
Tabular foundation models (TFMs) provide predictive distributions for regression, but their prediction regions can exhibit undercoverage or overcoverage even when point predictions are accurate. We introduce C-USIM (Conditionally-Uniformized Score Integration Method), a lightweight application of highest predictive density split conformal prediction that accommodates multimodal predictions. Given calibration and test outputs, it requires no additional training or model inference. It provides finite-sample marginal validity under our assumptions. We bound conditional-marginal coverage gaps using distribution-estimation error and score discreteness, and examine coverage heterogeneity through percentile rank-score plots. Experiments with TabPFN and TabICL show improved marginal coverage accuracy and lower average conditional and group coverage errors. Under a fixed data budget, allocating more observations to calibration can reduce marginal coverage error despite less accurate point predictions.
Sungwoo Park, Sunghee Park, Won Chang
Department of Statistics Seoul National University Seoul, Republic of Korea
Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning. However, many critical applications in mechanics and computational science require not only accurate predictions but also reliable uncertainty quantification (UQ). Herein we investigate the UQ capabilities of tabular FMs in regression tasks through a comprehensive empirical study comparing Tabular Prior-Data Fitted Networks (TabPFN) against Gaussian processes (GPs). We systematically evaluate these two methods across a host of regression problems with varying complexity, dataset sizes, and input dimensionalities. We use a default setting to build all the GPs and for a fair comparison against TabPFN v2.5. Our findings highlight an important trade-off between explicit and learned priors: while TabPFN achieves highly competitive performance for complex, high-dimensional problems with sufficient data, GPs often provide superior predictive accuracy and UQ in data-scarce settings. Moreover, when the chosen kernel constitutes a good prior for the underlying function, GP performance can substantially exceed that of TabPFN. Our results can be reproduced from https://github.com/kianswarehouse/GPvsPFN.
Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh +2
Department of Mechanical and Aerospace Engineering, University of California, Irvine · Department of Civil and Environmental Engineering, University of California, Irvine