TabPFN Extensions for Interpretable Geotechnical Modelling

Taiga Saito; Yu Otake; Daijiro Mizutani; Stephen Wu

cs.CEMar 22, 2026

TabPFN Extensions for Interpretable Geotechnical Modelling

Authors: Taiga Saito, Yu Otake, Daijiro Mizutani, Stephen Wu

Organizations: Department of Civil and Environmental Engineering, Tohoku University, 6-6-06 Aramaki Aoba, Aoba-ku, Sendai, Miyagi 980-8579, Japan · Research Organization of Information and Systems, The Institute of Statistical Mathematics 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan · Department of Statistical Science, The Graduate University for Advanced Studies 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan

Abstract

Geotechnical site characterisation relies on sparse, heterogeneous borehole data, where uncertainty quantification and interpretability matter as much as predictive accuracy. We evaluate TabPFN~\citep{Hollmann2025}, a tabular foundation model, and its tabpfn-extensions library on two geotechnical tasks: (1) soil-type classification from N-value and shear-wave velocity data as a controlled illustrative case, and (2) iterative imputation of five mechanical parameters ( $s_\mathrm{u}$ , $E_{\mathrm{u}}$ , ${σ'}_\mathrm{p}$ , $C_\mathrm{c}$ , $C_\mathrm{v}$ ) in BM/AirportSoilProperties/2/2025. Without retraining, we apply cosine-similarity analysis to TabPFN embeddings, visualise predictive distributions, and compute SHAP attributions. On the regression benchmark we compare TabPFN with mean imputation, linear regression, random forests, XGBoost, and HBM; introduce a proxy decomposition of predictive uncertainty across context-perturbation classes; and propagate marginal $C_\mathrm{c}$ and ${σ'}_\mathrm{p}$ distributions through a one-dimensional consolidation model to obtain the reliability index $β$ and serviceability exceedance probability $P_\mathrm{f}$ . Embeddings exhibit label-consistent Clay/Sand grouping; iterative imputation reduces RMSE for all five targets, with TabPFN lowest on four; SHAP attributions are consistent with the Skempton compression-index correlation and the inverse preconsolidation-pressure-water-content dependence; the within-posterior component is largest in the proxy decomposition. We position the contribution as a worked evaluation workflow that may complement established methods for data-scarce geotechnics, not as algorithmic innovation.

Figures & tables

Training set ( $n=16$ )				Test set ( $n=16$ )
No.	$N$	$V_{\mathrm{s}}$ (m/s)	Soil	No.	$N$	$V_{\mathrm{s}}$ (m/s)	Soil
1	1	100.000	Clay	1	4	158.740	Clay
2	2	125.992	Clay	2	5	170.998	Clay
3	7	191.293	Clay	3	9	208.008	Clay
4	12	228.943	Clay	4	10	215.443	Clay
5	15	246.621	Clay	5	11	222.398	Clay

Table 1: All samples in the synthetic geotechnical dataset. No. corresponds to the axis labels in Figure 3 : training samples are ordered along the row axis and test samples along the column axis (Clay first, sorted by ascending

N

-value; Sand second, sorted by ascending

N

-value).

Figure 1: Scatter plots of training (circles) and test (stars) data in the $N$ – $V_{\mathrm{s}}$ space, coloured by soil class (Clay: blue, Sand: orange). Training and test sets each contain 16 samples. Reference curves $V_{\mathrm{s}}=100N^{1/3}$ and $V_{\mathrm{s}}=80N^{1/3}$ from the Japanese railway seismic design standard ( Railway Technical Research Institute 2012 ) are overlaid; Clay samples follow the upper curve and Sand samples the lower curve.

Figure 2: Predicted probability of Sand class $P(\text{Sand})$ over the $N$ – $V_{\mathrm{s}}$ domain, with the training samples overlaid (circles). The bold contour marks the decision boundary at $P(\text{Sand})=0.5$ ; grey contours indicate the 0.1, 0.25, 0.75, and 0.9 levels.

Figure 3: Cosine similarity heatmap of TabPFN embeddings between test and training samples (axis labels correspond to Table 1 No.). The block-diagonal structure indicates that, in this controlled case, test embeddings are more similar to training embeddings from the same constructed class. Test sample No. 8 (first Sand test sample; $N=14$ , $V_{\mathrm{s}}=193$ m/s) shows visibly lower similarity with the training Sand block, illustrating how the heatmap can flag a boundary sample.

Figure 4: Normalised RMSE (RMSE / RMSE at iteration 1) per iteration for each mechanical parameter. Values below 1.0 indicate improvement over the initial estimate.

Parameter	RMSE (iter. 1)	RMSE (iter. 10)
$s_{\mathrm{u}}$ (kN/m 2 )	13.08	9.30
$E_{\mathrm{u}}$ (kN/m 2 )	3277	2692
${\sigma^{\prime}}_{\mathrm{p}}$ (kN/m 2 )	30.47	25.89
$C_{\mathrm{c}}$ (—)	0.118	0.106
$C_{\mathrm{v}}$ (m 2 /s)	35.72	34.58

Table 2: RMSE at iteration 1 and iteration 10 for each mechanical parameter.

Figure 5: Posterior distributions at iteration 10 for all five mechanical parameters across test samples. Violin width represents the probability density; black bar = median; yellow circle = true value; black filled circle = observed value (parameter was not missing for that sample and therefore not predicted).

Figure 6: Mean absolute SHAP values for each input feature across all test samples, shown separately for each mechanical parameter. Blue bars: index properties; orange bars: non-target mechanical parameters used as features in the iterative imputation scheme.

Figure 7: SHAP value versus feature value for the dominant predictors of each mechanical target. Columns (a)–(e) correspond to

s_{\mathrm{u}}

E_{\mathrm{u}}

{\sigma^{\prime}}_{\mathrm{p}}

C_{\mathrm{c}}

, and

C_{\mathrm{v}}

, respectively. The upper row shows the best non-target mechanical predictor per target by mean

|\mathrm{SHAP}|

(

s_{\mathrm{u}}\!\leftarrow\!E_{\mathrm{u}}

E_{\mathrm{u}}\!\leftarrow\!s_{\mathrm{u}}

{\sigma^{\prime}}_{\mathrm{p}}\!\leftarrow\!s_{\mathrm{u}}

C_{\mathrm{c}}\!\leftarrow\!C_{\mathrm{v}}

C_{\mathrm{v}}\!\leftarrow\!C_{\mathrm{c}}

); the lower row shows the best index-property predictor per target (

s_{\mathrm{u}},E_{\mathrm{u}}\!\leftarrow\!PL

;

{\sigma^{\prime}}_{\mathrm{p}}\!\leftarrow\!w

;

C_{\mathrm{c}},C_{\mathrm{v}}\!\leftarrow\!LL

). Positive SHAP values indicate that the feature increases the predicted value; negative values indicate a decrease.

Figure 8: Embedding-based borehole-structure diagnostic for the five BM/AirportSoilProperties mechanical targets. (a)

k\!=\!3,\,5,\,10,\,20

nearest-neighbour borehole purity per target at the default

n_{\mathrm{top}}\!=\!8

; the dashed line is the random-neighbour baseline. (b) PCA-reduced cosine

\Delta_{\mathrm{cos}}

n_{\mathrm{top}}\!=\!8

as a function of dimensionality

d

. (c)

k\!=\!3

borehole-purity ratio (mean purity divided by the random-neighbour baseline) against

n_{\mathrm{top}}\!\in\!\{4,8,12,16\}

; the dotted line marks ratio

=\!1

(chance level). (d) PCA-reduced cosine

\Delta_{\mathrm{cos}}

d\!=\!20

against

n_{\mathrm{top}}

	Mean imp.	Linear	HBM	RF	XGBoost	TabPFN
$C_{\mathrm{c}}$	0.332	0.127	0.128	0.108	0.123	0.106
$E_{\mathrm{u}}$ (kN/m 2 )	3834	2952	2588	3212	2376	2692
$s_{\mathrm{u}}$ (kN/m 2 )	24.50	16.19	9.74	14.64	16.69	9.30
${\sigma^{\prime}}_{\mathrm{p}}$ (kN/m 2 )	49.93	35.44	28.24	32.92	38.79	25.89
$C_{\mathrm{v}}$ (m 2 /s)	168.71	119.16	59.32	46.91	60.90	34.58

Table 3: Final-iteration RMSE on the BM/AirportSoilProperties 20-sample missing pattern. Units follow each target. Lowest value per target is in bold.

Figure 9: Matched-

\kappa

proxy decomposition of TabPFN’s predictive variance on the BM/AirportSoilProperties 20-sample missing pattern. Coloured curves are the across-scenario components for three context perturbations against

\kappa

: bootstrap with replacement (WR; dashed), without replacement (WOR; solid), and borehole-level holdout (holdout; solid), with

95\%

percentile-bootstrap CI bands from

20

scenarios per cell. The dashed blue baseline is the multi-seed run-to-run component estimated at

\kappa=1.0

with the context fixed. The violet line is the within-posterior component averaged across the three context classes; no CI band is shown for this cross-class average.

Figure 10: Per-sample settlement comparison between TabPFN (blue circles), HBM (orange squares), and the deterministic ground truth (gold circles) for the 15 jointly imputed samples. Markers show

S

evaluated at the saved posterior mean of

(C_{\mathrm{c}},{\sigma^{\prime}}_{\mathrm{p}})

; error bars show 5%/95% Monte Carlo intervals propagated through Eq. ( 3 ). HBM marginals are reconstructed by log-normal quantile matching.

Capability	HBM	Conv. ML †	TabPFN + Ext.
Posterior predictive distribution	Yes (parametric)	No ∗	Yes (native)
Embedding similarity	Not native	No	Yes
SHAP feature importance	Indirect ‡	Yes	Yes
No retraining required	No	No	Yes
Correlation structure	Parametric	—	Implicit / prior-learned
User-specified priors	Yes	No	No

Table 4: TabPFN-centric capability summary; complementary strengths of HBM and conventional ML are discussed in the surrounding paragraph.

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 649 papers

On the Uncertainty Quantification Ability of Tabular Foundation Models
May 31, 2026Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh +2Tabular Foundation Models Uncertainty Quantification
Department of Mechanical and Aerospace Engineering, University of California, Irvine · Department of Civil and Environmental Engineering, University of California, Irvine
TabPFN-3.5: Technical Report
Sep 15, 2026Benjamin Jäger, Nick Erickson, Léo Grinsztajn +44Tabular Foundation Models Inference-Time Optimization
Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques
Aug 14, 2026Haibin Xiong, Shaoheng Dai, Peng Lan +4Materials Property Prediction Uncertainty Quantification
School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia. · School of Infrastructure Engineering, Nanchang University, Xuefu Road, 999, Nanchang 330031, China. · School of Civil Engineering, Central South University, Changsha, 410075, China.

May 31, 2026J/K move · Enter open · S save

On the Uncertainty Quantification Ability of Tabular Foundation Models

Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh, Oriol Vendrell-Gallart+1

Department of Mechanical and Aerospace Engineering, University of California, Irvine · Department of Civil and Environmental Engineering, University of California, Irvine

Tabular Foundation Models Uncertainty Quantification Prior-Data Fitted Networks Tabular Regression Gaussian Process Regression

Open paper AskPDF arXiv

Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning. However, many critical applications in mechanics and computational science require not only accurate predictions but also reliable uncertainty quantification (UQ). Herein we investigate the UQ capabilities of tabular FMs in regression tasks through a comprehensive empirical study comparing Tabular Prior-Data Fitted Networks (TabPFN) against Gaussian processes (GPs). We systematically evaluate these two methods across a host of regression problems with varying complexity, dataset sizes, and input dimensionalities. We use a default setting to build all the GPs and for a fair comparison against TabPFN v2.5. Our findings highlight an important trade-off between explicit and learned priors: while TabPFN achieves highly competitive performance for complex, high-dimensional problems with sufficient data, GPs often provide superior predictive accuracy and UQ in data-scarce settings. Moreover, when the chosen kernel constitutes a good prior for the underlying function, GP performance can substantially exceed that of TabPFN. Our results can be reproduced from https://github.com/kianswarehouse/GPvsPFN.

Abstract

Uncertainty Quantification

Explore similar work

On the Uncertainty Quantification Ability of Tabular Foundation Models

TabPFN-3.5: Technical Report

Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques