Feature Selection for Tabular Foundation Models: Exploring Feature Utility in Low-Data Molecular Prediction

Yiqi Yao; Miquel Duran-Frigola

cs.LGSep 25, 2026

Feature Selection for Tabular Foundation Models: Exploring Feature Utility in Low-Data Molecular Prediction

Authors: Yiqi Yao, Miquel Duran-Frigola

Organizations: Harvey Mudd College · Ersilia Open Source Initiative

Abstract

Choosing features is difficult with few labeled molecules, even for a tabular foundation model that learns in context. We study this problem with TabPFN for low-data molecular property prediction, separating input width, feature meaning, and predictive usefulness while keeping contexts, query molecules, and predictor settings matched. On ADME/Tox assays from Therapeutics Data Commons, endpoint-matched assay predictions improve a compact RDKit descriptor baseline, and a provenance audit shows that most audited query molecules already occur in the predictors' training sources. Across 77 FS-Mol development tasks and 157 held-out test assays drawn from ChEMBL, broad RDKit descriptor and fingerprint families provide a strong default, rank-averaging TabPFN with support-fitted fingerprint forests improves upon both models, and learned weighting of family predictions adds further gains at a larger prediction budget. A utility-trained selector builds a traceable 16-column input from 238 sourceassay predictors, improving upon the baseline while trailing wider representations and equalwidth structural SVD coordinates. Named outputs identify what each feature predicts and where it comes from. Their usefulness for a new assay is assessed alongside the other input features, and paired removal and replacement tests measure what each selected output contributes. Together, these studies offer a protocol for evaluating feature selection, representation choice, and data provenance with a fixed tabular foundation model for few-shot molecular property prediction.

Figures & tables

Separating input width, feature meaning, and predictive usefulness. Wide reference representations supply 527 to 2,048 coordinates, whereas the named-assay representation appends five traceable source-assay predictions to the 11-descriptor anchor, enabling direct tests of both the five-output portfolio and individual selected columns. For the TRPV3 calcium-flux assay at 64 labels, the selected MC4 binding output improves mean ROC AUC by $+0.27$ points over equal-width random replacements (Appendix H.1 ). Aggregate results across the 77-task development cohort are shown in Figure 1 .

Figure 1: Comparison of broad structural representations and five selected source-assay outputs across 77 FS-Mol development tasks. Full-query ROC-AUC gains over RDKit-11, with two context draws averaged per task. Bars indicate task-equal means and 95% intervals from 10,000 bootstrap resamples over 62 publications. The supplied input widths are 16, 527, and 2,048 columns; TabPFN uses one estimator and internally subsamples CheMeleon to 679 coordinates. Grey labels report the absolute ROC AUC of the RDKit-11 anchor. Paired wide-reference contrasts are tabulated in Appendix G , and the equal-width reconstructed-panel comparison is given in Table 1 .

Figure 2: Combining all 21 RDKit families improves upon the 11-descriptor anchor across all context sizes, and learned portfolio weighting yields further gains. A , Full-query mean ROC-AUC gains for each of the 21 single-family additions at 16 and 128 labeled molecules, ordered within each feature group by the 128-label mean. B , Half-query ROC-AUC gains over RDKit-11 for the combined 21-family representation (

+3.48

+3.79

+4.41

, and

+5.32

points), the single family with the highest leave-one-task-out mean utility, and a split-query diagnostic that selects a portfolio on one query half and evaluates it on the complementary half. C , Gains over a single all-21 forward pass from learned and context-free softmax weighting across 58 portfolio predictions, alongside uniform portfolio averaging. The equal-budget eight-forward comparison is shown in Figure 10 C. Error bars in B and C indicate 95% publication-bootstrap intervals for task-equal means across 77 tasks from 62 publications (two draws per task; Appendix E ).

Panel of Figure 3 — Figure 3: Offline utility learning and deployment for named source-assay selection. A , Across FS-Mol source episodes, an offline TabPFN teacher evaluates the ROC-AUC gain $U(C,Q,S)$ from appending a portfolio $S$ of source-assay predictions to RDKit-11, supervising a ridge student $F_{\theta}$ with signed single-output and pairwise terms conditioned on task metadata $m_{t}$ and support size $n$ . B , For a new destination assay (illustrated for TRPV3 Ca 2+ flux at $n=64$ ), cached task metadata and support size guide a provenance-screened beam search over 238 source-assay Random Forests (ineligible models shown in pale grey), appending five selected predictions to RDKit-11 for in-context TabPFN prediction.

Labels	Frozen five RF outputs	Five SVD coordinates	Full eligible RF panel	SVD $-$ frozen
16	$+0.33$ $[-0.37,+1.03]$	$+1.75$ $[+0.95,+2.53]$	$+1.73$ $[+0.69,+2.81]$	$+1.42$ $[+0.48,+2.34]$
32	$+0.10$ $[-0.68,+0.84]$	$+1.67$ $[+1.04,+2.28]$	$+1.20$ $[+0.10,+2.24]$	$+1.57$ $[+0.78,+2.39]$
64	$+0.30$ $[-0.28,+0.91]$	$+2.01$ $[+1.40,+2.68]$	$+1.42$ $[+0.52,+2.36]$	$+1.70$ $[+0.95,+2.54]$
128	$+0.85$ $[+0.27,+1.44]$	$+2.68$ $[+1.98,+3.40]$	$+2.19$ $[+1.18,+3.14]$	$+1.84$ $[+1.09,+2.58]$

Table 1: Equal-width compression and frozen-source-choice controls on matched FS-Mol episodes. Entries are AUC-point gains over RDKit-11 or paired differences, with nominal 95% publication-cluster intervals. The RF predictions are from the reconstructed panel; the shared RDKit family projection used by SVD-5 was refitted without structures from the 77 development tasks.

Figure 4: Paired deletion and replacement tests quantify the conditional contribution of individual source-assay outputs. A , Semantic description distance versus conditional deletion effect

D_{j}

(16-column portfolio minus 15-column portfolio with output

j

removed) across the 37 active pair-by-size conditions; the 59 conditions in which output

j

was not selected have

D_{j}=0

and are included in B and Appendix H . Labels a–c mark the illustrative pairs shown in C. B , Task-equal mean deletion effects across all 24 preselected pairs (four draws averaged per pair across 21 tasks from 18 publications), with 95% publication-bootstrap intervals. C , Four-draw deletion trajectories (grey circles), means (blue squares), and min-max ranges (shaded bars) for three illustrative source-destination pairs, annotated with semantic and lexical distance percentiles. Equal-width replacement distributions and complete pair tables are provided in Appendices H and H.1 .

Paired comparison at 32 labels	Gain	95% publication interval
All-21 $-$ RDKit-11	$+2.94$	$[+1.75,+5.47]$
SVD-5 $-$ RDKit-11	$+1.41$	$[+1.12,+2.54]$
All-21 $-$ SVD-5	$+1.53$	$[+0.61,+3.01]$
Named-5 $-$ SVD-5	$-1.38$	$[-1.69,-0.98]$
Hybrid-5 $-$ SVD-5	$-0.96$	$[-1.32,-0.50]$
All-21 (8-pass) $-$ All-21 (1-pass) †	$+0.22$	$[+0.05,+0.45]$

Table 2: Held-out representation and baseline comparisons across all 157 FS-Mol test assays (1,570 paired episodes at 32 labels; † 314 episodes across two draws for the eight-pass budget diagnostic) in

\Delta

AUPRC percentage points. SVD-5, Named-5, and Hybrid-5 each supply 16 input columns (the RDKit-11 anchor plus five coordinates), whereas All-21 supplies 527 columns. Bootstrap intervals resample the 28 test publications.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Labels	All-21 $-$ RDKit-11	SVD-5 $-$ RDKit-11	All-21 $-$ SVD-5
16	$+2.31\ [1.22,4.91]$	$+0.97\ [0.70,2.19]$	$+1.34\ [0.49,2.91]$
32	$+2.94\ [1.75,5.47]$	$+1.41\ [1.12,2.54]$	$+1.53\ [0.61,3.01]$
64	$+3.18\ [2.04,5.76]$	$+1.84\ [1.45,2.70]$	$+1.35\ [0.57,3.24]$

Appendix

Table 3: Paired held-out gains in

\Delta

AUPRC percentage points across 157 FS-Mol test assays (ten draws per support size), with 95% publication-cluster bootstrap intervals.

	Dev (77 tasks)		Test (157 tasks)		Paired test $\Delta$ AUPRC gain (pts, 95% CI)
Model / representation ( $n=32$ )	$\Delta$ AUPRC	ROC	$\Delta$ AUPRC	ROC	vs. All-21	vs. Tanimoto	vs. RF-ECFP
SVD-5 TabPFN (16 cols, 1.3 s)	$0.1777$	$0.6817$	$0.1243$	$0.6181$	$-1.53\ [-3.01,-0.61]$	$-0.70\ [-1.40,+1.29]$	$-1.66\ [-3.39,-0.47]$
Tanimoto SVM (ECFP4)	$0.1824$	$0.6853$	$0.1312$	$0.6229$	$-0.83\ [-3.72,-0.11]$	—	$-0.96\ [-3.39,-0.11]$
RF-ECFP+Desc (2,252 cols)	$0.2041$	$0.7048$	$0.1348$	$0.6279$	$-0.47\ [-0.84,+0.57]$	$+0.36\ [-0.62,+3.84]$	$-0.60\ [-0.85,+0.80]$
Tanimoto+Desc SVM	$0.1966$	$0.6961$	$0.1350$	$0.6273$	$-0.45\ [-1.84,+0.05]$	$+0.38\ [+0.07,+2.13]$	$-0.58\ [-1.66,+0.07]$
All-21 TabPFN (527 cols, 41.2 s)	—	—	$0.1395$	$0.6321$	—	$+0.83\ [+0.11,+3.72]$	$-0.13\ [-0.66,+0.73]$

Appendix

Table 4: Matched development (77 tasks from 62 publications, two draws) and held-out test (157 tasks from 28 publications, ten draws; 1,570 episodes per arm) evaluations at 32 support labels comparing TabPFN representations, single-task fingerprint baselines, and rank-averaged combinations. All paired differences and 95% publication-cluster bootstrap intervals are computed on the 157 held-out test tasks (4,000 resamples over 28 publications).

Figure 5: Upstream training-source overlap and paired ROC-AUC gains on the cached ADME/Tox cohort. A , Assay-level ROC AUC at 50 labels, averaged over three support draws. Right-hand annotations report the fraction of evaluated query observations found in retrieved upstream training sources for the three audited assays across splits. B , Paired ROC-AUC gains over RDKit-11 at 10, 25, and 50 labels across the nine assays (light circles: three-draw assay means; filled markers: nine-assay means; hollow squares: means over additional draws 3–5, with

+3.92

+1.47

, and

+1.52

points and a 10-label

t

-interval of

[-0.08,7.92]

Comparison	$n$	Effect	95% interval
Endpoint + related minus anchor	10	$+4.40$	[ $+1.40$ , $+7.43$ ]
Endpoint + related minus anchor	25	$+1.57$	[ $-1.38$ , $+4.65$ ]
Endpoint + related minus anchor	50	$+2.90$	[ $+0.52$ , $+5.71$ ]
Endpoint + related minus CheMeleon	10	$+3.61$	[ $-3.00$ , $+10.86$ ]
Endpoint + related minus CheMeleon	25	$+0.82$	[ $-3.81$ , $+5.25$ ]
Endpoint + related minus CheMeleon	50	$+2.27$	[ $-1.37$ , $+5.93$ ]

Appendix

Table 5: Paired ADME/Tox effects on the same nine-assay cohort. Effects are AUC percentage points; intervals resample assays.

$n$	Original gain	$t$ interval	Draws 3–5	Bootstrap interval
10	$+4.40$	[ $+0.55$ , $+8.24$ ]	$+3.92$	[ $+1.40$ , $+7.51$ ]
25	$+1.57$	[ $-2.27$ , $+5.42$ ]	$+1.47$	[ $-1.47$ , $+4.74$ ]
50	$+2.90$	[ $-0.41$ , $+6.20$ ]	$+1.52$	[ $-0.64$ , $+4.20$ ]

Appendix

Table 6: Endpoint-plus-related augmentation: uncertainty and draw sensitivity across the nine cached ADME/Tox assays.

Figure 6: Per-assay ROC-AUC trajectories across support sizes. Three-draw mean ROC AUC for the RDKit-11 anchor, wide baselines, and endpoint-matched augmentations across 10, 25, and 50 support labels on all nine assays, together with the eight-assay cohort at 100 labels (shaded region).

Figure 7: Record retention and active-class prevalence in the cached ADME/Tox cohort. Printed counts indicate cached versus total cleaned records (and cached percentage) per assay, while markers compare active-class fractions between cached and excluded records.

Family	Cols.	Construction	$n=16$	$32$	$64$	$128$
Topology	21	Explicit scalar values	$+1.25$	$+0.81$	$+0.79$	$+1.58$
BCUT	8	Explicit scalar values	$+2.06$	$+1.88$	$+1.80$	$+3.04$
EState	25	Explicit scalar values	$+1.11$	$+0.95$	$+1.44$	$+1.90$
PEOE	14	Explicit scalar values	$+1.20$	$+1.25$	$+1.63$	$+2.34$
SMR	11	Explicit scalar values	$+1.19$	$+0.83$	$+1.24$	$+1.69$
SlogP	12	Explicit scalar values	$+1.23$	$+1.33$	$+1.64$	$+1.92$

Appendix

Table 7: The 21-family catalogue adds 516 columns to the 11-column anchor. The last four columns give full-query task-mean AUC gains over the anchor, in points, at each context size.

Figure 8: Split-half reliability and support-signal portfolio selection across 77 development tasks. A , Median Spearman-Brown split-half utility correlation across context sizes, with the 0.6 reference line. B , Half-query ROC-AUC gain over All-21 for the split-query diagnostic. C , Half-query ROC-AUC difference relative to All-21 when selecting a single portfolio per episode using each support-set signal (columns labeled “min” select the minimum signal value; “Align.” denotes centered kernel-target alignment; “combo.” denotes a leave-one-task-out ridge combination; TabPFN CV is evaluated on 26 tasks and all other signals on 77 tasks). Error bars show 95% publication-bootstrap intervals.

Figure 9: Task-by-family utility matrix at 64 support labels. Full-query ROC-AUC gains over RDKit-11 across the 77 development tasks (columns, ordered by hierarchical correlation clustering) for each of the 21 RDKit families and the combined all-21 representation (rows). Black dots mark the top-scoring single family for each task, and right-hand bars show the fraction of tasks won by each family.

Figure 10: Softmax temperature sensitivity, feature ablations, and forward-budget controls for portfolio weighting. A , Half-query ROC-AUC gain over a single all-21 forward pass as a function of softmax temperature

T

(

T=0

corresponds to hard top-1 selection and

T=\infty

to uniform averaging). B , Ablation of utility-model inputs at 64 labels relative to uniform averaging across the 58 portfolios. C , Comparison against an eight-estimator all-21 TabPFN ensemble at 16 (light) and 64 (dark) labels across varying forward-pass budgets (uniform 58-portfolio averaging yields

+0.40

[+0.04,+0.80]

at 16 labels and

+0.11

[-0.26,+0.47]

at 64 labels; Table 9 ). Error bars indicate 95% publication-bootstrap intervals.

$n$	Learned minus uniform	Nominal publication interval
16	$+0.05$	[ $-0.10$ , $+0.20$ ]
32	$+0.28$	[ $+0.08$ , $+0.49$ ]
64	$+0.35$	[ $+0.14$ , $+0.56$ ]
128	$+0.49$	[ $+0.29$ , $+0.72$ ]

Appendix

Table 8: Learned versus uniform weighting of the same 58 portfolio predictions. Both arms use 58 forwards; this tests weighting within the multi-view library.

Recipe	Forwards	$n$	Effect	Publication-bootstrap interval
All-21, one estimator	1	16	$-0.19$	[ $-0.35$ , $-0.04$ ]
All-21, one estimator	1	64	$-0.43$	[ $-0.62$ , $-0.27$ ]
Learned top 8	8	16	$+0.30$	[ $-0.08$ , $+0.71$ ]
Learned top 8	8	64	$+0.23$	[ $-0.01$ , $+0.47$ ]
Learned full weighting	58	16	$+0.45$	[ $+0.15$ , $+0.81$ ]
Learned full weighting	58	64	$+0.46$	[ $+0.20$ , $+0.73$ ]

Appendix

Table 9: Prediction recipes relative to all 21 families with eight TabPFN estimators. The top-eight recipe has equal forward count; its paired intervals include zero.

Representation or policy	$n=16$	$n=32$	$n=64$	$n=128$
RDKit-11	$62.48$	$66.06$	$69.65$	$72.49$
All-21 RDKit	$65.96$	$69.87$	$74.04$	$77.77$
CheMeleon	$66.37$	$70.10$	$73.14$	$77.26$
Named selector	$63.06$	$66.28$	$70.12$	$73.41$
Global additive learner	$62.71$	$66.14$	$69.92$	$73.38$
Global pairwise learner	$62.52$	$66.32$	$69.95$	$73.28$

Appendix

Table 10: Matched named-output development performance: 77 tasks, 62 publications and two original support draws. AUC is reported as a percentage.

Comparator	$n$	Effect	Nominal interval	Tasks
RDKit-11	16	$+0.58$	[ $-0.00028$ , $+1.18$ ]	77
RDKit-11	32	$+0.22$	[ $-0.53$ , $+0.92$ ]	77
RDKit-11	64	$+0.46$	[ $-0.14$ , $+1.09$ ]	77
RDKit-11	128	$+0.92$	[ $+0.39$ , $+1.45$ ]	77
All-21 RDKit	16	$-2.90$	[ $-4.19$ , $-1.70$ ]	77
All-21 RDKit	32	$-3.59$	[ $-4.64$ , $-2.59$ ]	77

Appendix

Table 11: Named selector minus each comparator, in AUC percentage points. Intervals are nominal 95% publication-cluster bootstrap intervals (10,000 replicates).

Figure 11: Effect of task-metadata conditioning across support sizes. Paired ROC-AUC differences between the task-conditioned named selector and the matched global pairwise learner (filled bars) or shuffled-metadata control (hollow bars) across the 77 development tasks (two draws averaged per task; error bars indicate 95% publication-bootstrap intervals from Table 11 ).

Control	$n=16$	$n=32$	$n=64$	$n=128$
Global additive	$+0.63$ $[+0.26,+1.02]$	$+0.09$ $[-0.23,+0.43]$	$+0.15$ $[-0.06,+0.38]$	$-0.09$ $[-0.34,+0.18]$
Global pairwise	$+0.36$ $[+0.08,+0.64]$	$-0.08$ $[-0.32,+0.14]$	$+0.04$ $[-0.12,+0.20]$	$-0.04$ $[-0.29,+0.22]$
Shuffled metadata	$+0.31$ $[+0.06,+0.56]$	$-0.04$ $[-0.30,+0.23]$	$-0.06$ $[-0.26,+0.14]$	$+0.06$ $[-0.13,+0.24]$

Appendix

Table 12: Four-draw sensitivity of the named selector’s task conditioning. Entries are selector-minus-control mean AUC points and nominal 95% publication-cluster intervals on the same 77 development tasks.

Added features	Input width	$n$	Gain [nominal interval]
Eligible RF panel	203–248	16	$+1.73$ $[+0.69,+2.81]$
Eligible RF panel	203–248	32	$+1.20$ $[+0.10,+2.24]$
Eligible RF panel	203–248	64	$+1.42$ $[+0.52,+2.36]$
Eligible RF panel	203–248	128	$+2.19$ $[+1.18,+3.14]$
Five SVD coordinates	16	16	$+1.94$ $[+1.22,+2.69]$
Five SVD coordinates	16	32	$+1.46$ $[+0.82,+2.06]$

Appendix

Table 13: Catalogue and compression controls on a reconstructed source-model panel. Entries are paired gains over RDKit-11 in AUC points with nominal 95% publication-cluster intervals.

Labels	Tasks	Frozen RF gain	SVD-5 gain	SVD $-$ frozen
16	72	$+0.29$ $[-0.46,+1.05]$	$+1.74$ $[+0.87,+2.58]$	$+1.45$ $[+0.48,+2.43]$
32	72	$+0.08$ $[-0.76,+0.88]$	$+1.62$ $[+0.97,+2.26]$	$+1.55$ $[+0.71,+2.41]$
64	72	$+0.24$ $[-0.36,+0.88]$	$+1.91$ $[+1.33,+2.54]$	$+1.67$ $[+0.90,+2.52]$
128	72	$+0.83$ $[+0.22,+1.41]$	$+2.58$ $[+1.89,+3.29]$	$+1.75$ $[+0.98,+2.50]$

Appendix

Table 14: Paired sensitivity after excluding five tasks with changed source-eligibility grades in the reconstructed audit. The selected five RF outputs and SVD-5 input each have 16 columns. Entries are AUC-point effects with nominal 95% publication-cluster intervals.

$n$	Effect	Nominal publication interval	Tasks	Publications
16	$-0.33$	[ $-0.71$ , $-0.07$ ]	21	18
32	$+0.11$	[ $-0.22$ , $+0.41$ ]	21	18
64	$+0.18$	[ $-0.07$ , $+0.49$ ]	21	18
128	$-0.06$	[ $-0.31$ , $+0.14$ ]	21	18

Appendix

Table 15: Conditional member contribution in the fixed 24-case sample. Four support draws are averaged within each case, cases within task, then tasks equally; effects are full-minus-member-removal AUC points.

Figure 12: Conditional deletion effects across all 24 sampled source-destination pairs and four support sizes. A , Four-draw mean deletion effect $D_{j}$ (full 16-column portfolio minus 15-column portfolio with output $j$ removed, in ROC-AUC points) across all 96 pair-by-size conditions. Blue cells indicate $D_{j}>0$ (retaining output $j$ improves AUC), red cells indicate $D_{j}<0$ , and hatched grey cells mark the 59 conditions in which output $j$ was not selected ( $D_{j}=0$ across 236 draws). B , Individual draw values and min-max spans across the four paired support draws for each of the 37 active pair-by-size conditions. ChEMBL assay identifiers and protocols are listed in Table 16 .

Source output $\rightarrow$ destination assay	Source assay	Destination assay
ERK2 assay $\rightarrow$ Orexin-2 receptor	CHEMBL3888525	CHEMBL3705823
Fungal DHFR $\rightarrow$ IRAK4	CHEMBL829401	CHEMBL3887893
CDK4/cyclin D1 $\rightarrow$ FGFR3	CHEMBL832404	CHEMBL3880344
ICMT $\rightarrow$ FAK1	CHEMBL1803626	CHEMBL3705306
MC3 $\rightarrow$ mGluR5	CHEMBL896691	CHEMBL4053746
PAF platelet assay $\rightarrow$ JAK3	CHEMBL823404	CHEMBL3707951

Appendix

Table 16: ChEMBL source and destination assay identifiers for the 24 sampled pairs.

Source $\rightarrow$ destination	$n$	Deletion	Replacement	Better
MC4 binding $\rightarrow$ TRPV3	64	+0.74	+0.27	31/80
HCV NS3 assay $\rightarrow$ JAK2	16	+0.65	+1.81	15/80
Fungal DHFR $\rightarrow$ IRAK4	64	+0.10	+0.32	41/80

Appendix

Table 17: Equal-width replacement controls for the three highlighted pairs. Deletion compares the 16-column portfolio against the 15-column leave-one-out portfolio; Replacement compares it against the mean of 20 width-16 random replacements per draw across four draws (in ROC-AUC points). Better counts random alternatives that score higher than the original output across the 80 paired replacement fits (with one exact tie for fungal DHFR).

Source $\rightarrow$ destination	Draw 0	Draw 1	Draw 2	Draw 3
MC4 binding $\rightarrow$ TRPV3	+0.50	+0.31	+0.03	+0.26
HCV NS3 assay $\rightarrow$ JAK2	+4.60	+0.98	+0.95	+0.70
Fungal DHFR $\rightarrow$ IRAK4	-0.40	-0.56	+0.63	+1.62

Appendix

Table 18: Per-draw original-minus-mean-replacement differences (in ROC-AUC points), averaged over the 20 sampled alternatives within each of the four support draws.

Source $\rightarrow$ destination	$n$	Mean	Q1	Median	Q3	Better
MC4 binding $\rightarrow$ TRPV3	64	$+0.27$	$-0.29$	$+0.25$	$+0.81$	31/80
HCV NS3 $\rightarrow$ JAK2	16	$+1.81$	$+0.44$	$+1.30$	$+2.18$	15/80
Fungal DHFR $\rightarrow$ IRAK4	64	$+0.32$	$-0.51$	$-0.02$	$+0.87$	41/80

Appendix

Table 19: Distribution of original-minus-replacement AUC differences at fixed width. Mean, quartiles and median are AUC points; negative differences favor the random replacement.

$n$	Semantic percentile	Lexical percentile
16	$55.5$	$51.5$
32	$48.3$	$49.7$
64	$55.9$	$59.8$
128	$51.6$	$47.4$

Appendix

Table 20: Task-equal mean distance percentiles of selected members. A uniformly drawn eligible member has expectation 50; larger values mean more distant.

Added head	Regret reduction	Required	Passed
MiniLM frozen residual	$+0.02684$	$+0.10000$	No
Laya frozen residual	$-0.00902$	$+0.10000$	No
Labeled-support residual	$+0.03863$	$+0.10000$	No
Label-invariant geometry	$+0.04326$	$+0.10000$	No

Appendix

Table 21: Out-of-publication source-fold residual-head ablations. Mean outer-fold measured-choice regret reductions are AUC percentage points; the threshold is the prespecified screening criterion.

Molecular Property Prediction

Molecular property prediction estimates chemical, physical, and biological properties from molecular structure and context.

See all 201 papers

Apr 17, 2026

Tabular foundation models for in-context prediction of molecular properties

Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a highly accurate and cost-efficient alternative for property prediction in practical applications.

Karim K. Ben Hicham, Jan G. Rittig, Martin Grohe +1

RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen, Germany. · Lehrstuhl Informatik 7, RWTH Aachen University, Aachen, Germany. · JARA-CSD, Aachen, Germany. +1

Tabular Foundation Models Molecular Property Prediction Tabular ICL Molecular Representation Learning Molecular Foundation Models

Ask PDF

Jun 30, 2026

Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?

Predicting biomolecular properties from limited labeled data is a central bottleneck in protein engineering and small-molecule design. As strong pretrained encoders now supply rich fixed-length representations, the difficulty has shifted from representation learning to building a data-efficient predictor for the few-shot regime. Tabular foundation models such as TabPFN and TabICL are unlikely candidates for this role: they are in-context learners pretrained on synthetic tables drawn from random causal graphs, a generative prior with no obvious correspondence to the processes that produce protein sequences or molecular graphs. That this tabular, causal inductive bias should transfer to biomolecular data at all is counter-intuitive, yet we find it does. Treating each method as a predictor-representation pair, we evaluate across two domains. We find that on protein fitness regression tasks these in-context learning models coupled with ESM Cambrian representations achieve or exceed state-of-the-art results on ProteinGym, and outperform task-specific supervised regressors on a diverse esterase catalytic activity dataset. For small-molecule classification with ECFP/RDKit descriptors, no single predictor-representation pairing dominates across TDC ADMET, MoleculeNet, FS-Mol, and DrugOOD, but they are competitive with the existing task-specific state-of-the-art. Crucially, on both protein and small-molecule few-shot tasks, these predictor-representation pairs offer strong performance. We conclude that tabular foundation models can be strong biomolecular predictors, but only when coupled with expressive representations.

Davy Guan, Lu Zhang, Asiri Wijesinghe +7

Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia

Tabular Foundation Models Protein Fitness Prediction Few-Shot Learning Molecular Property Prediction Tabular ICL

Ask PDF

Apr 17, 2026

A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era

Molecular property prediction integrates quantum chemistry, cheminformatics, and deep learning to connect molecular structure with physicochemical and biological behavior. This survey traces four complementary paradigms, including Quantum, Descriptor Machine Learning, Geometric Deep Learning, and Foundation Models, and outlines a unified taxonomy linking molecular representations, model architectures, and interdisciplinary applications. Benchmark analyses integrate evidence from both widely used datasets and datasets reflecting industry perspectives, encompassing quantum, physicochemical, physiological, and biophysical domains. The survey examines current standards in data curation, splitting strategies, and evaluation protocols, highlighting challenges including inconsistent stereochemistry, heterogeneous assay sources, and reproducibility limitations under random or poorly defined splits. These observations motivate the modernization of benchmark design toward more transparent, time- and scaffold-aware methodologies. We further propose three forward-looking directions: (i) physics-aware learning embedding quantum consistency, (ii) uncertainty-calibrated foundation models for trustworthy inference, and (iii) realistic multimodal benchmark ecosystems integrating computational and experimental data. Repository: https://github.com/Zongru-Li/Survey-and-Benchmarks-of-DL-for-Molecular-Property-Prediction-in-the-Foundation-Model-Era.

Zongru Li, Xingsheng Chen, Honggang Wen +8

The University of Hong Kong, Hong Kong SAR · Nanyang Technological University, Singapore · University of Cambridge, United Kingdom +4

Molecular Property Prediction Molecular Foundation Models Molecular Representation Learning

Ask PDF

Abstract

Molecular Property Prediction

Explore similar work

Tabular foundation models for in-context prediction of molecular properties

Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?

A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era