Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity

Michal Kmicikiewicz; Tommy Rochussen; Vincent Fortuin; Ewa Szczurek

cs.LGOct 5, 2026

Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity

Authors: Michal Kmicikiewicz, Tommy Rochussen, Vincent Fortuin, Ewa Szczurek

Organizations: Institute of AI for Health, Helmholtz Munich · School of Computation, Information and Technology, Technical University of Munich · Munich Center for Machine Learning · Helmholtz AI · Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg · Faculty of Mathematics, Informatics and Mechanics, University of Warsaw · Faculty of Mathematics, Informatics and Statistics, LMU Munich

Abstract

Accurate bioactivity prediction is a central challenge in early-stage drug discovery, as individual assays often contain too few measurements to train reliable models independently. Meta-learning offers a principled approach to this few-shot setting, but assay heterogeneity may limit its effectiveness. Here, we test this hypothesis and show that meta-learning performance degrades as meta-training tasks become more heterogeneous. To address this, we introduce MetaHeta, a meta-learning framework that accounts for assay heterogeneity by conditioning predictions on auxiliary data from related assays, with relatedness defined flexibly from available assay information. The architecture of MetaHeta combines linear attention over large auxiliary datasets with exact attention over scarce task-specific context, enabling efficient scaling to the former without compromising exact attention over the latter. We demonstrate the benefits of our approach on assays from ChEMBL and BindingDB, improving few-shot bioactivity prediction and downstream compound prioritization in retrospective Bayesian optimization.

Figures & tables

Figure 1: Failure mode of meta-learning under assay heterogeneity. A heterogeneous meta-dataset can induce a broad prior, such that limited task-specific data are insufficient to concentrate the posterior around functions relevant to a new assay.

Figure 2: Architecture overview of MetaHeta . A shared input MLP embeds context, target, and concatenated auxiliary observations, augmented with task-of-origin indicators, with target activities masked to $0$ . Encoder layers use exact self-attention for the context and FAVOR+ for auxiliary data, followed by context-to-auxiliary and target-to-context cross-attention. When $\mathcal{D}^{\mathrm{aux}}=\varnothing$ , the auxiliary branch is skipped. Finally, an output MLP parameterizes a mean-field Gaussian predictive distribution. Formal definitions of the attention operators are given in Appendix F .

Figure 3: Effect of meta-training task heterogeneity on few-shot bioactivity prediction. Performance of all the considered methods generally deteriorates as

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

is augmented with progressively less related tasks. MetaHeta remains the best-performing approach regardless of the training regime.

	Context size = 4		Context size = 8		Context size = 16		Context size = 32
Method	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$
TNP	$0.611\pm 0.019$	$-0.077\pm 0.029$	$0.827\pm 0.004$	$0.110\pm 0.025$	$0.900\pm 0.002$	$0.226\pm 0.020$	$0.931\pm 0.001$	$0.309\pm 0.015$
ANP	$0.601\pm 0.014$	$-0.001\pm 0.002$	$0.812\pm 0.004$	$0.025\pm 0.035$	$0.899\pm 0.001$	$0.228\pm 0.011$	$0.931\pm 0.001$	$0.321\pm 0.007$
CNP	$0.609\pm 0.014$	$0.124\pm 0.010$	$0.827\pm 0.004$	$0.156\pm 0.004$	$\mathbf{0.901\pm 0.002}$	$0.252\pm 0.016$	$0.930\pm 0.001$	$0.313\pm 0.006$
ADKF-IFT	$0.262\pm 0.027$	$0.028\pm 0.003$	$0.760\pm 0.007$	$0.079\pm 0.013$	$0.882\pm 0.002$	$0.174\pm 0.013$	$0.925\pm 0.002$	$0.282\pm 0.019$
DKT	$0.121\pm 0.018$	$0.021\pm 0.002$	$0.646\pm 0.011$	$0.071\pm 0.002$	$0.806\pm 0.006$	$0.158\pm 0.012$	$0.874\pm 0.003$	$0.285\pm 0.010$

Table 1: Few-shot prediction on EPHB test tasks after meta-training on

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

. MetaHeta matches or exceeds all baselines across both metrics and all context sizes.

	Context size = 4		Context size = 8		Context size = 16		Context size = 32
Method	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$
TNP	7.406 $\pm$ 2.519	-0.070 $\pm$ 0.070	2.381 $\pm$ 0.216	-0.048 $\pm$ 0.037	1.537 $\pm$ 0.026	0.153 $\pm$ 0.010	1.303 $\pm$ 0.028	0.199 $\pm$ 0.017
ICICL-TNP	2.291 $\pm$ 0.112	0.069 $\pm$ 0.038	1.786 $\pm$ 0.022	0.117 $\pm$ 0.020	1.485 $\pm$ 0.007	0.155 $\pm$ 0.020	1.362 $\pm$ 0.044	0.131 $\pm$ 0.026
ANP	8.020 $\pm$ 4.795	-0.048 $\pm$ 0.047	2.718 $\pm$ 0.265	-0.071 $\pm$ 0.045	1.697 $\pm$ 0.075	0.056 $\pm$ 0.044	1.418 $\pm$ 0.098	0.131 $\pm$ 0.057
CNP	5.945 $\pm$ 2.766	-0.176 $\pm$ 0.043	2.399 $\pm$ 0.229	-0.013 $\pm$ 0.010	1.715 $\pm$ 0.028	0.063 $\pm$ 0.003	1.472 $\pm$ 0.029	0.120 $\pm$ 0.008
ADKF-IFT	22.605 $\pm$ 9.400	0.060 $\pm$ 0.001	4.509 $\pm$ 0.374	0.110 $\pm$ 0.005	2.235 $\pm$ 0.029	0.173 $\pm$ 0.003	1.582 $\pm$ 0.052	$\mathbf{0.265\pm 0.009}$

Table 2: Performance comparison on FS-Mol. MetaHeta ranks best across seven of eight metric–context-size combinations, and ranks second on the remaining one.

Figure 4: Retrospective Bayesian optimization trajectories on eight BindingDB assays. Dashed horizontal lines ( ) mark the activity threshold for the top 5% of compounds in each assay; MetaHeta is the only method to identify such compounds in every assay.

Figure 5:

R^{2}_{os}

and inference time of MetaHeta and ICICL-TNP with varying cardinality of

\mathcal{D}^{\mathrm{aux}}

. Auxiliary conditioning improves predictive performance with modest incremental cost. Our architecture processes five auxiliary datasets faster than ICICL-TNP processes a single one.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Regime	$\rho_{\mathrm{act}}$	$d_{\mathrm{site}}$
EPHA	0.466	0.149
TK	0.316	0.367
EPHA + non-TK	0.181	0.512

Appendix

Table 3: Quantitative characterization of kinase relatedness across heterogeneity regimes.

\rho_{\mathrm{act}}

: average pairwise Pearson correlation between kinase activity profiles over the PKIS2 compound panel;

d_{\mathrm{site}}

: average pairwise normalized Hamming distance across the 29 active-site residues identified by Sheridan et al. (2009) .

Figure 6: Effect of meta-training task heterogeneity on few-shot bioactivity prediction. Performance of all the considered methods generally deteriorates as

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

is augmented with progressively less related tasks. MetaHeta remains the best-performing approach regardless of the training regime.

Method	Average rank
TNP	4.375
ANP	5.375
CNP	5.375
ADKF-IFT	7.250
DKT	6.625
TabPFN	4.000

Appendix

Table 4: Average rank across retrospective Bayesian optimization tasks. For each assay, methods are ranked according to their final best-observed activity averaged over five random seeds; ranks are then averaged across the eight assays.

Figure 7: NLL of MetaHeta and ICICL-TNP with varying cardinality of

\mathcal{D}^{\mathrm{aux}}

Explore similar work

AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing
Sep 29, 2026Dong Xu, Zhangfan Yang, Jiantao Wu +5Molecular Property Prediction Label-Free
School of Artificial Intelligence, Shenzhen University · EasternDawn · School of Computer Science, University of Nottingham Ningbo
ScreenShot: A Foundation Model for Few-Shot Combination Drug Screening
Aug 12, 2026Antoine de Mathelin, Christopher Tosh, Wesley TanseyDrug-Drug Interactions Clinical Prediction
Computational Oncology Memorial Sloan Kettering Cancer Center New York, NY 10065
DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks
Mar 12, 2025Wei Cui, Tongzi Wu, Jesse C. Cresswell +2Meta-Learning Few-Shot Learning
Layer 6 AI, Toronto, Canada

Sep 29, 2026, cs.AIJ/K move · Enter open · S save

AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing

Dong Xu, Zhangfan Yang, Jiantao Wu, Shipeng Zhang+4

School of Artificial Intelligence, Shenzhen University · EasternDawn · School of Computer Science, University of Nottingham Ningbo

Molecular Property Prediction Label-Free Clinical Prediction Frozen Prior Knowledge Integration

Open paper AskPDF arXiv

Laboratories often face a new molecular assay with 16-64 labels and a bank of predictors whose training data and parameters are unavailable. The practical question is which frozen outputs to include in a small local model. AssayRouter treats completed assays as pseudo-targets and labels each candidate by its post-fit utility: the reduction in held-out discovery loss when the candidate is added to the local target predictor. A shared regressor learns to predict this utility from candidate behavior on the support set, without source identity; on a new assay, one frozen ranking selects four sources and separate labels fit a convex combiner. We train only on completed ChEMBL-MT assays and evaluate 24 external regression assays across six frozen interface families. AssayRouter-C lowers strict four-call negative log-likelihood (NLL) by 0.0409 relative to Support-CV@4. Frozen candidate-label permutations confirm that candidate-utility correspondence carries the transferred information, and leave-one-interface-out training shows that the mapping generalizes to unseen predictor families. Completed assays therefore provide transferable supervision for scarce-label routing through frozen prediction interfaces.

Abstract

Explore similar work

AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing

ScreenShot: A Foundation Model for Few-Shot Combination Drug Screening

DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks