Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity

Michal Kmicikiewicz; Tommy Rochussen; Vincent Fortuin; Ewa Szczurek

cs.LGOct 5, 2026

Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity

Authors: Michal Kmicikiewicz, Tommy Rochussen, Vincent Fortuin, Ewa Szczurek

Organizations: Institute of AI for Health, Helmholtz Munich · School of Computation, Information and Technology, Technical University of Munich · Munich Center for Machine Learning · Helmholtz AI · Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg · Faculty of Mathematics, Informatics and Mechanics, University of Warsaw · Faculty of Mathematics, Informatics and Statistics, LMU Munich

Abstract

Accurate bioactivity prediction is a central challenge in early-stage drug discovery, as individual assays often contain too few measurements to train reliable models independently. Meta-learning offers a principled approach to this few-shot setting, but assay heterogeneity may limit its effectiveness. Here, we test this hypothesis and show that meta-learning performance degrades as meta-training tasks become more heterogeneous. To address this, we introduce MetaHeta, a meta-learning framework that accounts for assay heterogeneity by conditioning predictions on auxiliary data from related assays, with relatedness defined flexibly from available assay information. The architecture of MetaHeta combines linear attention over large auxiliary datasets with exact attention over scarce task-specific context, enabling efficient scaling to the former without compromising exact attention over the latter. We demonstrate the benefits of our approach on assays from ChEMBL and BindingDB, improving few-shot bioactivity prediction and downstream compound prioritization in retrospective Bayesian optimization.

Figures & tables

Figure 1: Failure mode of meta-learning under assay heterogeneity. A heterogeneous meta-dataset can induce a broad prior, such that limited task-specific data are insufficient to concentrate the posterior around functions relevant to a new assay.

Figure 2: Architecture overview of MetaHeta . A shared input MLP embeds context, target, and concatenated auxiliary observations, augmented with task-of-origin indicators, with target activities masked to $0$ . Encoder layers use exact self-attention for the context and FAVOR+ for auxiliary data, followed by context-to-auxiliary and target-to-context cross-attention. When $\mathcal{D}^{\mathrm{aux}}=\varnothing$ , the auxiliary branch is skipped. Finally, an output MLP parameterizes a mean-field Gaussian predictive distribution. Formal definitions of the attention operators are given in Appendix F .

Figure 3: Effect of meta-training task heterogeneity on few-shot bioactivity prediction. Performance of all the considered methods generally deteriorates as

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

is augmented with progressively less related tasks. MetaHeta remains the best-performing approach regardless of the training regime.

	Context size = 4		Context size = 8		Context size = 16		Context size = 32
Method	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$	F NLL $\bm{\uparrow}$	$F_{R^{2}_{os}}$ $\bm{\uparrow}$
TNP	$0.611\pm 0.019$	$-0.077\pm 0.029$	$0.827\pm 0.004$	$0.110\pm 0.025$	$0.900\pm 0.002$	$0.226\pm 0.020$	$0.931\pm 0.001$	$0.309\pm 0.015$
ANP	$0.601\pm 0.014$	$-0.001\pm 0.002$	$0.812\pm 0.004$	$0.025\pm 0.035$	$0.899\pm 0.001$	$0.228\pm 0.011$	$0.931\pm 0.001$	$0.321\pm 0.007$
CNP	$0.609\pm 0.014$	$0.124\pm 0.010$	$0.827\pm 0.004$	$0.156\pm 0.004$	$\mathbf{0.901\pm 0.002}$	$0.252\pm 0.016$	$0.930\pm 0.001$	$0.313\pm 0.006$
ADKF-IFT	$0.262\pm 0.027$	$0.028\pm 0.003$	$0.760\pm 0.007$	$0.079\pm 0.013$	$0.882\pm 0.002$	$0.174\pm 0.013$	$0.925\pm 0.002$	$0.282\pm 0.019$
DKT	$0.121\pm 0.018$	$0.021\pm 0.002$	$0.646\pm 0.011$	$0.071\pm 0.002$	$0.806\pm 0.006$	$0.158\pm 0.012$	$0.874\pm 0.003$	$0.285\pm 0.010$

Table 1: Few-shot prediction on EPHB test tasks after meta-training on

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

. MetaHeta matches or exceeds all baselines across both metrics and all context sizes.

	Context size = 4		Context size = 8		Context size = 16		Context size = 32
Method	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$	NLL $\bm{\downarrow}$	$R^{2}_{os}$ $\bm{\uparrow}$
TNP	7.406 $\pm$ 2.519	-0.070 $\pm$ 0.070	2.381 $\pm$ 0.216	-0.048 $\pm$ 0.037	1.537 $\pm$ 0.026	0.153 $\pm$ 0.010	1.303 $\pm$ 0.028	0.199 $\pm$ 0.017
ICICL-TNP	2.291 $\pm$ 0.112	0.069 $\pm$ 0.038	1.786 $\pm$ 0.022	0.117 $\pm$ 0.020	1.485 $\pm$ 0.007	0.155 $\pm$ 0.020	1.362 $\pm$ 0.044	0.131 $\pm$ 0.026
ANP	8.020 $\pm$ 4.795	-0.048 $\pm$ 0.047	2.718 $\pm$ 0.265	-0.071 $\pm$ 0.045	1.697 $\pm$ 0.075	0.056 $\pm$ 0.044	1.418 $\pm$ 0.098	0.131 $\pm$ 0.057
CNP	5.945 $\pm$ 2.766	-0.176 $\pm$ 0.043	2.399 $\pm$ 0.229	-0.013 $\pm$ 0.010	1.715 $\pm$ 0.028	0.063 $\pm$ 0.003	1.472 $\pm$ 0.029	0.120 $\pm$ 0.008
ADKF-IFT	22.605 $\pm$ 9.400	0.060 $\pm$ 0.001	4.509 $\pm$ 0.374	0.110 $\pm$ 0.005	2.235 $\pm$ 0.029	0.173 $\pm$ 0.003	1.582 $\pm$ 0.052	$\mathbf{0.265\pm 0.009}$

Table 2: Performance comparison on FS-Mol. MetaHeta ranks best across seven of eight metric–context-size combinations, and ranks second on the remaining one.

Figure 4: Retrospective Bayesian optimization trajectories on eight BindingDB assays. Dashed horizontal lines ( ) mark the activity threshold for the top 5% of compounds in each assay; MetaHeta is the only method to identify such compounds in every assay.

Figure 5:

R^{2}_{os}

and inference time of MetaHeta and ICICL-TNP with varying cardinality of

\mathcal{D}^{\mathrm{aux}}

. Auxiliary conditioning improves predictive performance with modest incremental cost. Our architecture processes five auxiliary datasets faster than ICICL-TNP processes a single one.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Regime	$\rho_{\mathrm{act}}$	$d_{\mathrm{site}}$
EPHA	0.466	0.149
TK	0.316	0.367
EPHA + non-TK	0.181	0.512

Appendix

Table 3: Quantitative characterization of kinase relatedness across heterogeneity regimes.

\rho_{\mathrm{act}}

: average pairwise Pearson correlation between kinase activity profiles over the PKIS2 compound panel;

d_{\mathrm{site}}

: average pairwise normalized Hamming distance across the 29 active-site residues identified by Sheridan et al. (2009) .

Figure 6: Effect of meta-training task heterogeneity on few-shot bioactivity prediction. Performance of all the considered methods generally deteriorates as

\mathcal{D}_{\mathrm{meta}}^{\mathrm{core}}

is augmented with progressively less related tasks. MetaHeta remains the best-performing approach regardless of the training regime.

Method	Average rank
TNP	4.375
ANP	5.375
CNP	5.375
ADKF-IFT	7.250
DKT	6.625
TabPFN	4.000

Appendix

Table 4: Average rank across retrospective Bayesian optimization tasks. For each assay, methods are ranked according to their final best-observed activity averaged over five random seeds; ranks are then averaged across the eight assays.

Figure 7: NLL of MetaHeta and ICICL-TNP with varying cardinality of

\mathcal{D}^{\mathrm{aux}}

Explore similar work

Sep 29, 2026cs.AI

AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing

Laboratories often face a new molecular assay with 16-64 labels and a bank of predictors whose training data and parameters are unavailable. The practical question is which frozen outputs to include in a small local model. AssayRouter treats completed assays as pseudo-targets and labels each candidate by its post-fit utility: the reduction in held-out discovery loss when the candidate is added to the local target predictor. A shared regressor learns to predict this utility from candidate behavior on the support set, without source identity; on a new assay, one frozen ranking selects four sources and separate labels fit a convex combiner. We train only on completed ChEMBL-MT assays and evaluate 24 external regression assays across six frozen interface families. AssayRouter-C lowers strict four-call negative log-likelihood (NLL) by 0.0409 relative to Support-CV@4. Frozen candidate-label permutations confirm that candidate-utility correspondence carries the transferred information, and leave-one-interface-out training shows that the mapping generalizes to unseen predictor families. Completed assays therefore provide transferable supervision for scarce-label routing through frozen prediction interfaces.

Dong Xu, Zhangfan Yang, Jiantao Wu +5

School of Artificial Intelligence, Shenzhen University · EasternDawn · School of Computer Science, University of Nottingham Ningbo

Molecular Property Prediction Label-Free Clinical Prediction Frozen Prior Knowledge Integration

Ask PDF

Aug 12, 2026cs.LG

ScreenShot: A Foundation Model for Few-Shot Combination Drug Screening

Treating patients with combinations of drugs reduces the risk of resistance to any individual drug. Finding effective combinations is difficult because the large search space makes combinatorial screens prohibitively expensive, time consuming, and often technically infeasible. Predictive models can fill this gap, yet existing methods typically require molecular profiling of each sample and per-cohort training, limiting their applicability when time and tissue are scarce. To address this challenge, we introduce ScreenShot, a hierarchical transformer pretrained on 40 drug screening datasets covering 3,700 drugs and 6,000 biological samples, whose architecture mirrors the nested structure of screening data. Given a few-shot context of observations from a new patient, ScreenShot predicts the response of the sample to combination therapies through in-context learning, operating directly on functional measurements with no fine-tuning and no molecular profiling. On four held-out datasets, ScreenShot outperforms all baselines in both prediction accuracy and identification of selectively effective treatments. ScreenShot's internal representations are directly useful for experimental design: we use them to drive a weighted k-means++ active learning strategy that selects which experiments to run, achieving the same hit detection as uniform screening with a third of the budget. Source code and interactive dashboard: https://github.com/tansey-lab/screenshot.

Antoine de Mathelin, Christopher Tosh, Wesley Tansey

Computational Oncology Memorial Sloan Kettering Cancer Center New York, NY 10065

Drug-Drug Interactions Clinical Prediction Combinations Foundation Model

Ask PDF

Mar 12, 2025cs.LG

DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks

Meta-learning represents a strong class of approaches for solving few-shot learning tasks. Nonetheless, recent research suggests that simply pre-training a generic encoder can potentially surpass meta-learning algorithms. In this paper, we first discuss the reasons why meta-learning fails to stand out in these few-shot learning experiments, and hypothesize that it is due to the few-shot learning tasks lacking diversity. We propose DRESS, a task-agnostic Disentangled REpresentation-based Self-Supervised meta-learning approach that enables fast model adaptation on highly diversified few-shot learning tasks. Specifically, DRESS utilizes disentangled representation learning to create self-supervised tasks that can fuel the meta-training process. Furthermore, we also propose a class-partition based metric for quantifying the task diversity directly on the input space. We validate the effectiveness of DRESS through experiments on datasets with multiple factors of variation and varying complexity. The results suggest that DRESS is able to outperform competing methods on the majority of the datasets and task setups. Through this paper, we advocate for a re-examination of proper setups for task adaptation studies, and aim to reignite interest in the potential of meta-learning for solving few-shot learning tasks via disentangled representations.

Wei Cui, Tongzi Wu, Jesse C. Cresswell +2

Layer 6 AI, Toronto, Canada

Meta-Learning Few-Shot Learning Encoders Garments

Ask PDF