Beyond Fixed Acquisition Functions: Temporal Difference Active Learning for Perturbation Discovery

Xudong Zhang; Jianzhe Li; Yitong Tang; Peijie Zhou

cs.LGOct 15, 2026

Beyond Fixed Acquisition Functions: Temporal Difference Active Learning for Perturbation Discovery

Authors: Xudong Zhang, Jianzhe Li, Yitong Tang, Peijie Zhou

Organizations: Center for Machine Learning Research, Peking University, Beijing, China · AI for Science Institute (AISI), Beijing, China

Abstract

Genetic and drug perturbation screens are important tools for elucidating disease mechanisms and validating therapeutic targets, but experimental costs limit the number of candidates that can be tested. Existing perturbation screening methods typically update the surrogate model with feedback while keeping the acquisition function fixed, limiting their ability to adapt selection to different response landscapes. Experimental feedback reveals current hits and informs subsequent selection by updating candidate predictions, providing a training signal for learning how to select experiments. We formulate budgeted perturbation discovery as a sequential decision problem and propose Temporal Difference Active Learning (TDAL), which learns an acquisition function online through temporal difference learning from a single screening trajectory, accounting for current hits and future discoveries. TDAL uses the Hellinger distance between hit probabilities for unmeasured candidates before and after an experiment to weight successor value, and reuses measured outcomes through leave-batch-out (LBO) to construct virtual transitions and alleviate training data scarcity. Across eight public screens spanning single, double, and triple perturbations under varied experimental configurations, TDAL achieves an average rank of 1.63 among eleven methods and ranks first on four tasks, demonstrating its adaptability across response landscapes.

Figures & tables

Figure 1: a, Active learning fits a surrogate model, selects experiments with an acquisition function, and updates the model with new observations (gray arrows). TDAL learns a dynamic acquisition function from accumulated transitions (purple path). b, A surrogate model is fitted to observations. Fixed acquisition functions (Greedy, Uncertainty, and Diversity) use its predictions to select the next batch, yielding different selections across illustrative landscapes. Hexagon fills show responses. Gray and colored outlines mark observed candidates and next batch selections, respectively.

Figure 2: TDAL framework. The surrogate is fitted to measurements, and the acquisition function sums hit probabilities and network outputs to select the next batch. TDAL incorporates new observations, refits the surrogate, and constructs real transitions. These enter replay alongside virtual LBO transitions reconstructed from measurements, training the acquisition function through TD learning with Hellinger distance weighting successor value according to prediction changes.

Figure 3: Discovery performance across tasks. a–h, Mean cumulative hits and recall with pointwise 95% confidence intervals across five seeds. i, Ranks by mean final hits (darker is higher).

Figure 4: Local hit structure and discovery gains. a, Real tasks. b, Noisy multicluster landscapes, with example hit locations. Axes show mean enrichment and discovery gains over Random. Points show mean gains with 95% confidence intervals across five repetitions.

Dataset	Surrogate only	w/o adaptive $w_{t}$	w/o LBO	w/o successor	w/o anchor	Full TDAL
IFN- $\gamma$	$186.4\pm 27.7$	$191.8\pm 41.1$	$222.2\pm 79.8$	$260.0\pm 11.7$	$\underline{262.2\pm 15.3}$	$\mathbf{270.6\pm 6.1}$
IL-2	$253.8\pm 15.2$	$319.6\pm 16.8$	$352.2\pm 11.5$	$\underline{361.4\pm 3.4}$	$342.8\pm 43.2$	$\mathbf{367.4\pm 3.0}$
Kuzmin	$3277.8\pm 35.5$	$3724.6\pm 142.4$	$4096.4\pm 208.1$	$4086.8\pm 89.3$	$\mathbf{4174.4\pm 121.4}$	$\underline{4141.0\pm 194.0}$
Jurkat	$4518.2\pm 13.3$	$4301.2\pm 248.3$	$4510.4\pm 39.3$	$4536.0\pm 15.3$	$\mathbf{4550.2\pm 3.8}$	$\underline{4542.8\pm 4.8}$

Table 1: Core ablations on four tasks. Final cumulative hits across five seeds (mean

\pm

SD). Bold and underlined values mark the highest and second-highest means.

Figure 5: Online learning and candidate reranking. a, Training loss and cumulative discoveries over screening rounds. b, Hit discovery in a t-SNE projection of representations (top) and ranking trajectories (bottom), with a comparison between TDAL and surrogate prediction ranks.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Research task	Surrogate updates	Biological knowledge	Online acquisition updates	Value bootstrapping	Experience augmentation
GeneDisco surrogate baselines	Perturbation screening	$\checkmark$	$\times$	$\times$	$\times$	$\times$
OAE	Perturbation screening	$\checkmark$	$\times$	$\times$	$\times$	$\times$
DiscoBAX	Perturbation screening	$\checkmark$	$\times$	$\times$	$\times$	$\times$
PoH	Perturbation screening	$\checkmark$	$\times$	$\times$	$\times$	$\times$
BioBO	Perturbation screening	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$
BioDiscoveryAgent	Perturbation screening	$\times$	$\checkmark$	$\times$	$\times$	$\times$

Appendix

Table A1: Comparison of active screening and experimental design methods.

\checkmark

\times

indicate whether a method uses each mechanism.

Symbol	Meaning
$\mathcal{X}$	Fixed pool of candidate perturbations.
$e(x)$	Phenotype of candidate $x$ after aligning the effect direction.
$\tau$	Threshold defining the desired phenotypic response.
$z(x)$	Indicator that candidate $x$ meets the hit threshold.
$D_{t}$	Measured data available before selection in round $t$ .
$U_{t}$	Unmeasured candidates available in round $t$ .

Appendix

Table A2: Key notation for the screening objective and TDAL.

Dataset	Perturbation	Phenotype	Hit rule
IFN- $\gamma$ ( Schmidt et al., 2022 )	Single gene	IFN- $\gamma$ production by T cells	Top 5% by absolute score
IL-2 ( Schmidt et al., 2022 )	Single gene	IL-2 production by T cells	Top 5% by absolute score
NK ( Zhuang et al., 2019 )	Single gene	Cancer cell sensitivity to NK cell killing	Top 5% by absolute score
Carnevale ( Carnevale et al., 2022 )	Single gene	T cell proliferation under adenosine suppression	Top 5% by absolute score
Simpson K562 ( Simpson et al., 2023 )	Two genes	K562 cell growth after dual gene repression	Top 5% by raw score
SCHEMATIC ( Fong et al., 2025 )	Two genes	Genetic interactions across cancer cell backgrounds	Bottom 5% by raw score

Appendix

Table C1: Perturbation types, phenotypes, and hit rules for eight tasks.

Dataset	$N$	Pool hits	$n_{0}$	$B$	$T$	$M$
IFN- $\gamma$	17,470	873	128	32	60	2,048
IL-2	17,470	873	128	32	60	2,048
NK	17,533	876	128	32	60	2,048
Carnevale	17,526	876	128	32	60	2,048
Simpson K562	143,345	7,167	1,024	128	200	26,624
SCHEMATIC	11,067	553	128	32	60	2,048

Appendix

Table C2: Candidate pools, hits, and budgets for eight tasks.

Component	Parameter	Value
Surrogate	Hidden widths and dropout	Widths 128, 64, dropout probability 0.2
Surrogate	Training/validation split	80%/20%
Surrogate	Optimizer	Adam, learning rate $10^{-3}$
Surrogate	Training batch and epochs	Batch size 64, at most 100 epochs, early stopping patience 10
Surrogate	MC samples	$K=50$
Acquisition network	Projection and hidden widths	Projection dimension 64, hidden widths 128, 128

Appendix

Table C3: Key TDAL hyperparameters.

Method	IFN- $\gamma$	IL-2	NK	Carnevale
TDAL	$270.6\pm 6.1$ $(31.00\%)$	$367.4\pm 3.0$ $(42.08\%)$	$\underline{109.4\pm 9.3}$ $\underline{(12.49\%)}$	$161.2\pm 8.6$ $(18.40\%)$
PoH	$256.6\pm 14.6$ $(29.39\%)$	$\underline{347.8\pm 6.0}$ $\underline{(39.84\%)}$	$101.0\pm 4.1$ $(11.53\%)$	$145.6\pm 2.7$ $(16.62\%)$
OAE	$194.0\pm 13.0$ $(22.22\%)$	$334.2\pm 6.4$ $(38.28\%)$	$97.2\pm 7.2$ $(11.10\%)$	$124.6\pm 5.8$ $(14.22\%)$
DiscoBAX	$218.0\pm 10.4$ $(24.97\%)$	$266.0\pm 17.7$ $(30.47\%)$	$81.0\pm 5.2$ $(9.25\%)$	$151.8\pm 5.9$ $(17.33\%)$
UCB	$196.4\pm 12.4$ $(22.50\%)$	$331.0\pm 10.2$ $(37.92\%)$	$85.4\pm 6.2$ $(9.75\%)$	$123.8\pm 8.3$ $(14.13\%)$
Expected Improvement	$185.8\pm 16.9$ $(21.28\%)$	$315.4\pm 7.1$ $(36.13\%)$	$92.6\pm 4.1$ $(10.57\%)$	$123.2\pm 8.3$ $(14.06\%)$

Appendix

Table D1: Final cumulative hits and recall on the four single gene perturbation tasks. Entries are mean

\pm

sample SD of cumulative hits across five random seeds, with mean recall in parentheses. The second highest mean in each column is underlined.

Method	Simpson K562	SCHEMATIC	Kuzmin yeast	Horlbeck Jurkat
TDAL	$\underline{6765.4\pm 102.5}$ $\underline{(94.40\%)}$	$160.4\pm 14.2$ $(29.01\%)$	$4141.0\pm 194.0$ $(38.59\%)$	$\underline{4542.8\pm 4.8}$ $\underline{(99.45\%)}$
PoH	$6771.0\pm 75.7$ $(94.47\%)$	$187.2\pm 5.1$ $(33.85\%)$	$\underline{4111.4\pm 42.2}$ $\underline{(38.31\%)}$	$4551.4\pm 1.5$ $(99.64\%)$
OAE	$5951.2\pm 36.4$ $(83.04\%)$	$104.8\pm 10.0$ $(18.95\%)$	$3420.4\pm 83.1$ $(31.87\%)$	$4489.2\pm 23.3$ $(98.27\%)$
UCB	$5955.4\pm 46.0$ $(83.09\%)$	$105.0\pm 6.6$ $(18.99\%)$	$3636.6\pm 157.1$ $(33.89\%)$	$4394.0\pm 103.0$ $(96.19\%)$
Expected Improvement	$5005.2\pm 176.9$ $(69.84\%)$	$117.0\pm 2.9$ $(21.16\%)$	$3013.4\pm 278.6$ $(28.08\%)$	$4134.2\pm 92.5$ $(90.50\%)$
Thompson Sampling	$5903.8\pm 17.1$ $(82.37\%)$	$99.6\pm 6.9$ $(18.01\%)$	$3196.2\pm 164.5$ $(29.78\%)$	$4044.8\pm 48.9$ $(88.55\%)$

Appendix

Table D2: Final cumulative hits and recall on the four main combinatorial perturbation tasks. Entries are mean

\pm

sample SD of cumulative hits across five random seeds, with mean recall in parentheses. The second highest mean in each column is underlined.

Method	Mean rank	Rank SD	Range	Top three	Seed rank SD
TDAL	1.63	0.74	1–3	8/8	0.713
PoH	2.13	1.13	1–4	7/8	0.686
CoreSet	4.63	3.16	1–8	4/8	0.498
OAE	5.38	2.07	3–9	2/8	1.233
UCB	5.63	2.45	3–10	1/8	1.131
TopUncertain	6.00	1.93	3–9	1/8	1.004

Appendix

Table D3: Rank consistency across the eight main tasks. Task ranks are based on mean final discovery counts over five seeds among the eleven common methods. Ties receive average ranks. Rank SD is the sample standard deviation across task ranks. Seed rank SD is the mean of the sample standard deviations across five seeds within each task.

Dataset	Hit enrichment (10 nearest neighbors)	Shared methods / Random	TDAL / Random
NK	$0.83\times$	$0.94\times$	$1.08\times$
Carnevale	$1.25\times$	$1.27\times$	$1.57\times$
SCHEMATIC	$1.51\times$	$1.36\times$	$1.63\times$
Kuzmin	$1.76\times$	$1.28\times$	$1.59\times$
IFN- $\gamma$	$2.53\times$	$2.15\times$	$2.89\times$
IL-2	$3.58\times$	$2.91\times$	$3.61\times$

Appendix

Table E1: Hit clustering and active discovery gains on real tasks.

Method	$1.05\times$	$2.53\times$	$3.83\times$	$10.00\times$	$17.71\times$
TDAL	$11.48\pm 0.39$	$17.33\pm 1.87$	$16.78\pm 1.38$	$18.01\pm 9.40$	$24.28\pm 9.23$
PoH	$11.66\pm 1.05$	$10.83\pm 4.00$	$16.76\pm 2.62$	$22.74\pm 4.19$	$20.82\pm 7.05$
OAE	$11.78\pm 0.60$	$16.49\pm 1.11$	$16.24\pm 0.74$	$16.99\pm 1.07$	$17.17\pm 0.17$
DiscoBAX	$11.51\pm 1.56$	$5.73\pm 1.46$	$5.73\pm 2.92$	$3.15\pm 1.57$	$3.53\pm 2.52$
UCB	$11.73\pm 0.75$	$16.19\pm 1.30$	$16.26\pm 1.09$	$16.60\pm 1.01$	$17.19\pm 0.17$
EI	$11.33\pm 1.62$	$6.34\pm 0.76$	$4.92\pm 1.37$	$2.45\pm 1.85$	$2.27\pm 1.35$

Appendix

Table E2: Final recall (%) on the five multicluster landscapes in Figure 4 b. Column headings give mean hit enrichment. Values are means

\pm

sample standard deviations over five paired repetitions.

Figure E1: Discovery trajectories for all twelve methods in Figure 4 b. Panel titles give mean hit enrichment. Lines show mean recall over five paired repetitions, with shaded 95% confidence intervals. Round 0 is the initial set of 128 candidates. All panels share axis ranges.

Figure F1: Training dynamics and candidate reranking on IL-2 across five random seeds. a, TD loss per round. b, Cumulative discoveries among the 85 hit candidates analyzed. The dashed line marks 30 completed rounds. c–g, Surrogate and TDAL ranks at selection for seeds 0–4, with a logarithmic horizontal axis and a dashed line at surrogate rank 32. Colors indicate selection in rounds 0–9, 10–29, and 30–59.

Seed	Rounds 0–9	Rounds 10–29	Rounds 30–59	Mean TDAL rank	Mean surrogate rank
0	4	7	74	16.6	9,780
1	1	15	69	16.1	9,442
2	2	12	71	16.8	9,574
3	3	18	64	17.2	9,522
4	4	18	63	16.4	9,479

Appendix

Table F1: Discovery stages and selection ranks of the hit candidates analyzed across five seeds. Stage columns count newly discovered candidates. Ranks are computed in the same unmeasured pool at selection and averaged over the 85 candidates.

Method	$B=16$ $T=120$	$B=32$ $T=60$	$B=64$ $T=30$	$B=128$ $T=15$
TDAL	$\mathbf{355.4\pm 7.8}$	$\mathbf{367.4\pm 3.0}$	$\mathbf{353.8\pm 10.1}$	$252.2\pm 59.5$
PoH	$\underline{348.8\pm 7.5}$	$\underline{347.8\pm 6.0}$	$\underline{347.0\pm 11.0}$	$\mathbf{351.4\pm 8.0}$
UCB	$336.6\pm 6.2$	$331.0\pm 10.2$	$340.0\pm 18.4$	$\underline{333.2\pm 15.3}$
BADGE	$263.8\pm 3.6$	$237.0\pm 12.0$	$216.8\pm 9.3$	$205.8\pm 8.5$
CoreSet	$301.2\pm 8.8$	$298.4\pm 4.8$	$297.0\pm 5.8$	$297.2\pm 9.4$

Appendix

Table G1: Batch size comparison at a fixed budget on IL-2. Values are final cumulative hits, reported as mean

\pm

sample standard deviation over five seeds. The highest mean in each column is bold and the second highest is underlined.

Method	$B=64$ $T=400$	$B=128$ $T=200$	$B=256$ $T=100$	$B=512$ $T=50$
TDAL	$\mathbf{6868.2\pm 65.8}$	$\underline{6765.4\pm 102.5}$	$\mathbf{6669.2\pm 153.1}$	$\underline{6638.0\pm 81.9}$
PoH	$\underline{6775.8\pm 41.5}$	$\mathbf{6771.0\pm 75.7}$	$\underline{6655.6\pm 78.8}$	$\mathbf{6670.8\pm 80.0}$
UCB	$5955.6\pm 52.5$	$5955.4\pm 46.0$	$5935.4\pm 50.3$	$5874.0\pm 25.6$
BADGE	$921.2\pm 27.9$	$932.8\pm 13.3$	$1006.8\pm 59.0$	$1015.8\pm 40.4$
CoreSet	$1827.0\pm 38.0$	$1693.6\pm 44.4$	$1522.2\pm 32.9$	$1377.6\pm 27.3$

Appendix

Table G2: Batch size comparison at a fixed budget on Simpson K562. Values are final cumulative hits, reported as mean

\pm

sample standard deviation over five seeds. The highest mean in each column is bold and the second highest is underlined.

Figure G1: Discovery performance and computational cost on IL-2. The horizontal axis shows mean time per round on a logarithmic scale, and the vertical axis shows final hit recall for twelve methods. Circle area is proportional to peak reserved GPU memory. Random uses zero reserved GPU memory and is shown as a hollow circle. Time and memory are means over three seeds over rounds 2–4. Recall is the mean over five seeds in the full screening experiment of 60 rounds.

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 288 papers

Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments
May 11, 2026Andrea Rubbi, Arpit Merchant, Samuel Ogden +4Bayesian Optimization Bayesian Experimental Design
Department of Computer Science and Technology, University of Cambridge, Cambridge, UK · Cambridge Center for AI in Medicine, University of Cambridge, Cambridge, UK · MediaTek Research, Cambridge, UK +1
Soft Strategy Selection for Batch-Mode Active Learning
Oct 4, 2026Rushil Gupta, Romain LopezActive Learning
Department of Computer Science, Courant Institute School of Mathematics, Computing, and Data Science, New York University · Department of Biology, New York University
Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens
Sep 10, 2026Carl Edwards, Edward De Brouwer, Xiner Li +5Benchmark Design Active Experimental Design
Genentech, South San Francisco, CA, USA

May 11, 2026J/K move · Enter open · S save

Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments

Andrea Rubbi, Arpit Merchant, Samuel Ogden, Amir Akbarnejad+3

Department of Computer Science and Technology, University of Cambridge, Cambridge, UK · Cambridge Center for AI in Medicine, University of Cambridge, Cambridge, UK · MediaTek Research, Cambridge, UK+1

Bayesian Optimization Bayesian Experimental Design

Open paper AskPDF arXiv

High-throughput gene perturbation experiments can test several genetic interventions in parallel, yet experimental budgets remain limited. A central goal is hit discovery: identifying as many perturbations as possible whose phenotypic effect exceeds a predefined threshold. Pure exploration strategies are statistically inefficient, wasting budget on low-value regions. Bayesian optimization methods offer a principled alternative but target a single global optimum, over-exploiting dominant modes while neglecting other high-value regions. We formalize hit discovery as a sequential experimental design problem and propose Probability-of-Hit, an acquisition function that directly targets threshold exceedance by ranking candidates according to their posterior probability of being a hit. We prove asymptotic optimality of this approach and demonstrate strong empirical performance on both synthetic benchmarks and real biological immunology datasets, including up to 6.4% improvement over baselines on the Schmidt IL-2 dataset.

Abstract

Active Learning

Explore similar work

Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments

Soft Strategy Selection for Batch-Mode Active Learning

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens