CADS: Measuring the Incremental Contribution of LLM Proposers in Sequential Biological Screens

Guanxing Chen; Nihui Shao; Haohuai He; Zhenchao Tang; Pengwei Hu; Zhi-An Huang

cs.AIOct 17, 2026

CADS: Measuring the Incremental Contribution of LLM Proposers in Sequential Biological Screens

Authors: Guanxing Chen, Nihui Shao, Haohuai He, Zhenchao Tang, Pengwei Hu, Zhi-An Huang

Organizations: City University of Hong Kong (Dongguan) · The Hong Kong Polytechnic University · Tencent AI Lab · Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences

Abstract

Large language models are increasingly used to guide sequential biological screens, yet standard evaluations rely on raw hit counts that conflate a proposer's value with the discovery capacity of the underlying algorithmic harness. Adaptive sampling and corrupted experimental readouts also leave standard feedback comparisons without a false-discovery guarantee. We present Certified Agentic Discovery Screens (CADS), an error-controlled measurement framework that quantifies a proposer's incremental value against an identical LLM-free loop. By integrating capped likelihood-ratio bounds, e-processes, and complete-family e-BH, CADS controls the terminal false discovery rate under the stated null and corruption conditions despite arbitrary dependence, adaptive sampling, and optional stopping. In DepMap-backed simulations, CADS exposes two evaluation artifacts: the LLM-free harness alone clears the feedback gate, and raw origin counts overstate LLM contributions because LLM picks displace fallback discoveries. On the initial 20 seeds, supplying tested-candidate history to Qwen2.5-7B shifts its net contribution from +0.35, with a 95% interval covering zero, to +2.90, with the interval above zero. This initial net effect did not persist in a 50-seed matched-quota replication. These results support net gain over an LLM-free loop, measured between separately certified sets, as the appropriate measure of proposer contribution.

Figures & tables

Figure 1: Counting is not measuring. The same loop (an LLM proposer with a re-selection policy and a random fallback) reads two ways. Counting its hits credits every one to the proposer and carries no error guarantee. Measuring reads it through an FDR certificate, reporting the proposer’s net over an identical no-LLM loop, each loop’s own declarations being FDR-controlled. They disagree sharply: the no-LLM loop alone scores

2.60

, so the net is only

+0.35

under a thin prompt.

Figure 2: CADS separates proposal, evidence, declaration, and evaluation. An arbitrary proposer returns candidate identifiers from a view containing only identifiers and a protocol-released summary of past readouts, which in our runs is high- and low-reading identifier lists. The assay/reporting channel returns numerical evidence. A trusted certificate caps likelihood ratios, accumulates e-processes, fills unobserved registered candidates with neutral e-value one, and applies e-BH to the complete family. Reference effects and labels sit below the truth-isolation barrier and are evaluation-only. Audit-CADS is a separate, conditional path requiring a stationary shared channel, audit sensitivity at least

\kappa

, and test–audit mismatch at most

\rho

Figure 3: CADS validity and proposer measurement. The top row uses the

m{=}6{,}000

sweep registry (

q{=}0.10

; means and 95% Monte Carlo intervals over 1000 paired campaigns in (a)–(b) and 200 in (c)). (a) Naive BH and uncapped LR e-BH exceed the target as contamination grows, whereas CADS stays near zero. (b) CADS pays for validity in power; naive BH’s higher apparent power is invalid. (c) CADS remains below

q

across one T-cell Perturb-seq, one PRISM, and six DepMap 24Q4 references, whereas naive BH does not (identifiers in Appendix D). The bottom row uses a separate

m{=}18{,}159

registry with 20 paired seeds and is never compared across the registry boundary. (d) The no-LLM loop clears the pre-specified feedback gate (

2.60

vs.

0.00

, paired

[1.35,3.85]

). (e) Origin-attributed discoveries (

2.95

) overstate the net over the identical no-LLM loop (

+0.35

) because LLM picks displace fallback discoveries. (f) An informed prompt moves Qwen from

+0.35

+2.90

( ∗ excludes zero); Mistral-Nemo moves in the same direction without excluding zero. The

50

-seed replication in Appendix G does not reproduce this initial magnitude. Zhou–Liu evidence is omitted from (a)–(b) because it makes no discovery and is nonzero only for PRISM in (c) (power

0.127

Method	FDR	MC-SE	Power	MC-SE	True	False
CADS	$1.33\times 10^{-4}$	$2.28\times 10^{-5}$	0.396	$3.76\times 10^{-4}$	245.7	0.033
Naive BH	0.372	$4.28\times 10^{-4}$	0.816	$4.40\times 10^{-4}$	506.9	300.8
Uncapped LR e-BH	0.251	$4.91\times 10^{-4}$	0.711	$4.92\times 10^{-4}$	441.8	147.9
Zhou-Liu evidence	0.000	0.000	0.000	0.000	0.000	0.000

Table 1: Primary operating point (

\varepsilon{=}0.10

q{=}0.10

, 1000 paired seeds). FDR is mean FDP; MC-SE is across campaigns. True/false are mean terminal discovery counts; Appendix D treats all-zero uncertainty.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

$q$	$\varepsilon$	CADS FDR (SE)	CADS power (SE)	Naive FDR	LR FDR
0.05	0.00	$2.84\times 10^{-5}$ ( $1.00\times 10^{-5}$ )	0.458 ( $3.35\times 10^{-4}$ )	0.000	0.000
0.05	0.05	$6.31\times 10^{-5}$ ( $1.57\times 10^{-5}$ )	0.405 ( $3.55\times 10^{-4}$ )	0.131	0.073
0.05	0.10	$5.57\times 10^{-5}$ ( $1.60\times 10^{-5}$ )	0.348 ( $3.99\times 10^{-4}$ )	0.337	0.232
0.05	0.20	$5.61\times 10^{-5}$ ( $1.98\times 10^{-5}$ )	0.226 ( $5.36\times 10^{-4}$ )	0.640	0.548
0.05	0.30	$1.79\times 10^{-5}$ ( $1.79\times 10^{-5}$ )	0.093 ( $6.63\times 10^{-4}$ )	0.768	0.718
0.10	0.00	$6.16\times 10^{-5}$ ( $1.44\times 10^{-5}$ )	0.476 ( $3.32\times 10^{-4}$ )	0.001	0.000

Appendix

Table S1: Complete primary DepMap grid. CADS cells give mean (MC-SE) over 1000 paired campaigns. Naive and uncapped-LR columns give mean FDR. All decisions use the full

m=6000

family.

Figure S1: Cross-setting operating points at

\varepsilon=q=0.10

. Dots are seed means and bars are 95% Monte Carlo intervals over 200 paired campaigns. ZL is the matched-schedule Zhou–Liu evidence diagnostic. The vertical dashed line in the FDR panel marks

q=0.10

Reference dataset	Naive FDR	LR FDR	CADS FDR (SE)	CADS power (SE)	ZL power
T-cell Perturb-seq DOP1B/Rest	0.231	0.154	$1.36\times 10^{-4}$ ( $6.03\times 10^{-5}$ )	0.189 ( $7.44\times 10^{-4}$ )	0.000
PRISM ACH-000827	0.151	0.088	$1.21\times 10^{-4}$ ( $3.56\times 10^{-5}$ )	0.741 ( $5.86\times 10^{-4}$ )	0.127
DepMap 24Q4 ACH-000446	0.432	0.303	$2.67\times 10^{-4}$ ( $7.39\times 10^{-5}$ )	0.364 ( $7.78\times 10^{-4}$ )	0.000
DepMap 24Q4 ACH-000662	0.490	0.352	$2.04\times 10^{-4}$ ( $6.67\times 10^{-5}$ )	0.364 ( $9.18\times 10^{-4}$ )	0.000
DepMap 24Q4 ACH-000053	0.531	0.384	$4.51\times 10^{-4}$ ( $7.77\times 10^{-5}$ )	0.471 ( $8.03\times 10^{-4}$ )	0.000
DepMap 24Q4 ACH-000622	0.548	0.402	$4.31\times 10^{-4}$ ( $8.93\times 10^{-5}$ )	0.417 ( $9.75\times 10^{-4}$ )	0.000

Appendix

Table S2: Cross-setting results at

\varepsilon{=}q{=}0.10

(200 paired seeds per row). All procedures use the same complete-family decision code. Values are seed means; Figure S1 gives the dot plot with 95% intervals. ZL denotes the matched-schedule Zhou–Liu evidence diagnostic, not Algorithm 2.

Zhou–Liu implementation	Required burn-in	Matched budget	Mean discoveries
Published Algorithm 2	3,528,000	45,899	0.0
Terminal Eq. (5) evidence diagnostic	–	45,899	0.0

Appendix

Table S3: Strict Zhou–Liu budget comparison over 200 paired seeds. The terminal evidence row is Eq. ( D.1 ) plus e-BH at the matched budget; it is not Algorithm 2.

Figure S2: Conditional Audit-CADS under its shared-channel assumptions. Means and 95% Monte Carlo intervals are shown as the number of one-read randomized controls grows. The first test round uses

U_{0}=1

. Dashed horizontal lines show the oracle-

\varepsilon

and fixed blind-cap power. Coverage is simultaneous across the cap path.

Reference dataset	Audit $\|A\|$	Final bound	Coverage	FDR	Power
DepMap CRISPR	50	0.336 (0.002)	1.000 (0.000)	0.000 (0.000)	0.023 (0.002)
DepMap CRISPR	100	0.271 (0.001)	1.000 (0.000)	0.000 (0.000)	0.078 (0.003)
DepMap CRISPR	200	0.225 ( $9.76\times 10^{-4}$ )	1.000 (0.000)	0.000 (0.000)	0.172 (0.002)
DepMap CRISPR	500	0.187 ( $5.68\times 10^{-4}$ )	1.000 (0.000)	$4.78\times 10^{-6}$ ( $4.78\times 10^{-6}$ )	0.244 (0.001)
DepMap CRISPR	1000	0.168 ( $3.87\times 10^{-4}$ )	1.000 (0.000)	$5.68\times 10^{-6}$ ( $5.68\times 10^{-6}$ )	0.276 ( $7.44\times 10^{-4}$ )
T-cell Perturb-seq	50	0.333 (0.004)	1.000 (0.000)	0.000 (0.000)	0.004 (0.001)

Appendix

Table S4: Audit-size results. Each cell is mean (MC-SE); DepMap uses 1000 seeds and T-cell Perturb-seq uses 200. Here

\varepsilon_{\rm test}=\varepsilon_{\rm audit}=0.10

\delta=0.01

\rho=0

, and the certificate level is 0.09. The audit is given the derived

\kappa=0.977

for the declared class

\Delta=2

; the simulated channel has sensitivity

0.998

and clean false-alarm rate

\alpha_{R}=0.023

$\kappa$	$\rho$	$\varepsilon_{\rm audit}$	Final bound (SE)	Coverage	Power (SE)
0.9	0.02	0.08	0.197 (0.002)	1.000	0.214 (0.004)
0.9	0.05	0.05	0.186 (0.002)	1.000	0.233 (0.003)
0.9	0.10	0.00	0.143 (0.000)	1.000	0.288 (0.001)
0.7	0.02	0.08	0.214 (0.002)	1.000	0.175 (0.005)
0.7	0.05	0.05	0.201 (0.002)	1.000	0.203 (0.004)
0.7	0.10	0.00	0.155 (0.000)	1.000	0.271 ( $9.73\times 10^{-4}$ )

Appendix

Table S5: Boundary-assumption stress at

|A|=200

on DepMap (200 seeds). Each mean with uncertainty is shown as mean (MC-SE). The generator sets

\varepsilon_{\rm audit}=\max(0,\varepsilon_{\rm test}-\rho)

and

\kappa_{\rm actual}=\kappa

; therefore increasing

\rho

also lowers the simulated audit corruption. Rows must not be interpreted as empirical validation of

\kappa

\rho

true clean null $f_{0}^{\prime}$	$\bar{\varepsilon}{=}0.10$	$0.15$	$0.20$	$0.30$
assumed $N(0,s^{2})$	1.0002	0.9043	0.8405	0.7557
null mean $+0.05$	1.0723 $\dagger$	0.9687	0.8994	0.8069
null mean $+0.10$	1.1491 $\dagger$	1.0366 $\dagger$	0.9610	0.8600
null mean $+0.20$	1.3154 $\dagger$	1.1816 $\dagger$	1.0913 $\dagger$	0.9703
null mean $+0.30$	1.4963 $\dagger$	1.3363 $\dagger$	1.2281 $\dagger$	1.0838 $\dagger$
null sd $\times 1.1$	1.0463 $\dagger$	0.9400	0.8693	0.7760

Appendix

Table S6: Clean-null misspecification. Implied

\mathbb{E}_{\rm null}[e]

at true

\varepsilon=0.10

; validity needs

\leq 1

and

\dagger

marks broken. Location and scale families perturb the reference-null pool only.

Channel	Bound	CADS FDR	Power	Naive FDR
frozen $\mathrm{U}(2,4)$ (reference)	holds	0.0001	0.3956	0.3720
cap-saturating point mass	holds	0.0001	0.3958	0.0083
subthreshold (just under $\tau$ )	holds	0.0002	0.3962	0.0204
mixed-sign	holds	0.0000	0.2435	0.1320
history-dependent (re-reads)	holds	0.0095	0.4736	0.7465
target-specific (near boundary)	holds	0.0009	0.4327	0.4304

Appendix

Table S7: Contamination stress, DepMap, 200 paired seeds per channel,

q=0.10

. All channels have marginal rate

0.10

. Only the last violates the per-read conditional bound the theorem requires.

Prompt	Model	Feedback	LLM slots	CADS true	Naive FDR
thin	Qwen2.5-7B	full	57.7	2.95	0.228
thin	Qwen2.5-7B	none	257.5	0.30	0.522
thin	Qwen2.5-7B	permuted	55.4	0.00	0.622
thin	Mistral-Nemo	full	131.5	2.95	0.154
thin	Mistral-Nemo	none	196.8	0.25	0.509
thin	Mistral-Nemo	permuted	167.9	0.00	0.407

Appendix

Table S8: Full proposer grid (

m=18{,}159

; 20 paired seeds per cell). “LLM slots” is the mean number of 260 fresh slots filled with valid unseen symbols; a seeded label-free draw fills the remainder. CADS FDR is mean FDP and equals

0.000

in every cell; naive BH ranges from

0.129

0.628

. Qwen and Mistral comprise the frozen contract; Llama-3.1-8B is an extension under the same protocol. The shipped verifier binds every cell to its frozen shard summary.

Large Language Model-Guided Scientific Discovery

Large language model-guided scientific discovery uses AI to propose, test, and refine hypotheses, models, experiments, and designs.

See all 274 papers

Budget-Sensitive Discovery Scoring: A Formally Verified Framework for Evaluating AI-Guided Scientific Selection
Mar 12, 2026Abhinaba Basu, Pavan ChakrabortyLarge Language Model-Guided Scientific Discovery FDR Control
Indian Institute of Information Technology Allahabad (IIITA) and National Institute of Electronics & Information Technology (NIELIT). · Indian Institute of Information Technology Allahabad (IIITA).
Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens
Sep 10, 2026Carl Edwards, Edward De Brouwer, Xiner Li +5Benchmark Design Active Experimental Design
Genentech, South San Francisco, CA, USA
Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search
Aug 16, 2026Zhongwei Yu, Yan Song, Xue Yan +9Bayesian Optimization Large Language Model-Guided Optimization
The Hong Kong University of Science and Technology (GZ) · University College London · AI Lab, The Yangtze River Delta +3

Mar 12, 2026J/K move · Enter open · S save

Budget-Sensitive Discovery Scoring: A Formally Verified Framework for Evaluating AI-Guided Scientific Selection

Abhinaba Basu, Pavan Chakraborty

Indian Institute of Information Technology Allahabad (IIITA) and National Institute of Electronics & Information Technology (NIELIT). · Indian Institute of Information Technology Allahabad (IIITA).

Large Language Model-Guided Scientific Discovery FDR Control

Open paper AskPDF arXiv

Scientific discovery increasingly relies on AI systems to select candidates for expensive experimental validation, yet no principled, budget-aware evaluation framework exists for comparing selection strategies -- a gap intensified by large language models (LLMs), which generate plausible scientific proposals without reliable downstream evaluation. We introduce the Budget-Sensitive Discovery Score (BSDS), a formally verified metric -- 20 theorems machine-checked by the Lean 4 proof assistant -- that jointly penalizes false discoveries (lambda-weighted FDR) and excessive abstention (gamma-weighted coverage gap) at each budget level. Its budget-averaged form, the Discovery Quality Score (DQS), provides a single summary statistic that no proposer can inflate by performing well at a cherry-picked budget. As a case study, we apply BSDS/DQS to: do LLMs add marginal value to an existing ML pipeline for drug discovery candidate selection? We evaluate 39 proposers -- 11 mechanistic variants, 14 zero-shot LLM configurations, and 14 few-shot LLM configurations -- using SMILES representations on MoleculeNet HIV (41,127 compounds, 3.5% active, 1,000 bootstrap replicates) under both random and scaffold splits. Three findings emerge. First, the simple RF-based Greedy-ML proposer achieves the best DQS (-0.046), outperforming all MLP variants and LLM configurations. Second, no LLM surpasses the Greedy-ML baseline under zero-shot or few-shot evaluation on HIV or Tox21, establishing that LLMs provide no marginal value over an existing trained classifier. Third, the proposer hierarchy generalizes across five MoleculeNet benchmarks spanning 0.18%-46.2% prevalence, a non-drug AV safety domain, and a 9x7 grid of penalty parameters (tau >= 0.636, mean tau = 0.863). The framework applies to any setting where candidates are selected under budget constraints and asymmetric error costs.

Abstract

Large Language Model-Guided Scientific Discovery

Explore similar work

Budget-Sensitive Discovery Scoring: A Formally Verified Framework for Evaluating AI-Guided Scientific Selection

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens

Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search