Agentic discovery of blood biomarker from distilled private health records

Seffi Cohen; Liat Antwarg Friedman; Amir Anisman; Ruth Johnson; Michelle M. Li; Ayush Noori; Ben Reis; Ran Balicer

cs.AIOct 3, 2026

Agentic discovery of blood biomarker from distilled private health records

Authors: Seffi Cohen, Liat Antwarg Friedman, Amir Anisman, Ruth Johnson, Michelle M. Li, Ayush Noori, Ben Reis, Ran Balicer, +2 more

Organizations: Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · The Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute, Boston, MA, USA · Clalit Research Institute, Innovation Division, Clalit Health Services, Ramat-Gan, Israel · Department of Epidemiology, Biostatistics and Community Health Sciences. Ben-Gurion University of the Negev, Be’er Sheva, Israel · Computer and Information Science, Ben Gurion University of the Negev, Be’er Sheva, Israel

Abstract

Routine complete blood counts (CBCs) could yield new biomarkers, but the private records needed to evaluate candidates cannot be shared with frontier language model agents that excel at discovery. We distilled the evidence held in the Clalit Health Services panel of over 5.4 million patients into a released scoring tool: for each of 13 immune-mediated diseases, a graph attention network was trained inside the data boundary to predict the case-control AUC of candidate CBC expressions, and only the trained weights were released. The tool grounds an agent's propose-score-refine loop in real-world data without exposing any patient data. In external validation, agent-discovered expressions improved on their literature-seeded starting points by a median of 4.18 AUC percentage points, and across three independent cohorts, reranking the candidates of three frontier research tools improved on their first choices in most comparisons, with gains that varied by cohort. The released scorer supports privacy-preserving biomarker hypothesis generation.

Figures & tables

Figure 1: Privacy-preserving public scoring framework for CBC biomarker discovery. A) Private clinical evidence remains inside the Clalit data boundary: anonymized EHR-derived cohorts, CBC measurements, and disease labels are used only within the protected environment. B) Synthetic CBC expression trees are generated from CBC variables and arithmetic operators, with leaves restricted to measured CBC features and no numeric literals. C) Disease-specific graph-attention networks learn to predict expression-level AUC from expression structure, producing compact scoring checkpoints. D) Only trained GAT weights are exported; patient rows, raw CBC values, and expression-level AUC labels remain locked inside the institutional boundary. E) A public frontier artificial intelligence (AI) agent uses the released scorer in a propose–score–rank–refine loop to prioritize interpretable CBC biomarker expressions.

ICD	Disease	$n_{\mathrm{Clalit}}$	$n_{\mathrm{MIMIC}}$	$n_{\mathrm{EHRShot}}$	$n_{\mathrm{NHANES}}$
242	Hyperthyroidism	6,138 (3,069)	73,171 (233)	3,308 (51)	49,510 (4,808) †
250	Type 1 Diabetes (T1D)	25,314 (12,657)	72,140 (306)	3,186 (88)	52,231 (6,057) †
277	Familial Mediterranean Fever (FMF)	9,533 (4,766)	29,929 (0)	2,568 (3)	n.a.
340	Multiple Sclerosis (MS)	5,960 (2,980)	72,712 (126)	2,815 (14)	n.a.
555	Crohn’s Disease	20,513 (10,256)	71,091 (243)	2,591 (30)	n.a.
556	Ulcerative Colitis (US)	20,385 (10,192)	69,635 (210)	2,490 (47)	n.a.

Table 1: Cohorts and disease panel.

Figure 2: Scorer-training agreement. a) Per-disease Spearman correlation between the scorer’s predicted AUC and the ground-truth AUC on the held-out synthetic-expression split. b) Predicted score versus the ground-truth AUC for the agent-proposed candidate expressions of a representative disease (rheumatoid arthritis).

ICD	Disease	Epochs	Val Spearman	Val MAE(AUC)	NDCG@5000
242	Hyperthyroidism	67	0.836	0.0133	0.998
250	Type 1 diabetes	106	0.933	0.0055	0.998
277	Familial Mediterranean fever	88	0.930	0.0078	0.999
340	Multiple sclerosis	90	0.801	0.0096	0.991
555	Crohn’s disease	109	0.975	0.0072	0.997
556	Ulcerative colitis	173	0.974	0.0071	0.999

Table 2: Per-disease scorer-training diagnostics on the held-out synthetic-expression split.

Disease	MIMIC-IV	EHRShot	NHANES
Hyperthyroidism	0.527 [0.503, 0.564] / 0.602 [0.572, 0.636]	0.586 [0.511, 0.655] / 0.551 [0.503, 0.620]	n.a.
Type 1 Diabetes	0.529 [0.503, 0.558] / 0.568 [0.540, 0.596]	0.514 [0.502, 0.581] / 0.560 [0.511, 0.622]	n.a.
Multiple Sclerosis	0.545 [0.504, 0.596] / 0.570 [0.525, 0.610]	0.581 [0.506, 0.694] / 0.553 [0.504, 0.693]	n.a.
Crohn’s Disease	0.535 [0.505, 0.569] / 0.570 [0.532, 0.607]	0.524 [0.501, 0.623] / 0.523 [0.502, 0.624]	n.a.
Ulcerative Colitis	0.532 [0.502, 0.565] / 0.572 [0.536, 0.604]	0.573 [0.507, 0.642] / 0.622 [0.528, 0.697]	n.a.
Psoriasis	0.515 [0.501, 0.543] / 0.597 [0.566, 0.627]	0.558 [0.508, 0.627] / 0.600 [0.529, 0.660]	0.592 [0.565, 0.617] / 0.570 [0.543, 0.597]

Table 3: External validation per disease and cohort.

Disease	Cohort	$n$	$n_{\mathrm{pos}}$	[email protected]%	prec@1%	prec@5%	enrich@1%
Hyperthyroidism	MIMIC-IV	73,171	233	0.27%	0.82%	0.52%	2.57 $\times$
	EHRShot	3,308	51	0.00%	0.00%	1.82%	0.00 $\times$
Type 1 diabetes	MIMIC-IV	72,140	306	0.28%	0.14%	0.33%	0.33 $\times$
	EHRShot	3,186	88	0.00%	3.12%	3.77%	1.13 $\times$
Familial Mediterranean fever	MIMIC-IV	29,929	0	n.a.	n.a.	n.a.	n.a.
	EHRShot	2,568	3	0.00%	0.00%	0.78%	0.00 $\times$

Table 4: Precision-at-K screening metrics for the discovered expressions, by external cohort and disease.

Disease	CBC feature	Research	3 Pro	BM Agent	ranked	(scorer $-$ single)
	Best single	GPT Deep	Gemini	SciSpace	Scorer-	$\Delta$ auc pp
Hyperthyroidism	0.584	0.552	0.535	0.584	0.587	$+0.25$
Type 1 Diabetes	0.594	n.a.	0.576	0.597	0.609	$+1.44$
Familial Mediterranean Fever	0.619	0.586	0.586	0.599	0.635	$+1.65$
Multiple Sclerosis	0.560	0.584	0.576	0.576	0.587	$+2.64$
Crohn’s Disease	0.608	0.631	0.631	0.615	0.645	$+3.69$

Table 5: Best single CBC feature (maximum over 14 features), the best candidate from each frontier LLM tool (maximum over that tool’s 5 candidates), and the best scorer-ranked candidate (maximum over 25 candidates), all evaluated on the same held-out development-cohort split.

ICD	Disease	Iters	Starting expr	Discovered	$\Delta$ auc pp	Audit class
242	Hyperthyroidism	10	0.527	0.602	$+7.40$	TOPUP_NEEDED
250	Type 1 Diabetes	6	0.529	0.568	$+3.87$	TOPUP_NEEDED
277	Familial Mediterranean Fever	5	0.684	0.729	$+4.48$ †	TOPUP_NEEDED
340	Multiple Sclerosis	6	0.545	0.570	$+2.47$	TOPUP_NEEDED
555	Crohn’s Disease	5	0.535	0.570	$+3.49$	TOPUP_NEEDED
556	Ulcerative Colitis	15	0.532	0.572	$+3.97$	AUDIT_OK

Table 6: Per-disease outcomes of the model-guided search.

ICD	Disease	Starting expression	Discovered expression	GAT score	Primary-cohort AUC [95% CI]
242	Hyperthyroidism	PLT*NEUTpct/LYMpct	(MCHHB+HCT-RDW)(HB+MCV*MCHC)	0.607	0.602 [0.572, 0.636] (MIMIC-IV)
250	Type 1 diabetes	(PLT*NEUTpct)/LYMpct	(WBC+RDW+EOS_pct)/(LYMpct+HB+HCT)	0.592	0.568 [0.540, 0.596] (MIMIC-IV)
277	Familial Mediterranean fever	NEUTpct/LYMpct	RDWNEUTpct(RDW+RDW)/(MCV-RDW)	0.613	n.a. †
340	Multiple sclerosis	NEUTpct/LYMpct	HB*RBC+LYMpct-NEUTpct	0.573	0.570 [0.525, 0.610] (MIMIC-IV)
555	Crohn’s disease	NEUTpct/LYMpct	(PLTRDW+NEUTpctRDW)/(LYMpct*HB+LYMpct)	0.658	0.570 [0.532, 0.607] (MIMIC-IV)
556	Ulcerative colitis	NEUTpct/LYMpct	(PLTRDW-LYMpctHB)(RDW(RDW+RDW+NEUTpct))/(LYMpct*HB-MONOpct)	0.666	0.572 [0.536, 0.604] (MIMIC-IV)

Table 7: Discovered CBC biomarker expressions, top 1 per disease.

Figure 3: External validation of the starting expression and the discovered expression on three independent cohorts. a) MIMIC-IV. b) EHRShot. c) NHANES, restricted to the two verified-diagnosis phenotypes. Grey circles mark the starting expression and blue squares the discovered expression; horizontal bars are bootstrap 95% confidence intervals, and hatched rows ( n.a. ) mark cells without usable cohort labels.

Figure 4: Iteration trajectory, AUC-pp lift, and plateau behavior across 13 immune-mediated diseases. a) Running-best predicted-AUC score over iterations

k

for each disease, coloured by a four-outcome audit summary; the legend labels correspond to the audit classes of Table 6 (converged = audit_ok , plateaued early = terminate_early , score drifted = terminate_late , needs more iteration = topup_needed ). b) Horizontal bar chart of

\Delta

AUC from iteration 1 to the chosen iteration. c) Fraction of total predicted-score gain realised by iteration

k

. Thirteen thin grey lines, one per disease; the black line is the median across diseases.

ICD	Disease	Iters	Starting expr	Discovered	$\Delta$ auc pp
242	Hyperthyroidism	15	0.553	0.519	$-3.41$
250	Type 1 Diabetes	15	0.531	0.525	$-0.62$
277	Familial Mediterranean Fever	15	0.684	0.677	$-0.71$
340	Multiple Sclerosis	15	0.545	0.512	$-3.30$
555	Crohn’s Disease	15	0.535	0.552	$+1.69$
556	Ulcerative Colitis	9	0.532	0.532	$0.00$

Table 8: Random-scoring control: per-disease outcomes when the trained GAT scorer is replaced by a uniform random tool.

Figure 5: Explanation of the discovered CBC biomarkers. Each discovered expression is drawn with every blood-count leaf coloured by its signed Shapley contribution to the out-of-sample MIMIC-IV disease prediction (red, higher level raises predicted risk; blue, lowers it), scaled to the strongest driver within that expression; the triangle marks the direction.

Disease	AUC	Exp.	Surp.	Dominant driver	Notable surprising
Systemic lupus	0.676	6	0	WBC $\downarrow$	—
Rheumatoid arthritis	0.615	5	1	RDW $\uparrow$	MCHC $\downarrow$
Hashimoto thyroiditis	0.606	3	2	LYM% $\uparrow$	LYM% $\uparrow$ , HCT $\uparrow$
Hyperthyroidism	0.602	4	1	HB $\downarrow$	RDW $\uparrow$
Psoriasis	0.597	4	4	EOS% $\uparrow$	BASO% $\uparrow$ , HB $\uparrow$
Sjögren’s syndrome	0.596	4	1	RDW $\uparrow$	MCV $\uparrow$

Table 9: Shapley attribution of each discovered biomarker’s components and their consistency with the clinical literature.

Figure 6: Synthetic CBC expression training data. a) Universe of distinct depth-stratified trees over the 14-feature CBC vocabulary $\times$ 4 arithmetic operators b) Approximate depth fractions at sampling.

Explore similar work

BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies
Oct 6, 2026David Li, Shaamil Karim, Christian GensbiglerAI Agent Benchmarks Data Analysis Agents
University of Waterloo
If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data
Apr 18, 2026Yanjun Cui, Ali Emami, Temiloluwa Prioleau +1Privacy-Preserving Language Models Healthcare
Dartmouth College · Emory University
BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine
May 7, 2026Remigiusz Kinas, Joanna Krawczyk, Rafał Powalski +6Multi-Agent Orchestration AI Agents for Scientific Discovery
Ingenix.AI, Warsaw, Poland

Oct 6, 2026J/K move · Enter open · S save

BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies

David Li, Shaamil Karim, Christian Gensbigler

University of Waterloo

AI Agent Benchmarks Data Analysis Agents Scientific Reproducibility Biomedical IR

Open paper AskPDF arXiv

We evaluate whether AI agents can match the reported findings of published biomedical studies using public data. Existing evaluations do not consistently separate analysis from prior knowledge or retrieval of the published answer. We introduce BioStudyBench, a benchmark of 25 long-horizon analysis tasks drawn from studies first published between July and September 2026, after the developer-reported knowledge cutoffs of the models we evaluate, semi-automatically filtered down from 404,019 PubMed records. In each task, the agent receives a neutral research question but no data files, so it must find and download the relevant public data, search the literature through tools that return only records dated before its cutoff, and report findings through data analysis. To measure gains over prior knowledge, we run every task both with and without access to data and tools. Across eight models, access to data and tools raises the pass rate by 47 percentage points on average over the no-data baseline. Open-weight models across sizes trail closed-weight models, with the best open-weight model passing 81.3% of tasks against 94.7% for the best closed-weight model.

Abstract

Explore similar work

BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies

If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data

BioResearcher: Scenario-Guided Multi-Agent for Translational Medicine