EviSearch: Trustworthy Extraction and Synthesis of Clinical Trial Evidence with Agents that Improve with Use

Naman Ahuja; Abhijit Chakraborty; Muhammad Ali Khan; Kaneez Zahra Rubab Khakwani; Mohamad Bassam Sonbol; Irbaz Bin Riaz; Vivek Gupta

cs.CLMar 23, 2026

EviSearch: Trustworthy Extraction and Synthesis of Clinical Trial Evidence with Agents that Improve with Use

Authors: Naman Ahuja, Abhijit Chakraborty, Muhammad Ali Khan, Kaneez Zahra Rubab Khakwani, Mohamad Bassam Sonbol, Irbaz Bin Riaz, Vivek Gupta

Organizations: Arizona State University · Mayo Clinic

Abstract

Structured extraction of evidence from clinical trial publications underpins systematic reviews and clinical guidelines, yet large language models are adopted for it only hesitantly: their outputs are difficult to verify, their use commonly requires transmitting documents to proprietary services, and they do not improve from the corrections their users make. We present EviSearch, a multi-agent system that addresses these three obstacles. Three tool-augmented agents with complementary access to a publication extract every column of an evidence table, and a value is admitted only after an attribution verifier has read it on its cited page, so that every value carries a page-level attribution. Disagreement between independent agents directs human review to the cells most likely to be wrong, and reviewer feedback refines the schema definitions and a curation knowledge base without updating model parameters. The agentic system runs entirely offline on open-weight models. On a clinician-annotated benchmark of randomized-trial publications, EviSearch attributes 100.0% of its values, reaches 91.70% accuracy autonomously, and reaches 95.22% after review of 15.6% of cells, exceeding random review of the strongest single agent at equal effort by 1.75 points.

Figures & tables

Panel of Figure 1 — Figure 1: Architecture of EviSearch. Three agents with complementary access to the publication extract every column; the Reconciliation Agent extracts independently before adjudicating the three candidates. A value is admitted only after the attribution verifier has read it on its cited page, and cells on which the extraction agents disagree are routed to review, whose corrections refine the schema and the curation knowledge base. All model inference within the dotted boundary runs on open-weight models locally.

	Accuracy (%)			Attrib. (%)
System	Run 1	Run 2	Mean	Cited	Verified
Single pass	88.93	89.10	89.01	0.0	0.0
PDF Query Agent (A)	89.42	90.00	89.71	99.7	0.0
Search Agent (B)	92.16	92.37	92.27	100.0	0.0
Oracle selection	93.20	93.23	93.21	99.9	0.0
EviSearch	91.88	91.52	91.70	100.0	98.9

Table 1: Extraction accuracy; the share of stated values citing a page and a quotation (Cited), and the share a second reading of that page reproduces (Verified). Only EviSearch reads each value a second time. Oracle selection takes the better of Agents A and B per cell; the last row assumes the reviewer corrects every erroneous queued cell.

Panel of Figure 2 — Figure 2: Accuracy after review against the fraction of cells reviewed. Triage takes the disagreed or uncorroborated cells first, then those both extraction agents left empty. Random review of a fraction $f$ recovers a fraction $f$ of the errors in expectation.

Schema	Knowledge	Agent A	Agent B	EviSearch
Auto-drafted	guidelines	88.35	90.86	89.91
Reviewed $\times$ 2	guidelines	89.31	91.80	90.49
Reviewed $\times$ 3	guidelines	89.44	91.23	91.04
Reviewed $\times$ 3	+ knowledge	89.71	92.27	91.70

Table 2: Accuracy (%) as the schema and the curation knowledge base are aligned through review (

\times

\times

3: rounds of schema review). Guidelines denotes the fixed extraction guidelines without curated knowledge. Means of two runs.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Agreement-	Provenance-
	gated	gated
Values dropped after a negative reading	2.1	1.1
of which correct (% of dropped)	53	50
Cells corrected by the reading	0.26	0.38
Cells made incorrect by the reading	0.34	0.34
Corroboration mark informative	no	yes

Appendix

Table 3: Effect of the verifier’s negative readings under the two admission policies, as a percentage of all cells unless stated; two runs pooled. Under agreement gating a negative reading excludes the value; under provenance gating it informs the adjudicating agent, and a value it does not reproduce is admitted, marked, and routed to review.

Figure 3: Defining a table. Each drafted definition carries its status: accepted as drafted, edited by the owner, or changed since the previous version. The version is locked before extraction.

Figure 4: The extracted table. Every cell carries its state, verified on the cited page, flagged for review, corrected or confirmed by a reviewer, or not reported, and opens at its attribution when selected.

Figure 5: The review view for one queued cell. The extraction agents disagree (“ADT” against “ADT alone”); the three candidates are shown with their cited pages and the verifier’s mark, and the source page opens with the value highlighted.

Figure 6: The curation knowledge base. Each note states its scope, the agents that receive it, the conventions it replaced, and the reason for its last edit; the fingerprint identifies the version that extraction runs recorded.

Explore similar work

EviStreams: Human-in-the-Loop AI Data Extraction for Systematic Reviews in Medicine
Sep 23, 2026Sai Karthik Kosuri, Ankita Shashikant Bhosale, Michael Glick +2Systematic Review Extraction
Center for Integrative Global Oral Health (CIGOH), Penn Dental Medicine, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania
EGTR-Review: Efficient Evidence-Grounded Scientific Peer Review Generation via Multi-Agent Teacher Distillation
Jun 4, 2026Xinpeng Qiu, Wang Yihu, Zhifeng Liu +2Peer Review Review
Department of Information Management, Peking University · PKU-WUHAN Institute for Artificial Intelligence, Peking University
EVE-Agent: Evidence-Verifiable Self-Evolving Agents
May 21, 2026Yamato Arai, Yuma IchikawaSelf-Evolving Agents Evidence-Grounded Questions
Fujitsu Limited Department of Basic Science · The University of Tokyo · Fujitsu Limited +1

Sep 23, 2026, cs.CLJ/K move · Enter open · S save

EviStreams: Human-in-the-Loop AI Data Extraction for Systematic Reviews in Medicine

Sai Karthik Kosuri, Ankita Shashikant Bhosale, Michael Glick, Alonso Carrasco-Labra+1

Center for Integrative Global Oral Health (CIGOH), Penn Dental Medicine, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania

Systematic Review Extraction Research Automation Human-In-The-Loop Text Corpora

Open paper AskPDF arXiv

Systematic reviews underpin clinical guidelines, yet their data-extraction step is a major expert-labor bottleneck bound by a protocolized workflow: two reviewers extract each study independently, an adjudicator resolves disagreements, and the team keeps an auditable record of how every value was produced. Large language models can assist with extraction, but that assistance must fit established review protocols and preserve reproducibility. We present EviStreams, a live, open-source, no-code web platform that puts review teams in control of AI-assisted extraction at three key stages: program design (a structured decomposition approved before any code runs), field specification (typed field definitions calibrated from a pilot), and extracted predictions (reviewer-blinded dual review with adjudication). Working through a form builder, a domain expert defines typed fields rather than prompts, runs extraction over uploaded PDFs, inspects every value alongside the supporting passage it came from, and resolves a reviewer-blinded dual review into an auditable consensus export. An evaluation across four clinical corpora and three frontier model families, released with the system, shows that extraction quality is shaped far more by the field specification than by the choice of model. EviStreams is live at https://evistreams.com/demo and released under Apache-2.0.

Abstract

Explore similar work

EviStreams: Human-in-the-Loop AI Data Extraction for Systematic Reviews in Medicine

EGTR-Review: Efficient Evidence-Grounded Scientific Peer Review Generation via Multi-Agent Teacher Distillation

EVE-Agent: Evidence-Verifiable Self-Evolving Agents