RECITE: Revising Eligibility Criteria Incorporating Textual Evidence
Organizations: University of California, San Francisco · University of California, Berkeley
Abstract
Overly restrictive eligibility criteria contribute to widespread enrollment failure in clinical trials, yet published recommendations for how specific criteria should be broadened remain disconnected from trial design at scale. We introduce RECITE (Revising Eligibility Criteria Incorporating Textual Evidence), a framework for automated discovery, extraction, and implementation of evidence-based eligibility criteria revisions. RECITE uses protocol amendment documents from ClinicalTrials.gov to construct a dataset of 11,913 curated instances across 5,735 trials, each pairing original criteria with documentary evidence and target revisions. We benchmark fifteen LLM configurations under truncated-to-context-window and retrieval-augmented generation conditions, on a 3,116-instance sample under an LLM-as-judge protocol validated against ten domain experts (judge vs. human quadratic-weighted Cohen's ). The best open-weight model, Gemma 2 9B, achieves -aligned ordinal score on a 0--4 scale (normalized 0.742) and the best closed-weight model, GPT-4o, reaches (normalized 0.745). Using an agentic literature discovery system, we then identified 44 paper--trial matches across five publications providing trial-design and EC-modification evidence. Our findings suggest that existing clinical knowledge can be operationalized into actionable trial design recommendations.
Figures & tables
| Statistic | Value |
|---|---|
| Total instances | 11,913 |
| Unique trials | 5,735 |
| Sampled for benchmark | 3,116 |
| Avg pre-amendment EC len. (chars) | 3,331 |
| Avg amended EC len. (chars) | 3,473 |
| Avg evidence len. (chars) | 134,180 |
| Aggregate | Full ( ) | sim .95 ( ) | ||||||
| Model | Bin Acc | Ord | NormOrd | % 3 | % 4 | % 3 | % 4 | Ret% |
| Base Models | ||||||||
| Qwen2.5-0.5B | .01 .11 | 0.7 0.9 | .179 .22 | 3.8 | 1.4 | 3.2 | 1.0 | 81.1 |
| Qwen2.5-3B | .09 .29 | 2.5 0.9 | .618 .23 | 49.5 | 12.6 | 42.8 | 8.0 | 81.1 |
| Qwen2.5-7B | .18 .39 | 2.8 0.9 | .700 .22 | 63.1 | 22.7 | 56.3 | 16.9 | 81.1 |
| Qwen2.5-7B-128K | .01 .08 | 0.2 0.6 | .050 .15 | 1.3 | 0.7 | 0.9 | 0.4 | 99.4 |
| Source | Des/EC | EC Change Directive | Accrual Impact Quote | Trial (NCT ID) | Enrollment | M / A | |
| Current | Est. | ||||||
| Harvey [ 19 ] | T/T | Allow brain mets, other malignancies, CrCl 30 mL/min | Potential to double eligible patient population | NCT03101995 | 18 | 36 | 4/5 |
| NCT04163432 | 43 | 86 | 4/5 | ||||
| Kim [ 20 ] | T/T | Allow brain mets, other malignancies, lower CrCl to 30 mL/min | Nearly 2 aNSCLC pts eligible (4851, 46.2%) | NCT02792582 | 160 | 320 | 3/4 |
| NCT03367819 | 44 | 88 | 3/4 | ||||
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Score | Relevance | Match | Applicability |
|---|---|---|---|
| 1 | Not relevant to EC or enrollment | No alignment; different conditions | Change implied, requires deep digging |
| 2 | Tangentially relevant; mentions trials broadly | Minimal; broad topic overlap only | Vague hints; may require full paper |
| 3 | Discusses EC but not central focus | Some alignment; similar condition, partial EC overlap | Mentioned but not overt; some interpretation needed |
| 4 | Focuses on EC, enrollment barriers, or participation | Good alignment; same/similar condition, substantial overlap | Directive present, minor interpretation needed |
| 5 | Directly about EC modification or broadening | Strong alignment across all dimensions | Overt in abstract; immediately applicable |
Clinical Information Extraction
Clinical Information Extraction turns clinical notes, reports, and patient narratives into structured, searchable facts and relations.
See all 115 papers