Agentic induction of common data elements and information extraction from sarcoma case report abstracts
Authors: Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah, Ahmed Reda Bahr, Irina Filchenko, Fabio Dennstädt
Organizations: Institute for Patient-Centered Digital Health, Bern University of Applied Sciences, Bern, Switzerland · Faculty of Medicine, University of Geneva, Geneva, Switzerland · Institute for Patient-Centered Digital Health, Bern University of Applied Sciences, Bern, Switzerland. · Faculty of Medicine, University of Geneva, Geneva, Switzerland. · F. Hoffmann-La Roche AG, Basel, Switzerland · F. Hoffmann-La Roche AG, Basel, Switzerland. · Kantonsspital Aarau, Aarau, Switzerland · Kantonsspital Aarau, Aarau, Switzerland. · University of Alexandria, Alexandria, Egypt · University of Alexandria, Alexandria, Egypt. · Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland. · Data Semantics and Decision Making Institute · Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Department of Digital Medicine, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Data Semantics and Decision Making Institute (DSDMI), St. Gallen, Switzerland. · Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland. · Department of Digital Medicine, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland.
Common data elements (CDEs) support clinical data comparability but are usually defined manually. We investigated whether large language model agents could derive a CDE schema from sarcoma case report abstracts and apply it at scale. An agent proposed 72,909 candidate CDEs from 24,588 PubMed abstracts; agentic consolidation yielded 1,570 CDEs. We retained 53 supported by at least 50 abstracts and attempted evidence-linked extraction across the corpus. Processing succeeded for all retained CDEs in 79.2% of abstracts, including responses indicating non-mention. Two human reviewers approved 83.0% of retained CDEs for relevance, 67.9% for clarity, and 56.6% for correctness. In 100 sampled abstracts, exact agreement with the value extracted by the LLM was 94.9% among 1,116 pairs with a non-empty answer (abstract-level bootstrap 95% CI, 93.0-96.4%). Errors concentrated in anatomical and temporal fields. These findings support agent-assisted schema drafting followed by expert curation before large-scale extraction.
Figures & tables
Figure 1: Illustrative overview of the study workflow. (a) An LLM agent proposed CDEs for each sarcoma case report abstract in an input corpus. Next, these CDEs were consolidated and standardised, resulting in an induced CDE schema. We selected 53 CDEs supported by at least 50 source abstracts. Two domain experts assessed schema correctness, relevance, and clarity. (b) Values for all 53 CDEs were extracted from each abstract using standard prompting. Every extracted answer was linked to an evidence text span in the source abstract. We validated the LLM-derived outputs using a subset of 100 abstracts. Furthermore, we performed a descriptive analysis based on the whole corpus of 24,577 unique abstracts.
Figure 2: Corpus construction and CDE schema induction. (A) Retrieval, screening, and removal of duplicate abstracts. (B) Proposal, deduplication, consolidation, standardisation, and selection of CDEs. Schema induction and information extraction used all 24,588 accepted abstracts; duplicate removal applied only to the descriptive analysis. Exact deduplication retained numeric and categorical CDEs separately, including 30 question texts occurring with both types. Value-set standardisation revised categorical vocabularies without reducing the number of CDEs.
Dimension
Reviewer 1 approval
Reviewer 2 approval
Reviewer agreement
κ
Approval by both reviewers
Correctness
48/53
30/53
35/53 (66.0%)
0.24
30/53 (56.6%)
Relevance
48/53
47/53
46/53 (86.8%)
0.29
44/53 (83.0%)
Clarity
39/53
50/53
36/53 (67.9%)
-0.10
36/53 (67.9%)
Table 1: Reviewer approval and agreement for the 53 retained CDEs.
CDE
Matches
Non-matches
Compared
Agreement (%)
All CDEs
1,059
57
1,116
94.9
Patient sex
75
2
77
97.4
Age at diagnosis
73
1
74
98.6
Histopathologic diagnosis (biopsy)
50
0
50
100.0
Distant metastases
44
2
46
95.7
Tumour recurrence during follow-up
37
0
37
100.0
Table 2: Selected results for answer-level exact agreement.
CDE (unit)
Non-empty
Included
Excluded
Median (IQR)
Patient age (years)
17,917
17,365
552
46 (26–63)
Age at diagnosis (years)
17,770
17,178
592
47 (26–63)
Maximum tumour dimension (cm)
2,578
1,224
1,354
7 (4–12)
Chemotherapy cycles
757
504
253
5 (3–6)
Total radiation dose (Gy)
448
302
146
50.4 (45–60)
Table 3: Descriptive summaries of five separate numeric CDEs in 24,577 unique abstracts.
Figure 3: Coverage of the retained CDEs in 24,577 unique abstracts.
Figure 4: Seed-based consolidation within each pass. For an unprocessed source CDE, embedding-based retrieval provided up to 20 candidates meeting a cosine-similarity threshold of 0.80 as input to the agent. The agent assessed semantic equivalence and created a master CDE from the seed and selected candidates, or retained the seed alone. Source identifiers were preserved, and the contributing source CDEs were marked as processed. The cycle continued until all source CDEs were processed for that pass; this stopping rule did not imply that no further consolidation was possible. The recorded first and second passes yielded 2,183 and 1,570 master CDEs, respectively.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Field
Match
MO
RO
Diff.
n
%
1
Chemotherapy Cycles Administered
4
0
0
0
4
100.0
2
Years To Local Recurrence After Surgery
4
0
0
1
5
80.0
3
Latency Years To Secondary Tumor
7
1
0
2
10
70.0
4
Disease Free Survival Months
18
0
0
3
21
85.7
5
Length Of Follow Up Years
20
0
0
0
20
100.0
6
Maximum Tumor Dimension Cm
8
0
0
0
8
100.0
Appendix
Supplementary Table S1: Answer-level evaluation for all 53 retained CDEs.
ID
Field and unchanged question
Type
Source n
Labels
Filled n (%)
1
Chemotherapy Cycles Administered How many cycles of chemotherapy were administered?
Numeric
51
–
757 (3.1%)
2
Years To Local Recurrence After Surgery How many years after initial surgery did the tumor recur locally?
Numeric
51
–
996 (4.1%)
3
Latency Years To Secondary Tumor How many years after the primary cancer treatment or diagnosis did a secondary tumor develop?
Numeric
83
–
2,467 (10.0%)
4
Disease Free Survival Months What is the duration of disease-free survival (months)?
Numeric
139
–
4,660 (19.0%)
5
Length Of Follow Up Years What is the length of follow-up reported (in years)?
Numeric
100
–
4,801 (19.5%)
6
Maximum Tumor Dimension Cm What is the maximum tumor dimension in centimeters?
Numeric
2,072
–
2,578 (10.5%)
Appendix
Supplementary Table S2: Retained schema and field coverage in 24,577 unique abstracts.
Field
Exact label
Count
Field n
%
15. Did Patient Develop Distant Metastases
No
5,755
11,050
52.1
Yes
5,295
11,050
47.9
16. Tumor Recurrence During Follow Up
Yes
3,956
7,696
51.4
No
3,740
7,696
48.6
17. Was Surgical Resection Performed
Yes
13,397
14,933
89.7
No
1,536
14,933
10.3
Appendix
Supplementary Table S3: Up to five most frequent exact labels per categorical field in 24,577 unique abstracts.
Information extraction from pathology reports is essential for cancer staging, tumor registry population. Yet key data remains embedded in narrative reports, making manual extraction labor-intensive and error-prone. Traditional supervised Natural Language Processing pipelines address this through fully supervised Named Entity Recognition and Relation Extraction, but require expensive manual annotation and suffer cascading failures when upstream entities are missed. In this study, we developed a zero-shot, agentic workflow, and evaluated five open-source generative Large Language Models (LLMs) to populate 13 College of American Pathologists synoptic fields from lung resection pathology reports. We compared them against a state-of-the-art supervised GatorTron NER-RE baseline using a novel, registry-aligned evaluation framework. The baseline achieved Micro-F1of 0.960, while the best zero-shot model (GPT-OSS-20B) achieved Micro-F1 of 0.893 (recall: 0.949), accurately extracting complex relations like Pathologic Stage without task-specific training. These results suggest that open-source, zero-shot agentic LLMs show great potential as a low-cost solution for extracting lung pathology information.
Aman Pathak, Cheng Peng, Mengxian Lyu +8
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · College of Nursing, Florida State University, Tallahassee, FL, USA
MIMIC is one of the most widely used public hospital datasets for machine learning on electronic health records, yet clinically important information, such as past medical history, allergies, and diagnostic findings, remains confined to free-text discharge summaries. Studies requiring this information must therefore build their own extraction pipelines, duplicating substantial effort across projects. We release MIMIC-Note-Extract (MNE), a large-scale dataset that uses a hybrid extraction pipeline combining deterministic parsing with schema-guided large language model generation to transform 16 sections from 300,720 discharge summaries into structured tables. MNE adds information that has no structured MIMIC counterpart and enriches related tables with broader coverage and greater clinical detail, including 10 times more patient-history entries and medication histories for more than twice as many admissions. Validation by medical doctors confirms extraction accuracy above 92% across sections. Using MNE, we construct a clinical decision-making benchmark for LLM agents, demonstrating how reusable structured representations of clinical narratives reduce data curation effort.
Kristian Schwethelm, Jonas Kuntzer, Conrad Ketzer +6
Chair for AI in Healthcare and Medicine, TU Munich (TUM) and TUM University Hospital, Germany · Department for Trauma Surgery, TUM University Hospital and TU Munich, Germany · Department for Diagnostic and Interventional Radiology, TUM University Hospital and TU Munich, Germany +4
Enterprise workflows increasingly rely on agents for schema-guided extraction: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on HuggingFace and GitHub.