Agentic induction of common data elements and information extraction from sarcoma case report abstracts
Authors: Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah, Ahmed Reda Bahr, Irina Filchenko, Fabio Dennstädt
Organizations: Institute for Patient-Centered Digital Health, Bern University of Applied Sciences, Bern, Switzerland · Faculty of Medicine, University of Geneva, Geneva, Switzerland · Institute for Patient-Centered Digital Health, Bern University of Applied Sciences, Bern, Switzerland. · Faculty of Medicine, University of Geneva, Geneva, Switzerland. · F. Hoffmann-La Roche AG, Basel, Switzerland · F. Hoffmann-La Roche AG, Basel, Switzerland. · Kantonsspital Aarau, Aarau, Switzerland · Kantonsspital Aarau, Aarau, Switzerland. · University of Alexandria, Alexandria, Egypt · University of Alexandria, Alexandria, Egypt. · Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland. · Data Semantics and Decision Making Institute · Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Department of Digital Medicine, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland · Data Semantics and Decision Making Institute (DSDMI), St. Gallen, Switzerland. · Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland. · Department of Digital Medicine, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland.
Common data elements (CDEs) support clinical data comparability but are usually defined manually. We investigated whether large language model agents could derive a CDE schema from sarcoma case report abstracts and apply it at scale. An agent proposed 72,909 candidate CDEs from 24,588 PubMed abstracts; agentic consolidation yielded 1,570 CDEs. We retained 53 supported by at least 50 abstracts and attempted evidence-linked extraction across the corpus. Processing succeeded for all retained CDEs in 79.2% of abstracts, including responses indicating non-mention. Two human reviewers approved 83.0% of retained CDEs for relevance, 67.9% for clarity, and 56.6% for correctness. In 100 sampled abstracts, exact agreement with the value extracted by the LLM was 94.9% among 1,116 pairs with a non-empty answer (abstract-level bootstrap 95% CI, 93.0-96.4%). Errors concentrated in anatomical and temporal fields. These findings support agent-assisted schema drafting followed by expert curation before large-scale extraction.
Figures & tables
Figure 1: Illustrative overview of the study workflow. (a) An LLM agent proposed CDEs for each sarcoma case report abstract in an input corpus. Next, these CDEs were consolidated and standardised, resulting in an induced CDE schema. We selected 53 CDEs supported by at least 50 source abstracts. Two domain experts assessed schema correctness, relevance, and clarity. (b) Values for all 53 CDEs were extracted from each abstract using standard prompting. Every extracted answer was linked to an evidence text span in the source abstract. We validated the LLM-derived outputs using a subset of 100 abstracts. Furthermore, we performed a descriptive analysis based on the whole corpus of 24,577 unique abstracts.
Figure 2: Corpus construction and CDE schema induction. (A) Retrieval, screening, and removal of duplicate abstracts. (B) Proposal, deduplication, consolidation, standardisation, and selection of CDEs. Schema induction and information extraction used all 24,588 accepted abstracts; duplicate removal applied only to the descriptive analysis. Exact deduplication retained numeric and categorical CDEs separately, including 30 question texts occurring with both types. Value-set standardisation revised categorical vocabularies without reducing the number of CDEs.
Dimension
Reviewer 1 approval
Reviewer 2 approval
Reviewer agreement
κ
Approval by both reviewers
Correctness
48/53
30/53
35/53 (66.0%)
0.24
30/53 (56.6%)
Relevance
48/53
47/53
46/53 (86.8%)
0.29
44/53 (83.0%)
Clarity
39/53
50/53
36/53 (67.9%)
-0.10
36/53 (67.9%)
Table 1: Reviewer approval and agreement for the 53 retained CDEs.
CDE
Matches
Non-matches
Compared
Agreement (%)
All CDEs
1,059
57
1,116
94.9
Patient sex
75
2
77
97.4
Age at diagnosis
73
1
74
98.6
Histopathologic diagnosis (biopsy)
50
0
50
100.0
Distant metastases
44
2
46
95.7
Tumour recurrence during follow-up
37
0
37
100.0
Table 2: Selected results for answer-level exact agreement.
CDE (unit)
Non-empty
Included
Excluded
Median (IQR)
Patient age (years)
17,917
17,365
552
46 (26–63)
Age at diagnosis (years)
17,770
17,178
592
47 (26–63)
Maximum tumour dimension (cm)
2,578
1,224
1,354
7 (4–12)
Chemotherapy cycles
757
504
253
5 (3–6)
Total radiation dose (Gy)
448
302
146
50.4 (45–60)
Table 3: Descriptive summaries of five separate numeric CDEs in 24,577 unique abstracts.
Figure 3: Coverage of the retained CDEs in 24,577 unique abstracts.
Figure 4: Seed-based consolidation within each pass. For an unprocessed source CDE, embedding-based retrieval provided up to 20 candidates meeting a cosine-similarity threshold of 0.80 as input to the agent. The agent assessed semantic equivalence and created a master CDE from the seed and selected candidates, or retained the seed alone. Source identifiers were preserved, and the contributing source CDEs were marked as processed. The cycle continued until all source CDEs were processed for that pass; this stopping rule did not imply that no further consolidation was possible. The recorded first and second passes yielded 2,183 and 1,570 master CDEs, respectively.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Field
Match
MO
RO
Diff.
n
%
1
Chemotherapy Cycles Administered
4
0
0
0
4
100.0
2
Years To Local Recurrence After Surgery
4
0
0
1
5
80.0
3
Latency Years To Secondary Tumor
7
1
0
2
10
70.0
4
Disease Free Survival Months
18
0
0
3
21
85.7
5
Length Of Follow Up Years
20
0
0
0
20
100.0
6
Maximum Tumor Dimension Cm
8
0
0
0
8
100.0
Appendix
Supplementary Table S1: Answer-level evaluation for all 53 retained CDEs.
ID
Field and unchanged question
Type
Source n
Labels
Filled n (%)
1
Chemotherapy Cycles Administered How many cycles of chemotherapy were administered?
Numeric
51
–
757 (3.1%)
2
Years To Local Recurrence After Surgery How many years after initial surgery did the tumor recur locally?
Numeric
51
–
996 (4.1%)
3
Latency Years To Secondary Tumor How many years after the primary cancer treatment or diagnosis did a secondary tumor develop?
Numeric
83
–
2,467 (10.0%)
4
Disease Free Survival Months What is the duration of disease-free survival (months)?
Numeric
139
–
4,660 (19.0%)
5
Length Of Follow Up Years What is the length of follow-up reported (in years)?
Numeric
100
–
4,801 (19.5%)
6
Maximum Tumor Dimension Cm What is the maximum tumor dimension in centimeters?
Numeric
2,072
–
2,578 (10.5%)
Appendix
Supplementary Table S2: Retained schema and field coverage in 24,577 unique abstracts.
Field
Exact label
Count
Field n
%
15. Did Patient Develop Distant Metastases
No
5,755
11,050
52.1
Yes
5,295
11,050
47.9
16. Tumor Recurrence During Follow Up
Yes
3,956
7,696
51.4
No
3,740
7,696
48.6
17. Was Surgical Resection Performed
Yes
13,397
14,933
89.7
No
1,536
14,933
10.3
Appendix
Supplementary Table S3: Up to five most frequent exact labels per categorical field in 24,577 unique abstracts.
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA · College of Nursing, Florida State University, Tallahassee, FL, USA
Chair for AI in Healthcare and Medicine, TU Munich (TUM) and TUM University Hospital, Germany · Department for Trauma Surgery, TUM University Hospital and TU Munich, Germany · Department for Diagnostic and Interventional Radiology, TUM University Hospital and TU Munich, Germany +4