Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reports

Jack Cummins; Sayantan Kumar; Ketan Tamirisa; Jeremy C. Weiss

cs.AISep 11, 2026

Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reports

Authors: Jack Cummins, Sayantan Kumar, Ketan Tamirisa, Jeremy C. Weiss

Organizations: Princeton University, Princeton, New Jersey, USA · National Library of Medicine, Bethesda, Maryland, USA · Washington University in St. Louis, St. Louis, Missouri, USA

Abstract

Existing pipelines for clinical timeline extraction from case reports are evaluated using an expert reference and are limited by imperfect reference annotations and imprecise event alignment. We developed GAVEL, an LLM judge protocol that compares two timelines with the case report and returns a discrepancy type, verdict, and report passage for each difference. We evaluated the event matcher, reviewed 2,738 findings from GPT5.6sol and DeepSeek V3.2, ranked six LLM extractors and two human annotators, and tested GAVEL guided merging. True match rates were 60% immediately below and 48% immediately above the 0.10 cutoff. Manual review confirmed 89.4% and 88.6% of findings. Across 126 reports, merged timelines were preferred in 77.0% of comparisons (95% CI, 69.8 to 84.1%) and reduced discrepancies attributed to the evaluated timeline from 7.63 to 0.85 per report. GAVEL supports report-based comparison and revision without treating either timeline as ground truth.

Figures & tables

Figure 1: The GAVEL adjudication protocol. The LLM judge uses the case report to compare two candidate timelines: it links events to report text, matches corresponding events, and compares values and timestamps. For each discrepancy, it returns a type, a verdict, and the report sentence cited for that verdict, or indicates that no supporting sentence was found. Neither timeline serves as the reference.

Source sentence	“During a telephone call 2 weeks after his visit, the patient described worsening erythema surrounding the biopsy site with yellow, thickened drainage. Thus, he was started on doxycycline, 100 mg twice daily for a week.”
Timeline A (GPT-5)	doxycycline 100 mg twice daily started at $t=336$ h
Timeline B (o4-mini)	doxycycline 100 mg twice daily started at $t=504$ h
Type	TIMING
Verdict	A . The initial visit anchors $t=0$ and the report places the call two weeks later, giving 336 h. Timeline B’s placement is three weeks after the visit and is unsupported.

Table 1: A worked GAVEL adjudication from report PMC6143713. Both timelines record the same event string, so the discrepancy is purely temporal. The report states the interval explicitly, and the judge returns a typed verdict linked to the source text supporting the verdict.

Figure 2: Source-confirmed match rates across PubMedBERT cosine-distance bands among 100 aligned event pairs (25 per stratum). Bars show the percent judged to describe the same clinical event; whiskers denote Wilson 95% confidence intervals. The shaded 0.08–0.12 region spans the 0.10 cutoff used in prior timeline evaluations and contains approximately 11% of all aligned pairs.

Error type	GPT-5.6-sol	DeepSeek V3.2	Total
Timing ruled unclear	59	67	126
Undercalled over-extraction	64	9	73
Anchor forced on two-sided case	21	14	35
Missed one-sided finding	1	13	14
Overcalled duplicate	7	7	14
Perception timing	6	8	14

Table 2: Manual validation and error taxonomy for GAVEL judge findings. Confirmation rate is calculated among findings reviewed by the primary reviewer. The five categories grouped as “Other” were malformed finding, false one-sided finding, wrong placement credited, matching error, and indeterminate timing.

Figure 3: Bradley–Terry ratings with case-bootstrap 95% intervals:(A) eight-source round robin; (B) corpus-scale class-leader comparison; and (C) head-to-head record among the corpus-scale entrants, giving reports won, lost, and tied with mean share of weighted discrepancy points.

Entrant	Default	Timing-weighted	Event-weighted
GPT-5	1698 (1588–1831)	1692 (1544–1868)	1701 (1617–1809)
Claude Opus 5	1672 (1577–1813)	1678 (1564–1852)	1668 (1585–1785)
o4-mini	1610 (1546–1687)	1642 (1559–1743)	1576 (1515–1652)
Annotator 1	1558 (1463–1652)	1560 (1444–1667)	1558 (1466–1651)
GLM-5.2	1494 (1402–1582)	1509 (1408–1592)	1492 (1415–1577)
Gemma 4	1441 (1354–1519)	1438 (1342–1517)	1428 (1344–1509)

Table 3: Leaderboard stability across three scoring rubrics. The default rubric weights over-annotation 3.0, false duplicates, wrong times and wrong values 2.0, missed positives 1.0, and missed negatives and recurrences 0.5. The timing rubric raises wrong time to 6.0 and lowers the omission categories to 0.2. The events rubric raises over-annotation and false duplicates to 6.0 and all omission categories to 3.0 while lowering wrong time to 1.5. Ratings are Bradley–Terry abilities on the mean-centred scale with case-bootstrap 95% intervals.

Figure 4: One report adjudicated and merged. Each row is a proposal made against the GPT-5 base timeline by one of the two proposer timelines, the merger’s decision, and the resulting entry. The merger consults the report before acting: the first proposal is declined because the report states the admission separately, while the last is applied because the report supports only that tirzepatide was advised against, not that it was stopped.

A. Eight-source round robin
Entrant	Wrong time	Missed positive	Missed negative	Missed recurrence	Wrong value	False duplicate	Over- annotated	Total
GPT-5	2.6	2.7	0.2	0.0	0.0	0.3	0.2	6.0
Claude Opus 5	1.4	5.1	0.2	0.1	0.1	0.3	0.0	7.2
o4-mini	0.9	5.0	0.7	0.0	0.1	0.3	0.1	7.1
Annotator 1 (A1)	3.3	2.6	0.6	0.0	0.5	0.3	0.1	7.4
GLM-5.2	3.2	5.7	0.4	0.1	0.7	0.7	0.1	10.9

Table 4: Mean pairwise discrepancy instances by entrant: (A) Stage 2 per report across seven opponents; (B) Stage 3 per entrant–report matchup; and (C) GPT-5 base versus merged timelines per report across 126 reports. Panel C values are shown to two decimal places. Because these are pairwise instances, the same underlying error may appear in comparisons with multiple opponents.

Clinical Information Extraction

Clinical Information Extraction turns clinical notes, reports, and patient narratives into structured, searchable facts and relations.

See all 106 papers

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication
Sep 11, 2026Sayantan Kumar, Nicolas Grimaldi, Jack Cummins +1Multimodal Grounding Document Information Extraction
National Library of Medicine, National Institutes of Health, USA · Princeton University, USA
Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment
May 14, 2026Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim +1Multimodal RAG Healthcare
Division of Intramural Research National Library of Medicine, National Institutes of Health Bethesda, MD, USA
TimelineAgent: An LLM-based Framework for Query-Centric Event Timeline Extraction Across Multiple Documents
Oct 19, 2026Qingkai Min, Yunxin Bao, Yifeng Lu +4Event Extraction LLM Information Extraction
Zhejiang University · School of Engineering, Westlake University · Institute of Advanced Technology, Westlake Institute for Advanced Study +3

Sep 11, 2026J/K move · Enter open · S save

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

Sayantan Kumar, Nicolas Grimaldi, Jack Cummins, Jeremy C. Weiss

National Library of Medicine, National Institutes of Health, USA · Princeton University, USA

Multimodal Grounding Document Information Extraction Clinical Information Extraction Longitudinal EHR Modeling Multimodal Model Evaluation

Open paper AskPDF arXiv

Clinical timelines support treatment-window analysis and leakage-free modeling, but discharge summaries often obscure chronology and structured EHR tables describe only part of the patient course. We present a UID-preserving framework that links each narrative event occurrence to its source span and retains that identity through text-only estimation, structured-evidence retrieval, timestamped source-row grounding, and joint revision. We also present GAVEL, an LLM judge that compares two UID-aligned timelines against the narrative and structured record, to augment prior matching and temporal assessments. Across six open-weight models and 40 mixed-critical-care summaries, the GLM 5.2 multimodal revision, as compared to its text-only variant, improved temporal agreement without reducing event recovery and performed competitively with clinician annotations, while other model revisions showed smaller gains and lower overall performance. Ablations showed that UIDs primarily preserve event retention, whereas source-row linkage supports temporal placement. Blinded human review upheld most GAVEL findings, and controlled adjudication favored multimodal over text-only GLM 5.2 but did not for DeepSeek V3.2. In developing the UID and judge pipeline, we are able to demonstrate 43% increased event recovery, a framework competitive with clinician annotations, and a system with occurrence-level provenance for both reconstruction and evaluation.

Abstract

Clinical Information Extraction

Explore similar work

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

TimelineAgent: An LLM-based Framework for Query-Centric Event Timeline Extraction Across Multiple Documents