Auditable Clinical Timeline Reconstruction with Provenance-Aware Evidence Graphs

Judith Jeyafreeda Andrew

cs.AIOct 5, 2026

Auditable Clinical Timeline Reconstruction with Provenance-Aware Evidence Graphs

Authors: Judith Jeyafreeda Andrew

Organizations: Public Health Unit, Medical Information Department, Bordeaux University Hospital (CHU Bordeaux), Bordeaux, France.

Abstract

A patient-timeline reconstruction system is auditable only if it keeps the mentions behind each answer, records how facts were revised, and declines to answer when the evidence is not in the text. This study tests these three properties on a fully synthetic corpus (1,000 patients, 3,353 notes, 220 revision edges). Two provenance-aware Evidence Graph operators reduced the node-plus-edge count to 67% and 63% (77-78% of serialized size) while preserving every answer and mention link across 6,813 query points answerable by recency; a fixed-window baseline returned no value for 53.4% of points, unflagged. On evidence-unavailable controls that announce the omission, a BioClinicalBERT gate and a zero-shot LLM gate responded mainly to the announcement. On marker-free controls, BERT abstained on 0 of 81 notes while its accuracy fell from 93.8% to 59.3% across all three relation classes; the LLM's coverage fell from 75.6% to 27.7% on notes its own model family judged undeterminable. Against 482 regenerated gold spans, the LLM's cited evidence reached recall 0.850 and precision 0.864; BERT's span head, trained without span labels, did not localize evidence. A temporally versioned provenance graph stored abstentions as typed, queryable edges. The clean task admits a 0.651-accuracy shortcut, and results describe implementation behaviour on synthetic data, not clinical performance.

Figures & tables

Figure 1: Architecture of the synthetic mechanism-validation pipeline. The generator produces notes, facts, mention links, revision links, adversarial variants and evidence-unavailable controls. Branch A builds an Evidence Graph directly from generator facts and scores compression against a revision-sensitive oracle; extractor predictions do not enter it. Branch B runs the evidence gates on evidence-available notes. Branch C runs the gate families on evidence-unavailable controls. The provenance graph is independent of the compression graph. The solid arrow marks the reported experiment (ungated versus gated LLM outputs); the dashed arrow marks ingestion of control-branch outputs, which the infrastructure supports but which was not evaluated here.

Figure 2: Construction and measured composition of the evidence-unavailable controls. remove_evidence produced one identical sentence for all 1,000 notes. mask_temporal_cue masked a cue in 421 notes and, in 579, kept the original note and appended a sentence stating that the cue was omitted.

Quantity	Value
Patients	1,000
Notes (total)	3,353 (realized mean 3.35 per patient)
Clinical facts	2,220
Patients with a documented revision	110 (11.0%)
Revision edges (ground truth)	220
Relation-eligible notes (gold relation and span)	1,000

Table 1: Dataset summary.

A. All query points ( $n=6{,}813$ )
Representation	Nodes	Edges	Total	Size ratio	Query acc.	Query cov.	Prov. recall
Uncompressed	6,926	7,146	14,072	1.00	1.000	1.000	1.000
Aggregation	2,220	7,146	9,366	0.67	1.000	1.000	1.000
Revision-representation	2,000	6,926	8,926	0.63	1.000	1.000	1.000
Temporal-approximation	2,000	6,978	8,978	0.64	0.987	0.996	1.000
Fixed-window ( $k=2$ )	3,982	4,202	8,184	0.58	0.453	1.000	0.575

Table 2: Representation comparison across compression strategies (6,813 sampled query points). 0 0 footnotetext: Size ratio is nodes plus edges divided by the uncompressed total; bytes ratio is the same for the JSON-serialized representation. Query acc. is accuracy over answered (non-ABSTAIN) points and Query cov. the fraction answered. A representation with no entry dated on or before the query day returns no value, which is scored as wrong and counted as covered. Over all points, temporal-approximation answered 6,788 (6,701 correct, 87 wrong without a flag, 1.3%) and returned ABSTAIN on 25 (0.4%); the fixed window returned no value on 3,641 (53.4%) and a wrong value on 84 (1.2%). Revision patients: the 110 patients with a revision edge. Post-revision: points on or after the day a revision of the queried concept took effect; temporal-approximation returned no ABSTAIN there. Because every revision post-dates the fact it revises, a revision-blind latest-fact-by-day rule scores 1.000 on all three sets.

Figure 3: Compression trade-off. (a) Representation size relative to the uncompressed graph (nodes plus edges). (b) Query accuracy and provenance recall on the same 6,813 sampled query points. The fixed-window baseline is the smallest representation and the only one that loses mention links. Temporal-approximation returns unflagged wrong values on 1.3% of points; the fixed window returns no value on 53.4% and a wrong value on 1.2%, none flagged.

System	Configuration	$n$	Abstain rate	Accuracy (answered)
BERT	Baseline (ungated)	150	0.000	0.740
BERT	Provenance-first (hard-gated)	150	0.960	1.000 (6 answered)
LLM	Baseline (ungated)	60	0.000	0.617
LLM	Provenance-first (gated)	60	0.550	0.556 (27 answered)

Table 3: Relation extraction under the original gates. Evidence recall is invalid as implemented (see text).

Gate	Rows	$n$ scorable	Recall	Precision	Span / note length
BERT	Ungated baseline	77 / 150	0.049	—	—
BERT	Availability-supervised (gated)	77 / 150	0.006	—	—
LLM	All scorable (abstentions scored on their cited span)	482 / 1,000	0.750	0.866	0.329
LLM	Answered rows only	379 / 1,000	0.850	0.864	0.339

Table 4: Evidence-span recall against regenerated gold spans. LLM precision is the fraction of the cited span that falls inside the gold span; span/note length is the cited span’s length as a fraction of the note’s length. BERT reports token-overlap recall only (Section 2.5 ); no LLM-style precision was computed for it. BERT heads trained on all-zero span labels.

Figure 4: Evidence-span recall against the 482 regenerated gold spans of Section 2.5 . (a) Recall for the ungated and availability-supervised (gated) BERT models (token-overlap, 77/150 scorable) and the LLM gate, pooled (abstained rows scored on the span they cited, 0 if none) and restricted to answered rows. (b) LLM recall, precision and the cited span’s length as a fraction of the note, on answered rows: high recall and precision together with a short span show that the gate did not simply cite most of the note.

System	Gate	Prompt	$n$	Coverage	False abst.	Answered acc. ( $n$ ans.)
BERT	Ungated baseline	—	150	1.000	0.000	0.740 (150)
BERT	Hard-gated	—	150	0.040	0.960	1.000 (6)
BERT	Availability-supervised	—	150	0.987 [0.953, 0.996]	0.013	0.899 [0.840, 0.938] (148)
LLM	Ungated baseline	original	60	1.000	0.000	0.617 (60)
LLM	Original gate	original	60	0.450	0.550	0.556 (27)
LLM	Availability-aware	no event defs.	1,000	0.587 [0.556, 0.617]	0.413	0.373 [0.335, 0.413] (587)

Table 5: Clean (evidence-available) inputs. Wilson 95% intervals in brackets for the new runs; the false-abstention rate is 1 minus coverage and shares its interval. Prompt: original is the earlier LLM pipeline of Table 3 ; event defs. means EVENT1/EVENT2 and the three relations are defined in the prompt. All other LLM results in this paper use the prompt with event definitions. The hard-gated BERT abstains below a token-coverage floor.

Gate	Condition	$n$	Clean cov.	Abstention recall	Unsafe-answer rate	$n$ ans.	Answered accuracy
BERT	remove_evidence	150	0.987	1.000	0.000	0	undefined
BERT	mask_temporal_cue	150	0.987	0.980 [0.943, 0.993]	0.007 [0.001, 0.037]	3	0.667
BERT	token inserted	61	0.967	1.000 [0.941, 1.000]	0.000 [0.000, 0.059]	0	undefined
BERT	notice, note intact	89	1.000	0.966 [0.906, 0.988]	0.011 [0.002, 0.061]	3	0.667
BERT	All controls	300	0.987	0.990 [0.971, 0.997]	0.003 [0.001, 0.019]	3	0.667
LLM	remove_evidence	1,000	0.700	1.000	0.000	0	undefined

Table 6: Evidence-unavailable controls, with each control’s paired clean coverage. Clean cov. is the answer rate of the same base notes without any intervention. Coverage on unavailable evidence equals 1 minus abstention recall (for example 0.331 for the LLM on mask_temporal_cue ) and shares its interval. Wilson 95% intervals in brackets; none is given for remove_evidence , a single distinct input. The indented rows split mask_temporal_cue by the control file’s masked_cue field: a mask token replaced a cue phrase, or the note was left intact with an appended omission notice. Answered accuracy is undefined when no input is answered.

Figure 5: Abstention behaviour of the availability-supervised BERT gate and the LLM gate. (a) Fraction of inputs answered, with Wilson 95% intervals; an ideal gate answers clean inputs and abstains on every control. Diamonds mark the answer rate of the same base notes when clean, so the distance from a diamond to its bar is the effect of the control. The two middle conditions split mask_temporal_cue by whether a cue was replaced by a mask token or the note was left intact with an appended notice. remove_evidence is a single distinct input. (b) Accuracy among answered inputs; hatched bars are mask_temporal_cue , scored against the gold relation of the original note. The dotted line is the majority-class rate.

Gate	Condition	$n$	Clean cov.	Clean acc.	Coverage (silent)	Accuracy (silent)
BERT	silent_keep	70	0.986	0.928	0.957 [0.881, 0.985]	0.836 [0.729, 0.906]
BERT	silent_removal , implied_only	44	0.977	0.930	1.000 [0.920, 1.000]	0.614 [0.466, 0.743]
BERT	silent_removal , strict	37	1.000	0.946	1.000 [0.906, 1.000]	0.568 [0.409, 0.713]
BERT	silent_removal , all	81	0.988	0.938	1.000 [0.955, 1.000]	0.593 [0.484, 0.693]
LLM	silent_keep	452	0.723	0.881	0.721 [0.678, 0.761]	0.850 [0.807, 0.884]
LLM	silent_removal , implied_only	269	0.651	0.851	0.688 [0.630, 0.740]	0.384 [0.317, 0.456]

Table 7: Abstention under silent removal, paired with clean coverage and accuracy on the same base notes. BERT is evaluated on the seed-7 test partition; the LLM, which needs no training split, on all accepted notes. Wilson 95% intervals in brackets.

	silent_removal				silent_keep
Gold	$n$	Clean	Edited acc.	Answered as	$n$	Clean	Edited
relation		acc.	[95% CI]	E1 $<$ E2/E2 $<$ E1/Ovl.		acc.	acc.
E1 $<$ E2	28	1.000	0.750 [0.566, 0.873]	21/0/7	26	1.000	0.960
Ovl.	23	0.913	0.609 [0.408, 0.778]	9/0/14	19	0.842	0.882
E2 $<$ E1	30	0.900	0.433 [0.274, 0.608]	4/13/13	25	0.920	0.680
All	81	0.938	0.593 [0.484, 0.693]	34/13/34	70	0.928	0.836

Table 8: BERT accuracy under silent removal and silent keep, split by gold relation. Availability-supervised (gated) BERT on the seed-7 test partition; accuracy is over answered notes and Wilson 95% intervals are in brackets. The gate answered every silent_removal note. Answered as gives the number of silent_removal notes assigned each relation. 0 0 footnotetext: E1

<

E2: EVENT1-BEFORE-EVENT2; E2

<

E1: EVENT2-BEFORE-EVENT1; Ovl.: EVENT1-OVERLAP-EVENT2. The silent_removal editor mentions the diagnosis first, which reverses mention order only for EVENT2-BEFORE-EVENT1 notes. Clean accuracy is on the unedited base notes. Notes answered: silent_removal clean 80 of 81, edited 81 of 81; silent_keep clean 69 of 70, edited 67 of 70. Produced by per_class_silent.py .

Figure 6: Abstention under silent removal. (a) Fraction of inputs answered, with Wilson 95% intervals; BERT on the seed-7 test partition ( $n$ = 70, 44, 37), the LLM on all accepted notes ( $n$ = 452, 269, 238). Diamonds mark the answer rate of the same base notes when clean. (b) Accuracy among answered inputs, with diamonds marking accuracy on the same notes when clean; the gap from diamond to bar is the cost of the edit. The dotted line is the majority-class rate.

Quantity	Value
Total edges	120
Abstained edges	33 (27.5%)
Relation types among non-abstained edges	EVENT1-BEFORE-EVENT2: 36; EVENT1-OVERLAP-EVENT2: 31; EVENT2-BEFORE-EVENT1: 20
Mean coverage (character-span fraction; not comparable to BERT token-mask coverage)	0.317
Disagreements flagged (baseline versus gated)	33 (all abstention-driven)

Table 9: Baseline-versus-gated LLM graph ingestion and disagreement flags.

Metric	BERT baseline	BERT gated	LLM baseline	LLM gated
Certification examples	30	30	20	20
Certification $\alpha$	0.01	0.01	0.05	0.05
Abstained before perturbation	0/30	28/30	0/20	16/20
Certification rate	0.000	0.000	0.250	0.000
Mean prediction confidence	0.926	0.042	0.939	0.158
Mean provenance confidence	0.567	0.033	0.850	0.158

Table 10: Certification and manipulation results for the original gates. Configurations differ in

\alpha

, sample size and evidence representation and are not directly comparable. BERT adversarial pairs include training-patient notes (1,319 of 1,887).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Script	Role
sythetic_data_generation.py	Corpus generation (Section 2.2)
provenance_first_Bio_Clinical_BERT.py	Hard-gated BERT extraction, certification (Sections 2.6, 2.10)
provenance_first_LLM.py	Gated LLM extraction, certification (Sections 2.6, 2.10)
compression_operators.py	Evidence Graph compression operators and evaluation, including serialized size and revision subsets (Section 2.7)
provenance_temporal_kg.py	Provenance graph, entity resolution, contradiction flags, federation-ready aggregation (Section 2.9)
make_evidence_unavailable_adversarial.py	Evidence-unavailable (marker-carrying) control generation (Section 2.3)

Appendix

Table 11: Script-to-role mapping.

Explore similar work

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment
May 14, 2026Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim +1Patient Trajectories Electronic Health Records
Division of Intramural Research National Library of Medicine, National Institutes of Health Bethesda, MD, USA
Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication
Sep 14, 2026Sayantan Kumar, Nicolas Grimaldi, Jack Cummins +1Electronic Health Records Generalized Linear Model
National Library of Medicine, National Institutes of Health, USA · Princeton University, USA
The Provenance Gap in Clinical AI: Evidence-Traceable Temporal Knowledge Graphs for Rare Disease Reasoning
Apr 18, 2026Md Shamim Ahmed, Maja Dusanic, Moritz Nikolai Kirschner +4Rare Disease Generation Provenance
Department of Mathematics and Computer Science, University of Southern Denmark, Campusvej 55, 5230 Odense, Denmark · Universitätsmedizin Göttingen, Robert-Koch-Straße 40, 37075 Göttingen, Germany

May 14, 2026, cs.CLJ/K move · Enter open · S save

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim, Jeremy C. Weiss

Division of Intramural Research National Library of Medicine, National Institutes of Health Bethesda, MD, USA

Patient Trajectories Electronic Health Records Clinical Notes Multimodal Alignment Tabular Data Text Analysis and Detection

Open paper AskPDF arXiv

Clinical language models increasingly operate over electronic health records (EHRs), yet patient records are not stored as temporally grounded trajectories. Clinical notes describe symptoms, assessments, and disease progression, but often compress or narratively reorder events. Structured EHR rows provide timestamps for labs, medications, vitals, and procedures, but capture only part of the clinical story. We formulate clinical timeline reconstruction as retrieval-augmented temporal grounding: constructing a patient trajectory by using narrative text for event semantics and structured rows as partial temporal evidence. We introduce a scaffolded workflow that extracts central narrative events, builds an initial temporal scaffold, attaches non-central events, and calibrates timestamps using retrieved structured EHR rows. We evaluate on 40 discharge summaries, including 15 i2b2-derived and 25 MIMIC-IV summaries, each with manual gold-standard timelines and aligned structured EHR data. Across models, multimodal calibration left event match rates largely unchanged and generally improved temporal performance: mean paired case-level multimodal-unimodal differences were positive in 7 of 12 model-metric comparisons across concordance and AULTC, with none negative. However, uncertainty was substantial given the 40-case sample; paired case-level bootstrap intervals excluded zero only for the DeepSeek V3.2 AULTC improvement. A gap analysis shows that 35.1% of text-derived events have no structured counterpart. These findings support treating structured EHR data as partial temporal evidence for narrative-derived patient trajectories.

Abstract

Explore similar work

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

The Provenance Gap in Clinical AI: Evidence-Traceable Temporal Knowledge Graphs for Rare Disease Reasoning