A Multi-Agent Pipeline for Source-Grounded Synthetic Note Generation from Longitudinal Structured EHR

Nina Fatehi; Reihaneh Hassanzadeh; Meysam Ghaffari; Animesh Agarwal; Carlos Morato

cs.CLAug 26, 2026

A Multi-Agent Pipeline for Source-Grounded Synthetic Note Generation from Longitudinal Structured EHR

Authors: Nina Fatehi, Reihaneh Hassanzadeh, Meysam Ghaffari, Animesh Agarwal, Carlos Morato

Organizations: Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA.

Abstract

Structured EHR is abundant but sparse, coded, and difficult to use directly for note-centric clinical modeling. We present MedNotes, a multi-agent synthetic data generation pipeline that converts longitudinal structured EHR into source-grounded clinical note representations under explicit quality control. MedNotes treats structured-data-to-text synthesis as a closed-loop agentic process: a generator proposes a note, evaluator agents diagnose factual, coverage, structural, and hallucination-related failures, and a router accepts, revises, or rejects the draft. On 1,485 EHRSHOT encounters, MedNotes achieves a 91.4% pass rate, with mean factual accuracy of 0.980, completeness of 99.1%, structural fidelity of 0.761, and 0.028 critical hallucinations per encounter. Iterative refinement improves acceptance from 69.4% to 91.4%. The resulting synthetic corpus improves downstream CPT prediction and paragraph-level section prediction when combined with limited real data.

Figures & tables

Figure 1: MedNotes pipeline architecture. Each encounter passes through generation, evaluation, aggregation, and decision stages. Factual verification, entity completeness, and structural fidelity are computed in parallel, followed by hallucination categorization. Evaluator outputs are converted into section-specific repair signals and positive anchors for the next round.

Metric	Value
Total encounters	1,485
Pass	1,357 (91.4%)
Discarded	128 (8.6%)
API failures	0 (0.0%)
Mean factual accuracy (final)	0.980
Mean completeness (final)	99.1%

Table 1: Overall results on the 100-patient evaluation cohort.

Round	Pass	% of accepted	Cumulative
Round 1	1,031	76.0%	69.4%
Round 2	266	19.6%	87.3%
Round 3	60	4.4%	91.4%
Discarded	128	—	—

Table 2: Pass rate by reflection round.

Model	Avg.	P	R	F1
Base Llama-3 8B Instruct	Macro	0.0152	0.0634	0.0234
	Micro	0.0366	0.0688	0.0478
+ Fine-tuned on 100 MIMIC	Macro	0.0173	0.0665	0.0244
	Micro	0.0441	0.0516	0.0476
+ Synthetic + 100 MIMIC	Macro	0.2137	0.1034	0.1030
	Micro	0.0827	0.1204	0.0980

Table 3: CPT prediction results on the held-out MIMIC test set.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Latency
Metric	Total	Per encounter
Processing time	–	76.8s
Throughput	–	46.9 enc/hr
Rounds executed
Total rounds	2,087	1.41
Terminal after Round 1	1,056	71.1%

Appendix

Table 5: System cost and latency analysis over 1,485 encounters. Token counts are empirical estimates and may vary with encounter complexity.

Failure mode at Round 1	$N$	Recovered	Recovery rate (95% CI)	Discard rate
Critical hallucination	39	34	87.2% [76.9, 97.1]	12.8%
Factuality failure	158	124	78.5% [70.9, 86.0]	21.5%
SFS failure	116	71	61.2% [51.1, 70.8]	38.8%
Completeness failure	16	8	50.0% [25.0, 75.0]	50.0%
Multiple simultaneous	125	89	71.2% [63.1, 79.1]	28.8%
Total	454	326	71.8% [67.2, 76.4]	28.2%

Appendix

Table 6: Error trajectory of Round-1 failures. Confidence intervals are patient-clustered 95% bootstrap intervals.

Version	Pass	Disc.	Rate	R0 Pass	R1 Rescue	R2 Rescue
v1	137	18	88.4%	89 (57.4%)	37 (23.9%)	11 (7.1%)
v2	130	25	83.9%	70 (45.2%)	45 (29.0%)	15 (9.7%)
v3	142	13	91.6%	101 (65.2%)	27 (17.4%)	14 (9.0%)
v4	146	9	94.2%	94 (60.6%)	37 (23.9%)	15 (9.7%)

Appendix

Table 7: Prompt version comparison on the 10-patient development cohort.

Metric	Recent-2	Relevant-2
Pass rate	26/30 (87%)	18/30 (60%)
Mean factual accuracy	98.28%	85.67%
Mean completeness	97.92%	82.69%
Mean structural fidelity	74.72%	75.19%

Appendix

Table 8: Pilot comparison of recent-history and relevance-based history selection.

Model	Section	Precision	Recall	F1
MediSOAP-100	Subjective	1.000	0.228	0.371
MediSOAP-100	Objective	0.553	0.976	0.706
MediSOAP-100	Assessment	0.976	0.968	0.972
MediSOAP-100	Plan	0.953	0.968	0.960
Synthetic	Subjective	0.887	0.284	0.430
Synthetic	Objective	0.946	0.348	0.509

Appendix

Table 9: Section-wise performance on the MediSOAP test set.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 629 papers

A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models
Jun 25, 2026William Poulett, Alice Waterhouse, Ben Wallace +4Synthetic Data Generation Synthetic Clinical Data Generation
NHS England Data Science and Applied AI Team
Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale
May 18, 2026Jinghui Liu, Sarvesh Soni, Anthony NguyenSynthetic Data Augmentation Automated Medical Coding
Australian e-Health Research Centre, CSIRO, Australia · National Library of Medicine, National Institutes of Health, USA
SynEHR: Joint Modeling Inter-visit Temporal Evolution and Intra-visit Clinical Structure for Longitudinal EHR Synthesis
Aug 21, 2026Ximiao Li, Lin Jiang, Rongchao Xu +3Synthetic Data Generation Longitudinal EHR Modeling
Florida State University Tallahassee, Florida, United States

Jun 25, 2026J/K move · Enter open · S save

A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models

William Poulett, Alice Waterhouse, Ben Wallace, Scarlett Kynoch+3

NHS England Data Science and Applied AI Team

Synthetic Data Generation Synthetic Clinical Data Generation

Open paper AskPDF arXiv

Synthetic data is increasingly used to enable the development and evaluation of AI systems in domains where access to real-world data is restricted. In healthcare, clinical documentation presents particular challenges due to its sensitivity. This work introduces a synthetic clinical notes pipeline and dataset designed to support the development of clinical AI tools while avoiding the privacy risks associated with real patient data. The dataset is generated using a modular pipeline that combines structured patient generation, semi-structured patient journey simulation, and unstructured clinical note generation using large language models. The pipeline is designed to prioritise internal consistency across longitudinal patient records, while also capturing variation in writing style, note structure, and clinical detail. Additional mechanisms, including LLM-based validation and augmentation steps, are used to improve faithfulness, realism, and diversity of the generated notes. We release a dataset of 70 synthetic patients, each associated with 20-50 clinical notes spanning a full hospital journey. The dataset is provided at multiple levels of validation, enabling users to balance realism and scalability depending on their use case. This dataset supports the development, testing, and evaluation of clinical AI systems, including summarisation tools, coding models, and decision support systems, without reliance on real patient data.

Abstract

Synthetic Data Generation

Explore similar work

A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models

Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scale

SynEHR: Joint Modeling Inter-visit Temporal Evolution and Intra-visit Clinical Structure for Longitudinal EHR Synthesis