Knowledge Graph-Augmented Ambient AI for Clinical Note Generation
Authors: Jakir Hossain, Yi-Fei Zhao, Hongjian Wang, Minmei Shih, Katie Leigh Mullen, Ahmad P. Tafti, Leming Zhou, Manoj Purohit, +4 more
Organizations: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA · Stern School of Business, New York University, New York, NY · Department of Occupational Therapy, University of Pittsburgh, Pittsburgh, PA · Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA · Medical College of Wisconsin, Milwaukee, WI · Clinical and Translational Sciences Institute, University of Pittsburgh, Pittsburgh, PA
Ambient AI is increasingly adopted in healthcare to automatically generate clinical notes from patient-clinician conversations, with the potential to substantially reduce clinician documentation burden. However, generated notes may omit clinically relevant information discussed during the encounter, creating information gaps that can affect downstream care. Knowledge graphs (KGs) constructed from encounter transcripts can provide a structured representation of what was discussed and enable systematic identification of missing information from generated notes that are critical for patient care. In this study, we introduce Coverage-Directed Revision (CDR), a model-agnostic framework that constructs a KG from the encounter transcript, identifies medical concepts absent from an initially generated note, and directs large language models (LLMs) to restore the missing information without modifying the underlying note-generation system. We evaluate CDR on two datasets: 1) Pitt-Bench, a local dataset comprising rehabilitation sessions, and 2) ACI-Bench, a public dataset for benchmarking clinical note generation. We tested four underlying LLMs widely used in ambient AI systems. The results show that CDR consistently improves content recall across all evaluated conditions. Our study provides a practical approach for improving the completeness of ambient AI-generated clinical documentation.
Figures & tables
Figure 1: An overview of the proposed knowledge graph-augmented ambient AI for clinical note generation.
SOAP prompt
KOSMOS transcript-only condition prompt
Role
Ours, and the primary baseline. The corpus notes (PT, OT, SLP) are narrative SOAP records organised around function, not chief complaint and physical exam, and this is the prompt they were written under, so baselines compare against that reference rather than a re-specified target.
A second draft base, not our own: the KOSMOS note-generation prompt run on our transcripts with no graph supplied, so the gain is measured on a draft we did not write. KOSMOS adapts it from the DocLens SOAP prompt 23 ; we reuse it here for reproduction.
Context
Zero-shot: one system message followed by the transcript, with no exemplars. Being zero-shot it transfers across base models without in-context material that could leak into the output.
Two-shot: complete ACI-Bench primary-care encounters and notes as exemplars, about 20 k characters of in-context material carried over from the source template.
Sections
Narrative, paragraph-style SOAP, with each section enumerated: symptoms, concerns and presentation (Subjective); vital signs, assessment results and performed activity (Objective); interpretation of both (Assessment); goals, patient education and session planning (Plan).
Also asks for SOAP, so format is not the difference. But the layout the model sees is ACI-Bench’s: History of Present Illness , Physical Exam , Results , Assessment and Plan , plus Chief Complaint , Review of Systems and Vitals Reviewed .
Citations and graph
Neither is mentioned: no citation markers are required, and the prompt says nothing about a KG.
Both are: per-sentence citations to transcript turn indices, and a system message announcing a transcript-derived graph, which is empty in this condition.
Table 1: The two draft prompts. Both ask for a SOAP note; they differ in context, section layout and citation requirements.
MedCon
FactEHR
Model
Draft
+ CDR
Δ
Draft
+ CDR
Δ
Pitt-Bench , SOAP-prompt base
Qwen2.5-32B
35.68
37.88
+2.20
24.84
29.55
+4.71
Qwen2.5-72B
37.57
42.95
+5.38
27.13
34.21
+7.08
Qwen3-235B
45.67
50.22
+4.55
38.93
41.56
+2.63
GPT-5 Pro
48.59
51.26
+2.67
43.37
46.61
+3.24
Table 2: Results of MedCon and FactEHR recall (%) on Pitt-Bench ( n=45 ) and ACI-Bench ( n=40 ).
Figure 2: Both panels are Pitt-Bench ( n=45 ). Left: the claims CDR adds, grounded against the transcript by the Qwen2.5-72B judge; the count beside each bar is the number of added claims. Right: KG quality. Grounded nodes is the share of graph nodes that map to a UMLS concept; Gold concepts covered is the share of the 648 gold-note concepts that appear in the graph.
in-transcript
beyond-transcript
Model
Label
n
Draft
+ CDR
Δ ( p )
n
Draft
+ CDR
Δ ( p )
Qwen2.5-32B
Majority ( ≥ 3 of 5)
387
45.0
51.9
+6.9 (0.000)
475
8.4
10.5
+2.1 (0.010)
Unanimous (5 of 5)
156
61.5
67.9
+6.4 (0.017)
225
2.7
3.1
+0.4 (0.317)
Qwen2.5-72B
Majority ( ≥ 3 of 5)
373
50.4
60.1
+9.7 (0.000)
446
9.0
11.0
+2.0 (0.064)
Unanimous (5 of 5)
149
66.4
75.2
+8.8 (0.014)
214
5.6
5.6
+0.0 (1.000)
Qwen3-235B
Majority ( ≥ 3 of 5)
387
64.1
68.7
+4.6 (0.022)
475
17.3
18.1
+0.8 (0.417)
Table 3: FactEHR claim recall split by whether the transcript entails the gold claim. The n columns partition each base model’s gold claims into the two strata; p -values are Wilcoxon signed-rank over encounters. All three base models share the same 862 gold claims, except Qwen2.5-72B, scored over 819 because its revision looped on two encounters that the judge excludes.
Ambient documentation systems are rapidly gaining adoption, yet their impact on clinical note quality remains poorly characterized. We introduce MedConv, a multilingual dataset of 300 clinical encounters in English, Danish, and German, and use it alongside the Ambient Clinical Intelligence benchmark (ACI-BENCH) to compare Corti, a clinical AI platform, with two leading, accessible ambient scribe software applications built on general-purpose AI. We present a controlled clinical evaluation framework that combines entailment metrics with LLM-judged pairwise comparisons across eight dimensions adopted from PDSQI-9. Results show that Corti's API-based text-generation infrastructure is on par with or outperforms leading commercial scribes. We further show that Corti's configurable API provides the flexibility necessary to fine-tune quality dimensions for specific documentation use cases. We present the evaluation methodology and release a dataset to support future reproducible comparison of ambient documentation systems.
Daniel Varab, Victor Petrén Bach Hansen, Asbjørn W. Helge +8
Ambient AI generates draft clinical notes from patient-clinician conversations, often using lay or consumer-oriented phrasing to support patient understanding instead of standardized clinical terminology. How clinicians revise these drafts for professional documentation conventions remains unclear. We quantified clinician editing for consumer-to- clinical normalization using a dictionary-confirmed transformation framework. We analyzed 71,173 AI-draft and finalized-note section pairs from 34,726 encounters. Confirmed transformations were defined as replacing a consumer expression with its dictionary-mapped clinical equivalent in the same section. Editing significantly reduced terminology density across all sections (p < 0.001). The Assessment and Plan accounted for the largest transformation volume (59.3%). Our analysis identified 7,576 transformation events across 4,114 note sections (5.8%), representing 1.2% consumer-term deletions. Transformation intensity varied across individual clinicians (p < 0.001). Overall, clinician post-editing demonstrates consistent shifts from conversational phrasing toward standardized, section- appropriate clinical terminology, supporting section-aware ambient AI design.
Ha Na Cho, Yawen Guo, Sairam Sutari +5
Department of Informatics, University of California, Irvine, Irvine, CA, USA · Institute for Clinical and Translational Science, University of California, Irvine, Irvine, CA, USA · Department of Medicine, University of California, Irvine, Irvine, CA, USA +1
Provider queries are clarifying requests sent by clinical documentation specialists to physicians to close gaps in the clinical note and ensure accurate billing. Prior work automates note drafting, ICD-10 coding, and order extraction assuming a complete transcript, leaving these gaps unaddressed. We study whether an LLM can automate the query loop, termed DAU (Draft, Ask, Update), across those three tasks. An audit of 3,000 real visits identifies the sources of missing documentation, from which we build five transcript-degradation benchmarks on public data. Analyzing 21k clarification turns on real conversations, we find useful-question predictors are task-specific: oracle confidence dominates, but note completeness needs only simple recall questions while ICD-10 coding needs harder, multi-option ones. About 9% of turns hurt performance, driven by redundant questions and non-answers that still trigger a rewrite. Deployment depends on learning "when not" as much as "what to" ask.