Clinical decision-making requires synthesizing heterogeneous evidence, including patient histories, clinical guidelines, and trajectories of comparable cases. While large language models (LLMs) offer strong reasoning capabilities, they remain prone to hallucinations and struggle to integrate long, structured medical documents. We present MED-COPILOT, an interactive clinical decision-support system designed for clinicians and medical trainees, which combines guideline-grounded GraphRAG retrieval with hybrid semantic-keyword similar-patient retrieval to support transparent and evidence-aware clinical reasoning. The system builds a structured knowledge graph from WHO and NICE guidelines, applies community-level summarization for efficient retrieval, and maintains a 36,000-case similar-patient database derived from SOAP-normalized MIMIC-IV notes and Synthea-generated records. We evaluate our framework on clinical note completion and medical question answering, and demonstrate that it consistently outperforms parametric LLM baselines and standard RAG, improving both generation fidelity and clinical reasoning accuracy. The full system is available at https://huggingface.co/spaces/Cryo3978/Med_GraphRAG , enabling users to inspect retrieved evidence, visualize token-level similarity contributions, and conduct guided follow-up analysis. Our results demonstrate a practical and interpretable approach to integrating structured guideline knowledge with patient-level analogical evidence for clinical LLMs.
Figures & tables
Figure 1: Overview of the proposed system, which performs dual evidence retrieval from similar patient cases and graph-structured clinical guidelines to support clinical inference.
Model
Text Generation
QA
MIMIC-IV Clinical Notes
MedQA
MMLU-clinical
BLEU
METEOR
ROUGE-1
ROUGE-2
ROUGE-L
BERTScore-F1
Acc
Acc
Baselines without Retrieval
DeepSeek-Chat
0.006
0.251
0.205
0.042
0.120
0.809
0.809
0.857
GPT-4.1-mini
0.012
0.282
0.301
0.064
0.144
0.825
0.846
0.868
Gemini 2.5
0.014
0.289
0.295
0.070
0.147
0.829
0.867
0.875
Table 1: Performance under different retrieval configurations on clinical note generation and QA benchmarks.
Setting
Text Generation
QA
MIMIC-IV Clinical Notes
MedQA
MMLU-clinical
BLEU
METEOR
ROUGE-1
ROUGE-2
ROUGE-L
BERTScore-F1
Acc
Acc
Baseline
0.012
0.282
0.301
0.064
0.144
0.825
0.846
0.868
+ SPR
0.028
0.315
0.324
0.099
0.159
0.828
0.851
0.914
+ GraphRAG
0.031
0.310
0.352
0.115
0.258
0.840
0.870
0.952
+ Both
0.035
0.321
0.380
0.124
0.291
0.851
0.912
0.971
Table 2: Ablation results isolating the contribution of similar-patient retrieval (SPR) and GraphRAG-based guideline reasoning.
Figure 2: Interactive question answering with dual evidence control.
Figure 3: Similar-patient panel with query-conditioned saliency highlighting. Clinical concepts are color-coded according to their semantic relevance to the input query (yellow: important; red: highly important).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Patient case input and query locking interface.
Figure 5: Retrieval of the most similar patient case from the curated repository.
Figure 6: Guideline-grounded evidence retrieval through the GraphRAG pipeline.
Figure 7: Interactive question answering with user-controlled evidence selection.
Figure 8: Detailed construction and retrieval pipeline of the GraphRAG module, including segmentation, graph extraction, indexing, and hybrid retrieval.
Clinical practice guidelines (CPGs) encode evidence-based decision logic that clinicians apply by evaluating patient variables, conditional criteria, and recommendation rules. However, existing methods often use CPGs as free-text training data or retrieval sources, underutilizing their procedural decision structure. To better exploit this structure, we introduce a guideline-derived training pipeline that transforms CPG recommendations into executable clinical decision logic and uses it to generate factual and counterfactual question-answering data. Theses data teach models both guideline-supported decisions and how decisions change under different patient conditions. Post-training a medical LLM on the generated data yields MedGuideX. Across four clinical reasoning benchmarks, MedGuideX achieves a 10.28% relative improvement in average accuracy. Physician evaluation further shows that MedGuideX better recovers clinician authored reasoning steps and produces physician-preferred rationales in faithfulness, validity, completeness, and clarity. Overall, our results show that executable decision logic from CPGs can be transformed into scalable supervision for building reliable medical LLMs.
Yuhao Shen, Lang Cao, Simo Du +4
University of Illinois Urbana-Champaign · 2The Chinese University of Hong Kong, Shenzhen · 3Albert Einstein College of Medicine
Large language models (LLMs) have shown considerable promise in clinical decision support. However, existing approaches often rely on multi-step reasoning or multi-agent collaboration to execute clinical guidelines, introducing substantial computational overhead without reliably preventing decision errors. To address these limitations, we propose a framework for Medical Decision Execution (MedDEx) that integrates structured patient states with deterministic rule execution to support guideline-based treatment planning. MedDEx separates clinical record interpretation from guideline execution, leveraging LLMs to extract patient information and generate treatment plans while assigning guideline-defined decision logic to a rule executor. Experiments across multiple clinical guideline benchmarks demonstrate that MedDEx improves decision accuracy while reducing inference costs. Further analyses highlight the critical roles of structured patient representations and deterministic rule execution, suggesting that additional LLM reasoning does not necessarily translate into better guideline adherence.
Clinical diagnosis requires answers that are accurate, verifiable, and explicitly grounded in official guidelines. While large language models excel at natural language processing, their tendency to hallucinate undermines their utility in high-stakes medical contexts where precision is essential. Existing retrieval-augmented generation (RAG) systems treat all evidence equally, producing noisy context and generic answers misaligned with clinical practice. We present ClinicBot, an AI system that translates guideline recommendations into trustworthy clinical support through three key advances: (1) structured extraction of clinical guidelines into semantic units (recommendations, tables, definitions, narrative) with explicit provenance, (2) evidence prioritization that ranks content by clinical significance and guideline structure rather than textual similarity, and (3) a web-based interface that presents concise, actionable answers with verifiable evidence. We will demonstrate ClinicBot using diabetes questions from real patients and an additional diabetes risk assessment tool that is faithful to the American Diabetes Association (ADA) Standards of Care in Diabetes (2025). The demonstration will illustrate how semantic knowledge extraction and hierarchical evidence ranking can reliably operate in a multi-agent setting to process complex clinical guidelines at scale.
Navapat Nananukul, Mayank Kejriwal
USC Information Sciences Institute, Los Angeles, California, USA