cs.CLApr 12, 2026

EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning

Authors: Hengyu Zhang, Xuyun Zhang, Pengxiang Zhan, Linhao Luo, Hang Lv, Yanchao Tan, Shirui Pan, Carl Yang

Organizations: Macquarie University Sydney, Australia · Fuzhou University Fuzhou, China · Monash University Melbourne, Australia · Griffith University Brisbane, Australia · Emory University Atlanta, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent advances in large language models (LLMs) have enabled promising progress in diagnosis prediction from electronic health records (EHRs). However, existing LLM-based approaches tend to overfit to historically observed diagnoses, often overlooking novel yet clinically important conditions that are critical for early intervention. To address this, we propose EviCare, an in-context reasoning framework that integrates deep model guidance into LLM-based diagnosis prediction. Rather than prompting LLMs directly with raw EHR inputs, EviCare performs (1) deep model inference for candidate selection, (2) evidential prioritization for set-based EHRs, and (3) relational evidence construction for novel diagnosis prediction. These signals are then composed into an adaptive in-context prompt to guide LLM reasoning in an accurate and interpretable manner. Extensive experiments on two real-world EHR benchmarks (MIMIC-III and MIMIC-IV) demonstrate that EviCare achieves significant performance gains, which consistently outperforms both LLM-only and deep model-only baselines by an average of 20.65% across precision and accuracy metrics. The improvements are particularly notable in challenging novel diagnosis prediction, yielding average improvements of 30.97%.

Figures & tables

Figure 1. Diagnosis Prediction Patterns Across Models. (a) LLMs tend to predict repeating historical diagnoses (e.g., CHD); (b) Deep models are effective at predicting novel diagnoses (e.g., MD) but lack explanation; (c) Our method integrates deep model-guided evidence into LLMs to achieve accurate and explainable predictions (e.g., EPC →\rightarrow MD). (d) Proportion of predictions: historical (solid bars) vs. novel (hatched bars); (e) Overall prediction performance (Precision@10, O-Correct); (f) Performance on novel diagnoses (Precision@10, N-Correct).
Figure 2. The overall framework of our proposed EviCare , which consists of (1) deep model inference for Candidate Selection , (2) Evidential Prioritization for set-based EHRs, (3) Relational Evidence Construction for novel diagnosis prediction, and (4) LLM-based Diagnosis Pred iction via evidence-driven in-context learning.
DatasetMIMIC-IIIMIMIC-IV
# of patients5,44979,393
# of visits14,141329,605
Avg. # visits per patient2.604.15
Avg. # CCS per visit12.0811.30
Avg. # Novel CCS per visit6.154.82
Max. # visits per patient29169
Table 1. Statistics of the datasets used in our experiments.
TaskNovelOverall
MetricVisit-LevelCode-LevelVisit-LevelCode-Level
P@5P@10Acc@5Acc@10P@10P@20Acc@10Acc@20
LLaMA3.11.01 ±0.092.22 ±0.470.43 ±0.192.19 ±0.4541.19 ±2.1142.06 ±1.0330.46 ±0.8439.88 ±0.71
Qwen30.65 ±0.052.76 ±0.510.68 ±0.162.54 ±0.2847.03 ±1.0548.61 ±1.1232.73 ±0.6646.10 ±1.02
RETAIN10.14 ±0.1716.41 ±0.246.95 ±0.1815.50 ±0.2138.69 ±0.1845.62 ±0.2627.68 ±0.2444.15 ±0.23
StageNet12.91 ±0.1220.39 ±0.238.76 ±0.1819.27 ±0.2335.70 ±0.1743.59 ±0.2425.71 ±0.2342.83 ±0.19
Table 2. Results of novel and overall diagnosis prediction performance on the MIMIC-III dataset with 5% training data. The best results are highlighted in bold while the second best are underlined .
MetricVisit-LevelCode-Level
 P@5P@10Acc@5Acc@10
LLaMA3.15.29 ±0.766.13 ±0.664.46 ±0.567.31 ±0.70
Qwen35.14 ±0.466.36 ±0.614.36 ±0.287.53 ±0.74
RETAIN8.12 ±0.0614.03 ±0.085.71 ±0.1112.14 ±0.15
StageNet7.81 ±0.0613.10 ±0.075.38 ±0.1011.20 ±0.11
TRANS9.03 ±0.0714.35 ±0.116.31 ±0.1412.32 ±0.13
Table 3. Results of novel diagnosis prediction performance on the MIMIC-IV dataset with 5% training data. The best results are highlighted in bold while the second best are underlined .
TaskNovelOverall
MetricVisit-LevelCode-LevelVisit-LevelCode-Level
P@5P@10Acc@5Acc@10P@10P@20Acc@10Acc@20
Qwen3-8B (Base)0.65 ±0.052.76 ±0.510.68 ±0.162.54 ±0.2847.03 ±1.0548.61 ±1.1232.73 ±0.6646.10 ±1.02
+ Candidate9.61 ±0.4212.11 ±0.576.45 ±0.4311.04 ±0.6250.81 ±0.8252.50 ±0.7435.89 ±0.7949.05 ±0.69
+ Prioritization10.71 ±0.7714.34 ±0.537.53 ±0.6813.95 ±1.0352.61 ±0.7654.19 ±0.7837.17 ±0.5551.52 ±0.70
+ Relational (Full)21.54 ±1.6024.41 ±1.1614.48 ±0.6422.05 ±0.5751.73 ±0.7255.05 ±0.9536.44 ±0.4752.26 ±0.74
Table 4. Ablation study of EviCare components on MIMIC-III.
Figure 3. P@10 performance on novel and overall diagnosis prediction under varying candidate CCS sizes (MIMIC-III).
MethodP@10Reasoning Example Highlights
Qwen (CoT)3.17“ Atrial flutter and lung edema are often seen in heart failure and pneumonia ”
Qwen (SC)4.16“ Cardiomyopathies and dysrhythmias are strong indicators of heart failure”
EviCare (Ours)24.58“AICD adjustment suggests conduction issues , which links to nonhypertensive heart failure”
Table 5. Comparison with different reasoning paradigm.
MethodDiagnosis Prediction Set
Input EHR DataICD : Cardiac complications, not elsewhere classified ( CNC ); Acute pancreatitis ( AP ); Peritoneal adhesions postoperative postinfection ( Papp ); Congestive heart failure, unspecified ( CHFu ); … CCS : Complications of surgical procedures or medical care ( CSPM ); Pancreatic disorders (not diabetes) ( PD ); Other gastrointestinal disorders ( OGD ); Congestive heart failure; nonhypertensive ( CHFn ); …
Ground-Truth Num:10Coronary atherosclerosis and other heart disease( CAHD ) ; CHFn , Peripheral and visceral atherosclerosis( PVA ); CSPM ; Deficiency and other anemia( DOA ); Acute posthemorrhagic anemia( APA ); Essential hypertension( EH );…
Qwen3-8B Correct: 3CHFn ( …directly listed in the patient’s history…is a complication of AP ..can lead respiratory failure…also in history .)
 CSPM ( …listed in the patient’s history…linked to Papp and AP … )
 PVA ( …patient’s history of heart disease and CNC supports this possibility… )
 Pulmonary heart disease (…patient history has CHFn ,…lead pulmonary hypertension an pulmonary heart disease.)
Table 6. Case study on a MIMIC-III patient comparing base model (Qwen3-8B), deep model (BoxLM) and EviCare under overall CCS prediction settings. Blue = Novel CCS in ground truth; Red = Incorrect prediction. Arrows (→) indicate evidence chains inferred from patient history to support predicted diagnoses.
LLMDeepNovelOverallAvg. Imp.
LLaMA3.1None2.2241.19–
 RETAIN20.7348.21425.4%
 BoxLM22.1550.09459.4%
Qwen3None2.7647.03–
 RETAIN23.5949.66380.1%
 BoxLM24.4151.73397.4%
Table 7. Generalization of EviCare across different LLMs on MIMIC-III. The metric used is Precision@10.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

MetricVisit-LevelCode-Level
 P@10P@20Acc@10Acc@20
LLaMA3.138.32 ±1.0243.62 ±1.3631.22 ±0.6333.82 ±0.72
Qwen350.42 ±0.9050.99 ±1.2036.79 ±0.6946.06 ±0.78
RETAIN43.19 ±0.0849.05 ±0.0731.49 ±0.0745.60 ±0.09
StageNet37.69 ±0.0843.45 ±0.0727.69 ±0.0640.89 ±0.07
TRANS36.00 ±0.0941.95 ±0.0826.52 ±0.0739.83 ±0.08
Appendix
Table 8. Results of overall diagnosis prediction performance on the MIMIC-IV dataset with 5% training data. The best results are highlighted in bold while the second best are underlined .
Figure 4. P@10 performance on novel and overall diagnosis prediction under different training data ratios (MIMIC-III).
NovelP@5P@10Acc@5Acc@10
Qwen (CoT)1.253.171.633.07
Qwen (SC)2.094.262.714.02
EviCare22.0324.5814.5622.12
OverallP@10P@20Acc@10Acc@20
Qwen (CoT)41.9145.6629.1243.80
Qwen (SC)42.2346.0930.1544.26
Appendix
Table 9. Performance Comparison of Different Reasoning Strategies.
Patient HistoryOther primary cardiomyopathies; Paroxysmal ventricular tachycardia; Fitting and adjustment of AICD; Cardiac dysrhythmias; Conduction disorders; …
Target DiagnosisCongestive heart failure; nonhypertensive
Qwen (CoT)The patient has a history of "Other primary cardiomyopathies" and "Paroxysmal ventricular tachycardia" which are both associated with heart failure.
Qwen (SC)1. The patient has "Other primary cardiomyopathies" and "Cardiac dysrhythmias", both of which are precursors or components of nonhypertensive heart failure. Although not explicitly stated, the progression is clinically plausible.
 2. The patient’s history includes "Other primary cardiomyopathies" (ICD) and "Cardiac dysrhythmias" (CCS), which are strong indicators of heart failure. Cardiomyopathies can progress to heart failure, and dysrhythmias like paroxysmal ventricular tachycardia are common in heart failure.
 3. (No relevant prediction)
EviCare (Ours)This is supported by the patient’s history of cardiac dysrhythmias, other primary cardiomyopathies, and AICD adjustment. These conditions are often associated with heart failure, especially in the context of chronic disease and device management.
Appendix
Table 10. Explanation comparison for a novel diagnosis prediction case.

Explore similar work

CardsList

  1. ReMedi: Reasoner for Medical Clinical Prediction

    May 2, 2026Yushi Cao, Yiming Chen, Hongchao Jiang +2Fine-TuningClinical Outcome Prediction

    ASUS Intelligent Cloud Services (AICS), Singapore · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan · National University of Singapore, Singapore

  2. Coverage-Aware Reasoning with Medical Tokens for Diagnosis Prediction

    Oct 7, 2026Kaisong Zhang, Haotian Fang, Junmeng Zhou +3Medical LLMsClinical Prediction

    Fuzhou University · Shanghai Jiao Tong University · Beihang University

  3. ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning

    Jun 1, 2026Bo-Hong Wang, Baicheng Peng, Ruilin Wang +3LLM GroundingLLM Alignment

    School of Computer Science, McGill University, Montreal, QC, Canada · Mila - Quebec AI Institute, Montreal, QC, Canada