KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

Xubin Chen; Yipeng Zhou; Wen Sun; Chengkai Huang; Xiaoming Fu; Quan Z. Sheng

cs.CLAug 21, 2026

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

Authors: Xubin Chen, Yipeng Zhou, Wen Sun, Chengkai Huang, Xiaoming Fu, Quan Z. Sheng

Organizations: Macquarie University · Beijing Intelligent Decision Medical Technology Co. Ltd. · University of New South Wales · University of Göttingen ·

Abstract

Automatic Medical Coding (AMC), which assigns standardized International Classification of Diseases (ICD) codes to clinical notes, is essential for medical reimbursement, quality reporting, and clinical research. Existing pre-trained language model (PLM)-based methods typically formulate AMC as an extreme multi-label classification problem over a predefined code set, while recent large language model (LLM)-based approaches instead frame it as generation or multi-step reasoning. However, key challenges remain, including the extreme length of clinical notes that hinders effective interpretation, the vast ICD label space, and complex coding rules that are not explicitly captured by LLMs. In this work, we propose Knowledge-Guided Reasoning over Clinical Evidence with LLMs (KREL), a framework that leverages LLMs for clinical text understanding and reasoning while integrating external ICD coding guidelines as structured knowledge. This design enables tight coupling between domain knowledge and LLM reasoning, reducing hallucinations and improving compliance with coding standards. Experiments on benchmark datasets show that KREL consistently outperforms strong PLM-based and state-of-the-art LLM-based baselines.

Figures & tables

Figure 1: The difference between our framework and existing approaches. Our method better leverages LLM reasoning capabilities by integrating external knowledge.

Figure 2: Overview of the KREL framework. (1) The query extractor reads the clinical note, identifies disease descriptions, and reformulates them into ICD-oriented queries (e.g., converting ”CKD stage 3b” into ”Chronic kidney disease stage 3b”). (2) The selector retrieves candidate ICD codes based on ICD hierarchy knowledge and converts structured coding relations into verification hints. (3) The verifier makes the final coding decisions by jointly considering the candidate codes and all available information and evidence.

(a) Benchmark-label-space setting
Method category	Model	MDACE (CM + PCS)			ACI-BENCH
Method category	Model	Precision	Recall	F1	Precision	Recall	F1
PLM	ICD	0.49	0.47	0.48	0.43	0.41	0.42
PLM	CA	0.46	0.45	0.45	0.44	0.42	0.43
LLM prompting	CoT	0.30	0.31	0.30	0.35	0.50	0.41
LLM prompting	CoT-SC	0.39	0.43	0.41	0.36	0.59	0.44

Table 1: Performance comparison under benchmark-label-space and full-label-space settings

Variant	Type	Precision	Recall	F1
KREL (full)	–	0.48	0.53	0.51
medSpaCy query	Replace	0.35	0.19	0.24
flat retrieval	Replace	0.46	0.50	0.48
w/o reranker	Remove	0.46	0.46	0.46
w/o rule injection	Remove	0.47	0.50	0.49
w/o verifier	Remove	0.11	0.62	0.19

Table 2: Ablation study on MDACE under the full-label-space setting.

Rule	GT (share)	KREL	CoT-SC	CLH
Pairwise	17 (3.14%)	12 / 70.6%	5 / 29.4%	2 / 11.8%
Combination	13 (2.40%)	8 / 61.5%	1 / 7.7%	1 / 7.7%
All	30 (5.55%)	20 / 66.7%	6 / 20.0%	3 / 10.0%

Table 3: Performance on rule-dependent ground-truth note–code pairs in MDACE. Model columns report correct predictions / recall.

Figure 3: Comparison of different strategies for candidate code retrieval on MDACE under the full-label-space setting.

Figure 4: Evolution of the candidate set size against recall and precision performance.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Label-space setting (size)	Code type	Test clinical notes	Test unique GT codes
MDACE	Benchmark (350)	ICD-10-CM + PCS	122	CM: 312; PCS: 38
ACI-Bench	Benchmark (132)	ICD-10-CM	120	132
MDACE	Full (72750)	ICD-10-CM	61	282
MIMIC-IV-500	Full (72750)	ICD-10-CM	500	1,812

Appendix

Table 4: Dataset and evaluation scope.

Statistic	MIMIC-IV	MIMIC-IV-500
Clinical notes	122,288	500
Mean GT codes/note	14.435	14.894
Median GT codes/note	13	14
GT assignments	1,765,181	7,447
Unique GT codes	16,155	1,812
Rare GT share	1.77%	1.84%

Appendix

Table 5: Statistics of MIMIC-IV-500 and the full MIMIC-IV pool. Rare GT denotes assignments whose code frequency in the full pool is

\leq 10

Metric	LLM prompting	LLM-agent (CLH)	KREL
Average predicted codes per note	5.98	5.11	9.67
False positives	222	187	304
False negatives	398	416	255
Combination-code recall	0.077	0.077	0.615
Sibling-code FP rate	0.329	0.219	0.227

Appendix

Table 6: Comparison of error patterns across method families on MDACE ICD-10-CM discharge summaries under the full-label-space setting.

Figure 5: Performance by ICD-10-CM chapter on MDACE under the full-label-space setting. The figure compares recall and F1 for LLM prompting, the CLH LLM-agent baseline, and KREL across five ICD chapters.

Metric	Value
True-positive code pairs	286
Evidence coverage	100%
Mention-anchor coverage	90.6%
Mean semantic cosine	0.746

Appendix

Table 7: Evidence alignment analysis on MDACE full-label-space ICD-10-CM discharge-summary evaluation. Evidence quality is evaluated only for true-positive code predictions.

Panel of Figure 6 — Figure 6: Prompt for extracting and formatting queries from clinical note samples.

Panel of Figure 7 — Figure 7: Instruction for Qwen3-Embedding-8B

Panel of Figure 8 — Figure 8: Instruction for Qwen3-Reranker-8B

Panel of Figure 9 — Figure 9: Prompt for rule-aware and evidence-grounded verification samples.

Automated Medical Coding

Automated medical coding uses clinical records and coding rules to assign standardized diagnosis and procedure codes for billing, care, and research.

See all 23 papers

Enhancing LLM Medical Coding with Structured External Knowledge
Apr 9, 2026Yidong Gan, David D. Nguyen, Yang Lin +4Automated Medical Coding Medical LLMs
Oracle Health and AI · The University of Sydney · CSIRO Data61
CoLa-ICD: A Knowledge-Enhanced Framework for Long-Tail Automated Medical Coding
Aug 31, 2026Yihang Cheng, Veronica Liesaputra, Andrew TrotmanLong-Tailed Classification Multi-Label Classification
University of Otago · Dunedin, New Zealand
Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding
Jun 11, 2026Ziqing Wang, Weihao Li, Shijie Chen +2Supervised Fine-Tuning Automated Medical Coding
Northwestern University

Apr 9, 2026J/K move · Enter open · S save

Enhancing LLM Medical Coding with Structured External Knowledge

Yidong Gan, David D. Nguyen, Yang Lin, Peter Zhong+3

Oracle Health and AI · The University of Sydney · CSIRO Data61

Automated Medical Coding Medical LLMs Knowledge Augmentation for Language Models Graph Retrieval-Augmented Generation

Open paper AskPDF arXiv

Accurate medical coding requires consulting authoritative resources such as the ICD tabular list and coding guidelines. Existing LLM-based automated methods largely rely on LLMs' internal knowledge, which is prone to hallucination and cannot keep pace with guideline updates. We introduce RAG-Coding, an agentic, training-free method that augments LLMs with structured external knowledge: the tabular list is encoded as a knowledge graph capturing hierarchical and instructional code relationships, and the guidelines are distilled into concise, code-specific summaries rather than retrieved as raw text. To enable our study, we also introduce MDACE-2025, expert re-annotations of the MDACE dataset under the 2025 ICD-10-CM/PCS guidelines, adding code sequencing and justification comments. On MDACE, RAG-Coding outperforms the best LLM-based baseline by 3--13% in micro-F1 across five LLM backbones, and achieves comparable micro- and macro-F1 to the supervised state-of-the-art, with higher recall (

+

11%) at the cost of precision (

-

6%). On MDACE-2025, RAG-Coding outperforms all baselines, demonstrating effective generalisation to updated guidelines. Ablations confirm stepwise gains, highlighting the importance of integrating structured external knowledge for LLM-based medical coding.

Abstract

Automated Medical Coding

Explore similar work

Enhancing LLM Medical Coding with Structured External Knowledge

CoLa-ICD: A Knowledge-Enhanced Framework for Long-Tail Automated Medical Coding

Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding