Enhancing LLM Medical Coding with Structured External Knowledge

Yidong Gan; David D. Nguyen; Yang Lin; Peter Zhong; Thanh Vu; Long Duong; Yuan-Fang Li

cs.CLApr 9, 2026

Enhancing LLM Medical Coding with Structured External Knowledge

Authors: Yidong Gan, David D. Nguyen, Yang Lin, Peter Zhong, Thanh Vu, Long Duong, Yuan-Fang Li

Organizations: Oracle Health and AI · The University of Sydney · CSIRO Data61

Abstract

Accurate medical coding requires consulting authoritative resources such as the ICD tabular list and coding guidelines. Existing LLM-based automated methods largely rely on LLMs' internal knowledge, which is prone to hallucination and cannot keep pace with guideline updates. We introduce RAG-Coding, an agentic, training-free method that augments LLMs with structured external knowledge: the tabular list is encoded as a knowledge graph capturing hierarchical and instructional code relationships, and the guidelines are distilled into concise, code-specific summaries rather than retrieved as raw text. To enable our study, we also introduce MDACE-2025, expert re-annotations of the MDACE dataset under the 2025 ICD-10-CM/PCS guidelines, adding code sequencing and justification comments. On MDACE, RAG-Coding outperforms the best LLM-based baseline by 3--13% in micro-F1 across five LLM backbones, and achieves comparable micro- and macro-F1 to the supervised state-of-the-art, with higher recall ( $+$ 11%) at the cost of precision ( $-$ 6%). On MDACE-2025, RAG-Coding outperforms all baselines, demonstrating effective generalisation to updated guidelines. Ablations confirm stepwise gains, highlighting the importance of integrating structured external knowledge for LLM-based medical coding.

Figures & tables

Figure 1: Overall architecture of the proposed RAG-Coding method. Blue lines denote input to the agents. Red lines denote output of the agents. Black lines denote intermediary operations that do not involve any agents.

Figure 2: Illustration of our guideline summariser agent’s retrieval process for an example code I27.20 (pulmonary hypertension, unspecified). The agent navigates the guidelines’ table of contents to retrieve the relevant general and chapter-specific guidelines applicable to I27.20.

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
Qwen3	Tree Search	0.08	0.17	0.11	0.05	0.05	0.04
	MedCodER	0.29	0.36	0.32	0.16	0.16	0.15
	CLH	0.27	0.49	0.35	0.17	0.18	0.17
	RAG-Coding (Ours)	0.43	0.39	0.41	0.20	0.18	0.18
Deepseek-V3	Tree Search	0.12	0.25	0.16	0.07	0.08	0.07

Table 1: Results on the MDACE test set. All methods use the entire code space during inference (

\approx

70K codes).

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
-	PLM-ICD	0.54	0.53	0.53	0.29	0.30	0.30
Qwen3	RAG-Coding (Ours)	0.47	0.39	0.43	0.21	0.20	0.19
Deepseek-V3	RAG-Coding (Ours)	0.50	0.43	0.46	0.27	0.24	0.24
GPT-4o	RAG-Coding (Ours)	0.44	0.48	0.46	0.21	0.22	0.21
GPT-4.1	RAG-Coding (Ours)	0.51	0.53	0.52	0.26	0.27	0.25

Table 2: Results on the MDACE test set. PLM-ICD covers only 20% of the ICD-10-CM code space (

\approx

16K codes) during inference. In contrast, RAG-Coding covers the entire code space (

\approx

70K codes) but applies post-processing to filter out any codes outside the 16K space used by PLM-ICD to make it a direct comparison.

#C	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
70K	Tree Search	0.20	0.28	0.24	0.12	0.13	0.12
	MedCodER	0.43	0.35	0.38	0.24	0.23	0.23
	CLH	0.45	0.44	0.45	0.26	0.27	0.26
	RAG-Coding (Ours)	0.57	0.55	0.56	0.34	0.34	0.33
16K	PLM-ICD	0.62	0.39	0.48	0.27	0.25	0.26

Table 3: Results on the MDACE-2025 test set. “#C” refers to the number of candidate codes used during inference. All LLM-based methods are evaluated with GPT-5 (low reasoning mode); full results with other LLM backbones are available in Appendix F .

	Micro
	Precision	Recall	F1
RAG-Coding (step 1)	0.34	0.54	0.41
RAG-Coding (steps 1-2)	0.42	0.53	0.47
RAG-Coding (steps 1-4)	0.48	0.54	0.51

Table 4: Ablation study on the MDACE test set (GPT-4.1 as the backbone LLM). Step 3 (retrieval and summarisation of guidelines) provides complementary information to step 4 and has no measurable impact on the metrics. Therefore, results for steps 1-3 are omitted, and the full RAG-Coding pipeline is reported as steps 1-4.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	Micro
Few-Shot Size	Precision	Recall	F1
$k=0$	0.30	0.53	0.38
$k=1$	0.36	0.51	0.42
$k=2$	0.40	0.51	0.44
$k=3$	0.38	0.50	0.43

Appendix

Table 5: Results of the Candidate Generator agent using different numbers of dynamic few-shot examples on the MDACE validation set.

Figure 3: Distribution of ICD-10-CM codes by chapter.

Dataset	# of unique ICD-10-CM diagnosis codes	# of unique ICD-10-PCS procedure codes	# of average codes per encounter	% of encounters with justifications
MDACE	320	43	11.11	-
MDACE-2025	451	16	14.84	11.48

Appendix

Table 6: Dataset statistics on the test split shared by MDACE and MDACE-2025. Key shifts include expanded diagnosis annotations, a reduction in procedure annotations attributable to guideline changes, and the addition of expert justification comments for complex coding decisions.

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
Qwen3	Tree Search	0.08 $\pm$ 0.01	0.17 $\pm$ 0.00	0.11 $\pm$ 0.01	0.05 $\pm$ 0.01	0.05 $\pm$ 0.00	0.04 $\pm$ 0.01
	MedCodER	0.29 $\pm$ 0.02	0.36 $\pm$ 0.03	0.32 $\pm$ 0.02	0.16 $\pm$ 0.01	0.16 $\pm$ 0.01	0.15 $\pm$ 0.01
	CLH	0.27 $\pm$ 0.03	0.49 $\pm$ 0.03	0.35 $\pm$ 0.03	0.17 $\pm$ 0.02	0.18 $\pm$ 0.01	0.17 $\pm$ 0.01
	RAG-Coding	0.43 $\pm$ 0.02	0.39 $\pm$ 0.01	0.41 $\pm$ 0.02	0.20 $\pm$ 0.01	0.18 $\pm$ 0.01	0.18 $\pm$ 0.01
Deepseek-V3	Tree Search	0.12 $\pm$ 0.01	0.25 $\pm$ 0.02	0.16 $\pm$ 0.02	0.07 $\pm$ 0.01	0.08 $\pm$ 0.01	0.07 $\pm$ 0.01

Appendix

Table 7: Extended version of Table 1 reporting mean

\pm

standard deviation over three runs on the MDACE test set.

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
-	PLM-ICD	0.54 $\pm$ 0.02	0.53 $\pm$ 0.02	0.53 $\pm$ 0.02	0.29 $\pm$ 0.02	0.30 $\pm$ 0.01	0.30 $\pm$ 0.02
Qwen3	RAG-Coding	0.47 $\pm$ 0.02	0.39 $\pm$ 0.02	0.43 $\pm$ 0.02	0.21 $\pm$ 0.01	0.20 $\pm$ 0.01	0.19 $\pm$ 0.02
Deepseek-V3	RAG-Coding	0.50 $\pm$ 0.03	0.43 $\pm$ 0.03	0.46 $\pm$ 0.03	0.27 $\pm$ 0.02	0.24 $\pm$ 0.01	0.24 $\pm$ 0.01
GPT-4o	RAG-Coding	0.44 $\pm$ 0.01	0.48 $\pm$ 0.01	0.46 $\pm$ 0.01	0.21 $\pm$ 0.01	0.22 $\pm$ 0.01	0.21 $\pm$ 0.01
GPT-4.1	RAG-Coding	0.51 $\pm$ 0.04	0.53 $\pm$ 0.01	0.52 $\pm$ 0.02	0.26 $\pm$ 0.01	0.27 $\pm$ 0.01	0.25 $\pm$ 0.01

Appendix

Table 8: Extended version of Table 2 reporting mean

\pm

standard deviation over three runs on the MDACE test set.

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
Qwen3	Tree Search	0.10 $\pm$ 0.01	0.15 $\pm$ 0.02	0.12 $\pm$ 0.01	0.06 $\pm$ 0.00	0.06 $\pm$ 0.00	0.05 $\pm$ 0.00
	MedCodER	0.32 $\pm$ 0.02	0.27 $\pm$ 0.02	0.29 $\pm$ 0.01	0.17 $\pm$ 0.01	0.15 $\pm$ 0.01	0.15 $\pm$ 0.01
	CLH	0.30 $\pm$ 0.02	0.38 $\pm$ 0.02	0.33 $\pm$ 0.02	0.18 $\pm$ 0.02	0.18 $\pm$ 0.01	0.17 $\pm$ 0.01
	RAG-Coding	0.46 $\pm$ 0.03	0.29 $\pm$ 0.02	0.35 $\pm$ 0.02	0.19 $\pm$ 0.02	0.17 $\pm$ 0.01	0.17 $\pm$ 0.02
Deepseek-V3	Tree Search	0.14 $\pm$ 0.01	0.22 $\pm$ 0.00	0.17 $\pm$ 0.00	0.09 $\pm$ 0.00	0.09 $\pm$ 0.00	0.08 $\pm$ 0.00

Appendix

Table 9: Extended version of Table 3 , covering more LLM backbones and reporting mean

\pm

standard deviation over three runs on the MDACE-2025 test set.

LLM	Method	Micro			Macro
		Precision	Recall	F1	Precision	Recall	F1
-	PLM-ICD	0.62 $\pm$ 0.02	0.39 $\pm$ 0.01	0.48 $\pm$ 0.02	0.27 $\pm$ 0.01	0.25 $\pm$ 0.01	0.26 $\pm$ 0.01
Qwen3	RAG-Coding	0.50 $\pm$ 0.02	0.29 $\pm$ 0.02	0.36 $\pm$ 0.02	0.20 $\pm$ 0.01	0.17 $\pm$ 0.01	0.18 $\pm$ 0.01
Deepseek-V3	RAG-Coding	0.52 $\pm$ 0.02	0.35 $\pm$ 0.02	0.42 $\pm$ 0.01	0.24 $\pm$ 0.01	0.20 $\pm$ 0.01	0.22 $\pm$ 0.01
GPT-4o	RAG-Coding	0.45 $\pm$ 0.02	0.34 $\pm$ 0.01	0.39 $\pm$ 0.01	0.19 $\pm$ 0.01	0.19 $\pm$ 0.01	0.18 $\pm$ 0.01
GPT-4.1	RAG-Coding	0.56 $\pm$ 0.03	0.40 $\pm$ 0.01	0.46 $\pm$ 0.01	0.25 $\pm$ 0.01	0.24 $\pm$ 0.00	0.23 $\pm$ 0.01

Appendix

Table 10: Extended version of Table 3 , covering more LLM backbones and reporting mean

\pm

standard deviation over three runs on the MDACE-2025 test set.

	Micro
	Precision	Recall	F1
RAG-Coding (step 1)	0.34	0.54	0.41
with SC	0.35	0.53	0.41
with SC $\times 2$	0.35	0.53	0.41

Appendix

Table 11: Ablation study on the MDACE test set. We use GPT-4.1 as the backbone LLM. ‘SC’ denotes self-correction, where the LLM validates the candidate codes (output of step 1) using only its internal knowledge. ‘SC

\times

2’ applies self-correction twice.

Reason for Code Removal	Accuracy
Symptom subsumed by diagnosis	13/13 (100%)
Wrong sub-code (laterality/specificity)	13/13 (100%)
Code description–note mismatch	8/8 (100%)
Undocumented/inferred diagnosis	5/5 (100%)
Comorbidity over-pruning	0/4 (0%)
Total	39/43 (91%)

Appendix

Table 12: Step 2 code removals categorised by reason. Accuracy is defined as the proportion of correct removals (i.e., the removed code was not a true target code).

	Tokens Consumed	API Cost ($ USD)
RAG-Coding (step 1)	13,046.30	0.03
RAG-Coding (step 2)	6,779.02	0.02
RAG-Coding (step 3)	85,636.73	0.17
RAG-Coding (step 4)	7,360.03	0.03
Total	112,822.08	0.25

Appendix

Table 13: Cost breakdown for RAG-Coding (GPT-4.1 backbone). This experiment was conducted on the MDACE validation set. Each row shows token (prompt and completion) usage and actual API cost for individual RAG-Coding components. The final row is the total across all components. Note: Step 3 output is cached, so its cost approaches zero in repeated/long-term use.

Automated Medical Coding

Automated medical coding uses clinical records and coding rules to assign standardized diagnosis and procedure codes for billing, care, and research.

See all 23 papers

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs
Aug 21, 2026Xubin Chen, Yipeng Zhou, Wen Sun +3Automated Medical Coding Medical LLMs
Macquarie University · Beijing Intelligent Decision Medical Technology Co. Ltd. · University of New South Wales +2
Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding
Jun 11, 2026Ziqing Wang, Weihao Li, Shijie Chen +2Supervised Fine-Tuning Automated Medical Coding
Northwestern University
Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data
Mar 6, 2026John Cook, Michael Wyatt, Peng Wei +11Automated Medical Coding Medical LLMs
Veradigm · Snowflake

Aug 21, 2026J/K move · Enter open · S save

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

Xubin Chen, Yipeng Zhou, Wen Sun, Chengkai Huang+2

Macquarie University · Beijing Intelligent Decision Medical Technology Co. Ltd. · University of New South Wales+2

Automated Medical Coding Medical LLMs LLM Reasoning Knowledge Augmentation for Language Models

Open paper AskPDF arXiv

Automatic Medical Coding (AMC), which assigns standardized International Classification of Diseases (ICD) codes to clinical notes, is essential for medical reimbursement, quality reporting, and clinical research. Existing pre-trained language model (PLM)-based methods typically formulate AMC as an extreme multi-label classification problem over a predefined code set, while recent large language model (LLM)-based approaches instead frame it as generation or multi-step reasoning. However, key challenges remain, including the extreme length of clinical notes that hinders effective interpretation, the vast ICD label space, and complex coding rules that are not explicitly captured by LLMs. In this work, we propose Knowledge-Guided Reasoning over Clinical Evidence with LLMs (KREL), a framework that leverages LLMs for clinical text understanding and reasoning while integrating external ICD coding guidelines as structured knowledge. This design enables tight coupling between domain knowledge and LLM reasoning, reducing hallucinations and improving compliance with coding standards. Experiments on benchmark datasets show that KREL consistently outperforms strong PLM-based and state-of-the-art LLM-based baselines.

Abstract

Automated Medical Coding

Explore similar work

KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs

Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding

Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data