Understanding the Limits of Agentic ICD Coding

Chong Yock Eng; Yushi Cao; Yiming Chen; Kezhi Mao; Hongchao Jiang

cs.CLSep 12, 2026

Understanding the Limits of Agentic ICD Coding

Authors: Chong Yock Eng, Yushi Cao, Yiming Chen, Kezhi Mao, Hongchao Jiang

Organizations: Nanyang Technological University, Singapore · ASUS Intelligent Cloud Services (AICS), Singapore

Abstract

ICD-10-CM codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting. Standard ICD-10-CM benchmarks report aggregate metrics that obscure performance on complex coding scenarios. We evaluate neural, workflow, and agentic systems on a rarity-stratified set of MIMIC-IV discharge summaries and identify two orthogonal failure modes. Neural classifiers exhibit a 0.43 micro-F1 gap between rare and common codes. Workflow systems handle rare codes well but score near zero on injury and external cause codes that require multi-step guideline following. A tool-augmented agentic configuration with structured access to official ICD-10-CM reference materials recovers up to 0.34 micro-F1 on this subset. No single system dominates across all conditions.

Figures & tables

Figure 1: Comparison of three AI paradigms for automated clinical coding

	All Codes ( $n{=}150$ )			Multi-Step Codes ( $n{=}153$ )
Model	Rare	Unc.	Common	Rare	Unc.	Common
PLM-ICD	.227 [.17,.28]	.471 [.44,.51]	.656 [.63,.68]	.256 [.20,.31]	.375 [.30,.45]	.435 [.36,.51]
CodeSeeker (Opus 4.6)	.560 [.50,.61]	.483 [.45,.52]	.607 [.58,.63]	.010 [.00,.03]	.027 [.00,.07]	.013 [.00,.04]
CodeSeeker (GPT-5.2)	.341 [.26,.42]	.353 [.31,.39]	.436 [.40,.47]	.021 [.00,.05]	.028 [.00,.07]	.000 [.00,.00]
Claude Opus 4.6	.354 [.30,.41]	.425 [.39,.46]	.577 [.55,.60]	.276 [.22,.33]	.363 [.28,.45]	.280 [.21,.35]
GPT-5.2	.329 [.27,.39]	.368 [.33,.40]	.538 [.51,.57]	.190 [.13,.25]	.250 [.17,.33]	.265 [.19,.34]

Table 1: Micro-F1 across rarity bins for all codes (150 notes, 50 per bin) and the multi-step navigation subset (153 notes; 193/142/145 code instances in the rare/uncommon/common bins). Unc. : uncommon (101–1,000 training occurrences); Common :

>

1,000. Point estimate with 95% bootstrap CI in brackets. Best point estimate per column in bold.

Panel of Figure 2 — Figure 2: Distribution of median relative discovery position for correct vs. incorrect codes. Statistical test in Table 11 .

Model	Disc.	Sub%	Prec%	Calls/TP
Claude Opus 4.6	67K	2.8	56	8.7
GPT-5.2	63K	3.1	50	8.4
GLM-5 Turbo	124K	1.7	49	11.4
Mistral Large 3	72K	3.6	36	12.0
Qwen 3.5 Plus	117K	1.7	49	10.9
Kimi K2.5	178K	1.6	36	13.5

Table 2: Coding funnel. Sub% = submitted / discovered. Prec% = TP / submitted. Calls/TP = tool calls per correct code.

Chapter	Disc.	Verif.	Sub./Spec.
Ch. 19 S/T (injury)	66%	9%	3%
Ch. 20 V/W/X (mechanism)	86%	0%	0%
Ch. 20 Y92 (place)	83%	6%	0.3%
Ch. 20 Y-other	79%	14%	0%

Table 3: Per-stage failure decomposition for missed multi-step codes, by chapter.

Panel of Figure 3 — Figure 3: Performance of PLM-ICD as a tool.

Model	Config	Rare	Unc.	Com.
Opus 4.6	full prompt	.267	.357	.318
	full prompt	[.10,.46]	[.21,.50]	[.19,.44]
	seq. removed	.241	.385	.316
	seq. removed	[.08,.44]	[.24,.53]	[.18,.44]
GPT-5.2	full prompt	.151	.243	.322
	full prompt	[.00,.30]	[.11,.37]	[.19,.44]

Table 4: Prompt-level sequencing ablation.

		All Codes		Multi-Step
Model	Band	Tools	None	Tools	None
Opus 4.6	Rare	.354	.182	.276	.142
	Unc.	.425	.285	.363	.249
	Com.	.577	.498	.280	.050
Qwen 3.5 Plus	Rare	.281	.153	.220	.110
	Unc.	.364	.314	.291	.130

Table 5: Tool-access ablation, micro-F1. Tools : full nine-tool environment. None : no tool access. Precision, recall, support, and confidence intervals in Appendix C .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

With PLM-ICD tool
Model	Context	PLM	Reason.
Claude Opus 4.6 + PLM	200K	$\bullet$	$\circ$
Claude Opus 4.6	200K	$\circ$	$\circ$
GPT-5.2 + PLM	1M	$\bullet$	$\bullet$
GPT-5.2	1M	$\circ$	$\bullet$
No PLM-ICD tool

Appendix

Table 6: Model configurations.

\bullet

\circ

= enabled/disabled. PLM : PLM-ICD neural primer. Reason. : chain-of-thought reasoning. All models share the same tool environment (200 turns, 200 tool calls, 600s timeout, max_tokens =16,384).

Navigation (discovery-gated)
Tool	Function
suggest_candidate_codes	PLM-ICD Tool; disabled in no-PLM runs
search_alphabetical_index	Index search and hierarchical drill
search_drug_table	Substance-to-code lookup
search_neoplasm_table	Anatomical site-to-code lookup
search_external_cause_index	Cause, place, and activity codes

Appendix

Table 7: Tool environment. Navigation tools populate the discovery set; verification and submission are gated to that set.

			Full-tools			No-tools
Model	Band	Sup	Precision	Recall	F1	Precision	Recall	F1
Opus 4.6	Rare	233	.317 [.26,.38]	.399 [.33,.47]	.354 [.30,.41]	.127 [.10,.15]	.322 [.27,.38]	.182 [.15,.21]
	Uncommon	660	.467 [.43,.51]	.391 [.35,.43]	.425 [.39,.46]	.245 [.21,.27]	.341 [.30,.38]	.285 [.25,.32]
	Common	1403	.674 [.65,.70]	.505 [.47,.54]	.577 [.55,.60]	.543 [.51,.57]	.460 [.43,.49]	.498 [.47,.52]
Qwen 3.5 Plus	Rare	233	.230 [.19,.27]	.360 [.30,.42]	.281 [.23,.33]	.107 [.08,.13]	.266 [.21,.33]	.153 [.12,.19]
	Uncommon	660	.389 [.35,.43]	.342 [.31,.38]	.364 [.33,.40]	.302 [.27,.34]	.326 [.29,.36]	.314 [.28,.35]

Appendix

Table 8: Tool-access ablation across all codes.

			Full-tools			No-tools
Model	Band	Sup	Precision	Recall	F1	Precision	Recall	F1
Opus 4.6	Rare	193	.283 [.22,.34]	.269 [.20,.34]	.276 [.22,.33]	.127 [.09,.17]	.161 [.11,.21]	.142 [.10,.19]
	Uncommon	142	.488 [.39,.60]	.289 [.21,.37]	.363 [.28,.45]	.388 [.27,.51]	.183 [.12,.25]	.249 [.17,.33]
	Common	145	.333 [.25,.42]	.241 [.18,.31]	.280 [.21,.35]	.267 [.06,.50]	.028 [.01,.06]	.050 [.01,.10]
Qwen 3.5 Plus	Rare	193	.234 [.16,.32]	.207 [.15,.27]	.220 [.16,.29]	.102 [.06,.14]	.119 [.07,.17]	.110 [.06,.15]
	Uncommon	142	.437 [.31,.57]	.218 [.15,.30]	.291 [.20,.38]	.279 [.15,.42]	.085 [.04,.13]	.130 [.07,.20]

Appendix

Table 9: Tool-access ablation across multi-step codes.

No PLM-ICD tool
Model	Lat.	Tok.	Calls	Sub.	Fail
	(s)	(K)		(%)	(%)
Claude Opus 4.6	204	25.6	64	100	0
GPT-5.2	91	19.4	59	100	0
GLM-5 Turbo	224	26.3	78	88	12
Qwen 3.5 Plus	363	26.2	71	92	8

Appendix

Table 10: Per-model resource usage.

Model	$n$ (TP/FP)	$\Delta$ Med.	$\delta$	$p$
Claude Opus 4.6	1,026 / 770	+0.079	0.233	$<\!10^{-16}$
GPT-5.2	972 / 955	+0.106	0.200	$<\!10^{-13}$
GLM-5 Turbo	991 / 1,007	+0.062	0.239	$<\!10^{-19}$
Mistral Large 3	904 / 1,544	+0.163	0.261	$<\!10^{-26}$
Qwen 3.5 Plus	971 / 981	+0.140	0.216	$<\!10^{-15}$
Kimi K2.5	993 / 1,753	+0.095	0.301	$<\!10^{-38}$

Appendix

Table 11: Statistical test of the TP–FP discovery-position gap.

Code	Decision	Rationale
K52.9	Include	Discharge dx; Section III.C applies
K35.80	Include	Discharge dx; CT confirms; no peritonitis
G47.33	Include	Chronic PMH condition; inpatient convention
R59.0	Exclude	Integral finding; Guideline I.B.5
D72.829	Exclude	Lab finding; integral to process
Z87.891	Exclude	Not documented in note

Appendix

Table 12: Inclusion and exclusion decisions for sample 11087585-DS-10.

Automated Medical Coding

Automated medical coding uses clinical records and coding rules to assign standardized diagnosis and procedure codes for billing, care, and research.

See all 23 papers

Bridging the Version Gap: Multi-version Training Improves ICD Code Prediction, Especially for Rare Codes
May 18, 2026Jinghui Liu, Anthony NguyenAutomated Medical Coding Healthcare
Australian e-Health Research Centre, CSIRO
Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting
Aug 17, 2026Junda Wang, Meysam Ghaffari, Akshat Choube +3Clinical Prediction Deep Research Agents
University of Massachusetts Amherst, MA, USA · Optum, MN, USA · University of Massachusetts Lowell, MA, USA
Multimodal Dual-Encoder Retrieval for Automated ICD Coding
Oct 3, 2026Abhinav Bohra, Anuj BohraAutomated Medical Coding Multimodal Learning
Amazon Seattle, USA · Rutgers University New Jersey, USA

May 18, 2026J/K move · Enter open · S save

Bridging the Version Gap: Multi-version Training Improves ICD Code Prediction, Especially for Rare Codes

Jinghui Liu, Anthony Nguyen

Australian e-Health Research Centre, CSIRO

Automated Medical Coding Healthcare

Open paper AskPDF arXiv

Clinical coding maps clinical documentation to standardized medical codes, an essential yet time-consuming administrative task that could benefit from automation. Current models on ICD coding are typically optimized for codes from a specific ICD version. However, in reality, ICD systems evolve continuously, and different versions are adopted across time periods and regions. Moreover, ICD coding suffers from the long-tail problem, and rare code performance can be a bottleneck for developing implementable models. We examine whether it is viable to train version-independent models by combining data annotated in different ICD versions, which may help address these challenges. We add ICD-9 data to the training of a modified label-wise attention model for ICD-10 prediction, and find that despite the version mismatch, adding ICD-9 yields a 27% increase in micro F1 for 18K rare ICD codes compared to training on ICD-10 alone. On 8K frequent ICD-10 codes, the multi-version training also substantially improves macro metrics, with far fewer model parameters.

Abstract

Automated Medical Coding

Explore similar work

Bridging the Version Gap: Multi-version Training Improves ICD Code Prediction, Especially for Rare Codes

Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting

Multimodal Dual-Encoder Retrieval for Automated ICD Coding