ICD-10-CM codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting. Standard ICD-10-CM benchmarks report aggregate metrics that obscure performance on complex coding scenarios. We evaluate neural, workflow, and agentic systems on a rarity-stratified set of MIMIC-IV discharge summaries and identify two orthogonal failure modes. Neural classifiers exhibit a 0.43 micro-F1 gap between rare and common codes. Workflow systems handle rare codes well but score near zero on injury and external cause codes that require multi-step guideline following. A tool-augmented agentic configuration with structured access to official ICD-10-CM reference materials recovers up to 0.34 micro-F1 on this subset. No single system dominates across all conditions.
Figures & tables
Figure 1: Comparison of three AI paradigms for automated clinical coding
All Codes ( n=150 )
Multi-Step Codes ( n=153 )
Model
Rare
Unc.
Common
Rare
Unc.
Common
PLM-ICD
.227 [.17,.28]
.471 [.44,.51]
.656 [.63,.68]
.256 [.20,.31]
.375 [.30,.45]
.435 [.36,.51]
CodeSeeker (Opus 4.6)
.560 [.50,.61]
.483 [.45,.52]
.607 [.58,.63]
.010 [.00,.03]
.027 [.00,.07]
.013 [.00,.04]
CodeSeeker (GPT-5.2)
.341 [.26,.42]
.353 [.31,.39]
.436 [.40,.47]
.021 [.00,.05]
.028 [.00,.07]
.000 [.00,.00]
Claude Opus 4.6
.354 [.30,.41]
.425 [.39,.46]
.577 [.55,.60]
.276 [.22,.33]
.363 [.28,.45]
.280 [.21,.35]
GPT-5.2
.329 [.27,.39]
.368 [.33,.40]
.538 [.51,.57]
.190 [.13,.25]
.250 [.17,.33]
.265 [.19,.34]
Table 1: Micro-F1 across rarity bins for all codes (150 notes, 50 per bin) and the multi-step navigation subset (153 notes; 193/142/145 code instances in the rare/uncommon/common bins). Unc. : uncommon (101–1,000 training occurrences); Common : > 1,000. Point estimate with 95% bootstrap CI in brackets. Best point estimate per column in bold.
Figure 2: Distribution of median relative discovery position for correct vs. incorrect codes. Statistical test in Table 11 .
Table 3: Per-stage failure decomposition for missed multi-step codes, by chapter.
Figure 3: Performance of PLM-ICD as a tool.
Model
Config
Rare
Unc.
Com.
Opus 4.6
full prompt
.267
.357
.318
[.10,.46]
[.21,.50]
[.19,.44]
seq. removed
.241
.385
.316
[.08,.44]
[.24,.53]
[.18,.44]
GPT-5.2
full prompt
.151
.243
.322
[.00,.30]
[.11,.37]
[.19,.44]
Table 4: Prompt-level sequencing ablation.
All Codes
Multi-Step
Model
Band
Tools
None
Tools
None
Opus 4.6
Rare
.354
.182
.276
.142
Unc.
.425
.285
.363
.249
Com.
.577
.498
.280
.050
Qwen 3.5 Plus
Rare
.281
.153
.220
.110
Unc.
.364
.314
.291
.130
Table 5: Tool-access ablation, micro-F1. Tools : full nine-tool environment. None : no tool access. Precision, recall, support, and confidence intervals in Appendix C .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Context
PLM
Reason.
With PLM-ICD tool
Claude Opus 4.6 + PLM
200K
∙
∘
Claude Opus 4.6
200K
∘
∘
GPT-5.2 + PLM
1M
∙
∙
GPT-5.2
1M
∘
∙
No PLM-ICD tool
Appendix
Table 6: Model configurations. ∙ / ∘ = enabled/disabled. PLM : PLM-ICD neural primer. Reason. : chain-of-thought reasoning. All models share the same tool environment (200 turns, 200 tool calls, 600s timeout, max_tokens =16,384).
Tool
Function
Navigation (discovery-gated)
suggest_candidate_codes
PLM-ICD Tool; disabled in no-PLM runs
search_alphabetical_index
Index search and hierarchical drill
search_drug_table
Substance-to-code lookup
search_neoplasm_table
Anatomical site-to-code lookup
search_external_cause_index
Cause, place, and activity codes
Appendix
Table 7: Tool environment. Navigation tools populate the discovery set; verification and submission are gated to that set.
Full-tools
No-tools
Model
Band
Sup
Precision
Recall
F1
Precision
Recall
F1
Opus 4.6
Rare
233
.317 [.26,.38]
.399 [.33,.47]
.354 [.30,.41]
.127 [.10,.15]
.322 [.27,.38]
.182 [.15,.21]
Uncommon
660
.467 [.43,.51]
.391 [.35,.43]
.425 [.39,.46]
.245 [.21,.27]
.341 [.30,.38]
.285 [.25,.32]
Common
1403
.674 [.65,.70]
.505 [.47,.54]
.577 [.55,.60]
.543 [.51,.57]
.460 [.43,.49]
.498 [.47,.52]
Qwen 3.5 Plus
Rare
233
.230 [.19,.27]
.360 [.30,.42]
.281 [.23,.33]
.107 [.08,.13]
.266 [.21,.33]
.153 [.12,.19]
Uncommon
660
.389 [.35,.43]
.342 [.31,.38]
.364 [.33,.40]
.302 [.27,.34]
.326 [.29,.36]
.314 [.28,.35]
Appendix
Table 8: Tool-access ablation across all codes.
Full-tools
No-tools
Model
Band
Sup
Precision
Recall
F1
Precision
Recall
F1
Opus 4.6
Rare
193
.283 [.22,.34]
.269 [.20,.34]
.276 [.22,.33]
.127 [.09,.17]
.161 [.11,.21]
.142 [.10,.19]
Uncommon
142
.488 [.39,.60]
.289 [.21,.37]
.363 [.28,.45]
.388 [.27,.51]
.183 [.12,.25]
.249 [.17,.33]
Common
145
.333 [.25,.42]
.241 [.18,.31]
.280 [.21,.35]
.267 [.06,.50]
.028 [.01,.06]
.050 [.01,.10]
Qwen 3.5 Plus
Rare
193
.234 [.16,.32]
.207 [.15,.27]
.220 [.16,.29]
.102 [.06,.14]
.119 [.07,.17]
.110 [.06,.15]
Uncommon
142
.437 [.31,.57]
.218 [.15,.30]
.291 [.20,.38]
.279 [.15,.42]
.085 [.04,.13]
.130 [.07,.20]
Appendix
Table 9: Tool-access ablation across multi-step codes.
Model
Lat.
Tok.
Calls
Sub.
Fail
(s)
(K)
(%)
(%)
No PLM-ICD tool
Claude Opus 4.6
204
25.6
64
100
0
GPT-5.2
91
19.4
59
100
0
GLM-5 Turbo
224
26.3
78
88
12
Qwen 3.5 Plus
363
26.2
71
92
8
Appendix
Table 10: Per-model resource usage.
Model
n (TP/FP)
Δ Med.
δ
p
Claude Opus 4.6
1,026 / 770
+0.079
0.233
<10−16
GPT-5.2
972 / 955
+0.106
0.200
<10−13
GLM-5 Turbo
991 / 1,007
+0.062
0.239
<10−19
Mistral Large 3
904 / 1,544
+0.163
0.261
<10−26
Qwen 3.5 Plus
971 / 981
+0.140
0.216
<10−15
Kimi K2.5
993 / 1,753
+0.095
0.301
<10−38
Appendix
Table 11: Statistical test of the TP–FP discovery-position gap.
Code
Decision
Rationale
K52.9
Include
Discharge dx; Section III.C applies
K35.80
Include
Discharge dx; CT confirms; no peritonitis
G47.33
Include
Chronic PMH condition; inpatient convention
R59.0
Exclude
Integral finding; Guideline I.B.5
D72.829
Exclude
Lab finding; integral to process
Z87.891
Exclude
Not documented in note
Appendix
Table 12: Inclusion and exclusion decisions for sample 11087585-DS-10.
Clinical coding maps clinical documentation to standardized medical codes, an essential yet time-consuming administrative task that could benefit from automation. Current models on ICD coding are typically optimized for codes from a specific ICD version. However, in reality, ICD systems evolve continuously, and different versions are adopted across time periods and regions. Moreover, ICD coding suffers from the long-tail problem, and rare code performance can be a bottleneck for developing implementable models. We examine whether it is viable to train version-independent models by combining data annotated in different ICD versions, which may help address these challenges. We add ICD-9 data to the training of a modified label-wise attention model for ICD-10 prediction, and find that despite the version mismatch, adding ICD-9 yields a 27% increase in micro F1 for 18K rare ICD codes compared to training on ICD-10 alone. On 8K frequent ICD-10 codes, the multi-version training also substantially improves macro metrics, with far fewer model parameters.
Next-encounter ICD forecasting predicts which standardized diagnosis codes will be documented at a future visit from the longitudinal record available beforehand. The task is prospective and multi-label: the target note does not yet exist, and several codes may be correct. Structured EHR foundation models capture recurrence and temporal progression, whereas language foundation models generate flexible diagnostic hypotheses. We introduce ICD-Deepresearch, a DeepResearch workflow that composes these predictive foundation models with medical search and ICD dictionaries. Because no source reveals the future code set, research evaluates candidate transitions by linking patient evidence, external clinical relations, and exact code semantics under a fixed top-K budget. Candidate Generation uses SparseEHR to produce an EHR Prior that initializes two bounded Research Expansion rounds; an independent GPT-5 Direct Forecast supplies complementary candidates. Final Selection validates, deduplicates, and jointly ranks both paths, after which a separate module writes rationales without changing predictions. Finally ICD-Deepresearch achieves patient-averaged precision/recall of 24.60/35.09% on MIMIC-III and 25.14/48.32% on MIMIC-IV. Physicians rate 51% and 68% of its retrieved documents useful, compared with 22% and 39% for standalone GPT-5 web search and 32% and 41% for Medical Deep Research. ICD-Deepresearch therefore improves over the registered local comparators while retrieving evidence with higher physician-rated usefulness than the standalone research systems
Junda Wang, Meysam Ghaffari, Akshat Choube +3
University of Massachusetts Amherst, MA, USA · Optum, MN, USA · University of Massachusetts Lowell, MA, USA
Accurate International Classification of Diseases (ICD) coding is crucial for large-scale clinical research, documentation, and billing. There are three primary problems with current ICD prediction methods: (1) They are unable to comprehend multimodal patient data because they rely on either structured EHR data or unstructured clinical notes. (2) They also struggle with scalability to a larger amount of ICD codes (9K+ codes in ICD-9), as traditional classifiers need dense output layers and often do not generalize well to long tail rare diseases. (3) They lack transparency for clinical use. To address these challenges, this research proposes a two-stage framework that first retrieves ICD codes using a multimodal dual-encoder retrieval model, where structured and unstructured patient data are integrated through gated fusion. The second stage refines the top-k retrieved candidates with an LLM-based re-ranker that provides ranked codes with clinically relevant explanations. Our experiments show that the proposed approach improves Micro-F1 and Precision over a multimodal dual-fusion classifier baseline. These improvements demonstrate that combining a gated multimodal retrieval system with LLM-based re-ranking is a practical alternative to dense multi-label classification for automated ICD coding.
Abhinav Bohra, Anuj Bohra
Amazon Seattle, USA · Rutgers University New Jersey, USA