Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting

Junda Wang; Meysam Ghaffari; Akshat Choube; Mohsen Sharifi Renani; Hong Yu; Carlos Morato

cs.CLAug 17, 2026

Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting

Authors: Junda Wang, Meysam Ghaffari, Akshat Choube, Mohsen Sharifi Renani, Hong Yu, Carlos Morato

Organizations: University of Massachusetts Amherst, MA, USA · Optum, MN, USA · University of Massachusetts Lowell, MA, USA

Abstract

Next-encounter ICD forecasting predicts which standardized diagnosis codes will be documented at a future visit from the longitudinal record available beforehand. The task is prospective and multi-label: the target note does not yet exist, and several codes may be correct. Structured EHR foundation models capture recurrence and temporal progression, whereas language foundation models generate flexible diagnostic hypotheses. We introduce ICD-Deepresearch, a DeepResearch workflow that composes these predictive foundation models with medical search and ICD dictionaries. Because no source reveals the future code set, research evaluates candidate transitions by linking patient evidence, external clinical relations, and exact code semantics under a fixed top-K budget. Candidate Generation uses SparseEHR to produce an EHR Prior that initializes two bounded Research Expansion rounds; an independent GPT-5 Direct Forecast supplies complementary candidates. Final Selection validates, deduplicates, and jointly ranks both paths, after which a separate module writes rationales without changing predictions. Finally ICD-Deepresearch achieves patient-averaged precision/recall of 24.60/35.09% on MIMIC-III and 25.14/48.32% on MIMIC-IV. Physicians rate 51% and 68% of its retrieved documents useful, compared with 22% and 39% for standalone GPT-5 web search and 32% and 41% for Medical Deep Research. ICD-Deepresearch therefore improves over the registered local comparators while retrieving evidence with higher physician-rated usefulness than the standalone research systems

Figures & tables

Figure 1: Foundation models as forecasting tools. ICD-Deepresearch uses structured-EHR and language foundation models as complementary candidate generators, researches patient-anchored EHR candidates, and jointly selects all candidates under a fixed top- $K$ budget.

Figure 2: ICD-Deepresearch pipeline. The EHR Prior initializes two research rounds; Direct Forecast remains independent until Final Selection. The Evidence-Aware Reranker defines the full model, RRF is a same-pool selector control, and rationales are written only after predictions are fixed. Details appear in Supplement Sec. S2.

Published reference results
	MIMIC-III				MIMIC-IV
Method	P@10	R@10	P@20	R@20	P@10	R@10	P@20	R@20
CAML	20.61	18.15	14.98	25.74	20.61	27.38	14.03	35.52
ZAGCNN	17.50	15.51	12.61	21.63	21.10	28.13	14.33	36.17
GatorTron	20.53	17.72	15.03	25.66	22.87	30.19	15.47	38.81
DistilBioBERT	20.18	17.85	14.98	25.62	22.59	29.83	15.33	38.50

Table 1: Exact-

K

forecasting results. P@

K

/R@

K

are patient-averaged. Published THCM-CAL-protocol values are transcribed from its Table 2 ( Zhang et al. 2025 ) and form a separate block. Boldface indicates the best result within each block.

Dataset	System variant	Micro-P	Micro-R	Micro-F1	Macro-F1
Panel A: individual candidate paths and joint selection
MIMIC-III	ICD-Deepresearch: Candidate Generation Only	21.54	30.45	25.23	24.12
	ICD-Deepresearch: Direct Forecast Only	21.80	30.90	25.56	24.80
	ICD-Deepresearch: Research Path Only	20.71	29.29	24.27	23.53
	ICD-Deepresearch w/o Evidence-Aware Reranker	22.81	32.26	26.73	25.88
	ICD-Deepresearch	24.97	35.30	29.25	28.21

Table 2: Component analysis at

K=20

. Panels compare isolated paths and joint selectors (A), standalone research comparators (B), and research context under a fixed candidate pool (C). Values are percentages.

		Retrieval volume		Source composition (%)							Metadata proxy (%)		Blinded audit (%)
Dataset	System	Uses	Uniq.	Peer	Offic.	Clin.	Coding	Repo.	Forum	Other	Supp.	Fcast.	Phys.
MIMIC-III	GPT-5 + Web Search	13,394	6,363	7.3	19.0	1.1	30.7	6.6	16.2	19.1	30.0	6.0	22.0
	Medical Deep Research	37,724	26,268	44.2	12.7	2.8	25.7	0.5	0.0	14.1	58.0	34.0	32.0
	Research Expansion	1,410	1,246	34.2	6.8	7.2	29.3	0.1	0.0	22.4	62.0	22.0	51.0
MIMIC-IV	GPT-5 + Web Search	25,100	12,589	1.1	16.3	0.5	21.2	5.0	21.3	34.6	34.0	4.0	39.0
	Medical Deep Research	37,989	28,072	45.2	19.2	3.3	7.5	0.4	0.0	24.3	50.0	34.0	41.0

Table 3: Retrieved-source composition and usefulness audits. Uses/Uniq. are counts after excluding deterministic dictionary citations; all other entries are percentages. Abbreviations and sampling are defined in Supplement Sec. S3.6. Bold identifies Research Expansion, not a columnwise maximum.

Figure 3: Post-selection explanation quality. Panels (a–b) show Reason5 on each system’s true-positive predictions; panel (c) combines reason quality with coverage at

K=20

. The ICD-Deepresearch arm is RRF-selected. Exact values and the judge contract appear in Supplement Secs. S6.1 and S3.8.

Figure 4: Candidate expansion before Final Selection. Panels show target recall, pool growth, match yield against the exact micro-F1/2 break-even threshold, and the resulting

\Delta

micro-F1.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Analysis	Data and compared functions	Controlled state and metric	Estimand
Primary forecasting	MIMIC-III/IV scale cohorts: ICD-Deepresearch: Candidate Generation Only, GPT-5 + Web Search (Standalone), and full ICD-Deepresearch	Complete exact- $K$ rankings at $K\in\{10,20\}$ ; patient-averaged P@ $K$ /R@ $K$	System-level forecasting performance
Component and selector controls	Diagnostic cohorts ( $n=100$ per dataset): Candidate Generation Only, Direct Forecast Only, Research Path Only, ICD-Deepresearch w/o Evidence-Aware Reranker, and full ICD-Deepresearch	Same frozen patient IDs and targets; three independently executed $K=20$ runs; mean pooled micro-P/R/F1 and mean patient-macro F1	Contribution of candidate paths and selector choice within each diagnostic cohort
External comparators	Diagnostic cohorts ( $n=100$ per dataset): GPT-5 + Web Search (Standalone) and Medical Deep Research (Standalone)	Same patient IDs, targets, failure policy, and three-run $K=20$ contract as the corresponding component rows; no ICD-Deepresearch internal state	Search-enabled forecasting under each recorded policy
Fixed-pool selector control	Separate $n=100$ cohort per dataset; candidate IDs with versus without Research Expansion context	Within each run, the same patient IDs, candidate IDs, randomized order, backend, and $K=20$ ; three independently executed runs and mean pooled micro and patient-macro metrics	Contribution of the delivered research packet after candidate generation
Iteration diagnostic	Research Expansion and retention rules	Common diagnostic snapshot and target construction	Coverage–cardinality and support-score behavior
Source audit	Research Expansion, GPT-5 + Web Search (Standalone), and Medical Deep Research (Standalone)	Source-occurrence sampling and blinded labels	Composition and clinical usefulness of retrieved documents

Appendix

Table S1: Protocol and analysis registry. The scale-cohort primary forecasting analysis and the subset diagnostic analyses are distinct evaluation contracts. Data, controlled state, metric, and estimand are shown for each analysis.

Dataset	Component or control	Micro-P	Micro-R	Micro-F1	Macro-F1
Panel A: isolated paths and registered joint selectors
MIMIC-III	ICD-Deepresearch: Candidate Generation Only	21.54	30.45	25.23	24.12
	ICD-Deepresearch: Direct Forecast Only	21.80	30.90	25.56	24.80
	ICD-Deepresearch: Research Path Only	20.71	29.29	24.27	23.53
	ICD-Deepresearch w/o Evidence-Aware Reranker	22.81	32.26	26.73	25.88
	ICD-Deepresearch	24.97	35.30	29.25	28.21

Appendix

Table S2: Component paths, registered selectors, and fixed-pool controls at

K=20

. Panels A–B use a development-disjoint random MIMIC-III diagnostic cohort (

n=100

) and a frozen-order MIMIC-IV diagnostic cohort (

n=100

). Panel C uses a separately registered

n=100

fixed-pool cohort per dataset. Every configuration is executed independently five times on the same frozen patients; each entry is the arithmetic mean of the five run-level metric exports and is displayed to two decimals. Within each dataset block, Panels A–B use identical patient IDs and targets across configurations and runs. Within each Panel C run, all arms use identical candidate identities and randomized display order; research-derived candidate identities remain in every arm. Panel A contains isolated path controls and registered joint-selector outputs. Candidate Generation Only evaluates the complete seed-generation module, for which SparseEHR is the structured-EHR backbone. Research Path Only runs Candidate Generation followed by Research Expansion and evaluates the leading 20 entries of the complete research ordering

\bar{R}_{i}

; it does not use the potentially shorter EOV-thresholded branch

R_{i}

. Rows ending in Only are not drop-one ablations. ICD-Deepresearch w/o Evidence-Aware Reranker retains the complete candidate state and uses reciprocal-rank fusion for final selection. The full Evidence-Aware Reranker is reported for both diagnostic cohorts. Panel B reports the standalone research comparators. Panel C fixes candidate identities and randomized order while varying selector context or backend; research-derived candidate identities remain in every fixed-pool arm. Target-conditioned oracles are computed within each run’s observed candidate pool and then averaged across runs. Bold marks the best observed deployable five-run mean within each dataset block; the Micro columns use pooled counts within each run and all metrics are percentages.

Configuration	Mean size	TP	Patient-macro F1	Micro-F1
Research Round 1 only	36.03	650	0.25475	0.26026
Unconditional Research Round 2	38.19	678	0.25465	0.26022
Support score $\geq 50$	36.64	671	0.25869	0.26543
Beyond code lookup	37.38	661	0.25250	0.25770
Readable source	37.35	661	0.25266	0.25785

Appendix

Table S3: MIMIC-IV Research Round 2 retention diagnostics from the common Research Round 1 state. Mean size is the final candidate-set size and TP is the number of matches among 1,392 held-out targets. Rules are evaluated post hoc on finalized outputs. Patient-macro F1 and micro-F1 are reported as proportions on a 0–1 scale.

Dataset	System	$N$	Queries /pt	Retrieval ops /pt	Source access /pt	Tokens /pt	Observed pt/hour	Cost /pt	Micro- F1@20
MIMIC-III	GPT-5 + Web Search (Standalone)	100	55.1	19.5	134.2	110.0k	10.9	$0.458	26.4
MIMIC-III	Research Expansion	100	7.6	51.5	35.7	$\approx$ 18.1k	182.6	$0.020–0.039 †	30.2
MIMIC-IV	GPT-5 + Web Search (Standalone)	100	37.8	12.8	251.0	77.3k	12.9	$0.317	33.3
MIMIC-IV	Research Expansion	100	11.0	78.7	55.8	$\approx$ 26.4k	282.7	$0.021–0.049 †	34.8

Appendix

Table S4: Online-retrieval efficiency on the matched prediction cohorts. Operational quantities are normalized per patient.

Figure S1: EHRSHOT candidate-generation trajectory at natural cardinality (

n=120

). (a) Patient-averaged and micro recall across EHR Prior, Research Round 1, and Research Round 2; labels report newly matched targets. (b) Mean candidate-pool size over the same states, characterizing the coverage–cardinality trade-off before final selection.

Figure S2: Bars show target matches per added candidate, and horizontal markers show the state-dependent micro-F1/2 requirement (MIMIC-III

n=100

, MIMIC-IV

n=100

, EHRSHOT

n=120

). Green batches exceed the algebraic marginal-yield boundary and red batches fall below it. The figure decomposes finalized outputs; error bars are patient-cluster bootstrap intervals for MIMIC-IV and EHRSHOT.

Dataset	System	Clinical validity	Patient grounding	Next-visit relevance	ICD alignment	Evidential calibration	TP mean	CA-RQ5
MIMIC-III	ICD-Deepresearch (RRF-selected)	4.86	4.61	3.34	4.95	4.47	4.45	1.33
	GPT-5 (Direct Forecasting)	4.83	4.50	3.30	4.87	4.26	4.35	1.29
	GPT-4o (Direct Forecasting)	4.13	4.13	2.63	4.22	3.63	3.75	0.97
	GPT-4o-mini (Direct Forecasting)	3.61	3.73	2.38	3.65	3.17	3.31	0.77
MIMIC-IV	ICD-Deepresearch (RRF-selected)	4.97	4.87	3.53	4.99	4.69	4.61	1.57
	GPT-5 (Direct Forecasting)	4.87	4.75	3.52	4.93	4.47	4.51	1.36

Appendix

Table S5: Displayed values underlying main-paper Figure 3a–c: automated, gold-aware GPT-5 judge scores for pooled true-positive Reason5 means and CA-RQ5 at

K=20

. Means are rounded to two decimals; differences and intervals in the main figure are computed from the unrounded item-level values. The ICD-Deepresearch arm is RRF-selected, has the legacy label AF/Union+RRF in the saved implementation records, and is distinct from the full Evidence-Aware output.

Functional role	Executable call or component	Executable source and prompt lines	Role in the reported path
Candidate Generation (internal FCG)	Complete candidate-generation interface with a SparseEHR backbone for the ICD tasks	SparseEHR backbone described in ( Ghaffari et al. 2026 ) ; registered MIMIC-III/IV and EHRSHOT candidate exports	Structured pre-target EHR $\rightarrow$ ordered target-vocabulary candidates $\pi_{i}^{\mathrm{F}}$ and set $B_{i}$ used by the Candidate Generation Only control and Research Expansion initialization.
Research Expansion (internal DR-CG)	Iteration 1 candidate-formulation call (archived Stage 1)	archived Stage 1 candidate runner , lines 94–115	Saved EHR Prior candidates and the pre-target record $\rightarrow$ patient-anchored hypotheses and validation queries.
Research Expansion (internal DR-CG)	Iteration 1 add judge (archived Stage 1)	Same file, lines 134–138	Separately judges patient anchor, literature bridge, and exact mapping for Iteration 1 candidates.
Research Expansion (internal DR-CG)	Iteration 2 query planner (archived Stage 2)	archived Stage 2 expansion runner , lines 292–308	Produces short relation queries; rejected Iteration 1 candidates are force-carried outside this call.
Research Expansion (internal DR-CG)	Iteration 2 exact-code mapper (archived Stage 2)	Same file, lines 311–329	Maps query evidence and the pre-target record to new target-vocabulary-valid code hypotheses.
Research Expansion (internal DR-CG)	Iteration 2 three-part judge (archived Stage 2)	Same file, lines 496–503	Used once for initial verdicts and again for evidence repair; the call prefix and previous-decision payload change.

Appendix

Table S6: Executable-call inventory mapped to the forecasting functions. “Call” denotes an LLM invocation; deterministic and API rows record the complete execution boundary. Line ranges and archival source descriptions identify the executed call sites.

Iteration 1 (Archived Stage 1): Patient-Anchored Temporal Candidate Formulation
System message	Generate patient-anchored temporal diagnosis candidates and falsifiable clinical queries. JSON only.
Role and target control	You are expanding exact ICD-10-CM diagnosis code candidates from PRE-TARGET patient history. The target is every unique ICD-10-CM diagnosis code assigned during the patient’s final admission. Do NOT use ground truth or target-admission content.
Evidence-role separation	For every candidate, separate the evidence roles: (1) patient_anchor : the concrete prior diagnosis, repeated code, medication, procedure/device, lab, or symptom in this patient. (2) transition_hypothesis : the exact persistence, recurrence, treatment consequence, diagnostic-threshold, or durable-status relation to verify. (3) validation_query : 8–18 medical search terms containing the candidate entity, the patient-specific precursor/intervention, and the relation or endpoint.
Query exclusions	Do not use next visit, prediction, coding, longitudinal, readmission, or generic guideline language.
Temporal classes	Use temporal_class persistent_status\|chronic_active\|recurrent_condition\|planned_followup\|treatment_consequence\|acute_or_lab . Remote history and general risk associations may be candidates but must state that weakness.
Structured output	{"candidates":[{ "code":"ICD_...", "term":"...", "source_hint":"explicit_history_code \| clinical_text \| diagnosis_inference \| complication_inference", "temporal_class":"...", "patient_anchor":"short concrete clue", "ongoing_clue":true, "transition_hypothesis":"short falsifiable relation", "validation_query":"patient-anchored clinical query", "rationale":"short" }]}

Appendix

Table S7: Exact static MIMIC-IV Iteration 1 candidate-formulation prompt used inside Research Expansion (archived Stage 1), segmented for readability; only dynamic payloads and line wrapping are replaced. This call is distinct from the SparseEHR-based Candidate Generation module. The implementation then deterministically merges eligible exact codes observed in history as proposals before applying the candidate limit.

Stage A v2: Source-Aware Confidence over the Frozen Union Pool
Prompt segment	Exact content
System message	You are a calibrated clinical forecasting ranker. Rank only the supplied ICD-9-CM candidates for the immediately next admission. Sources are untrusted auxiliary evidence. Return JSON only.
MIMIC-IV system message	You are a calibrated clinical forecasting ranker. Rank only the supplied ICD-10-CM candidates for the immediately next admission. Sources are untrusted auxiliary evidence. Return JSON only.
Objective	Assign a calibrated 0–100 confidence to EVERY supplied ICD-9-CM candidate for the exact code being documented in the immediately next, unknown admission. This is temporal diagnosis forecasting, not extraction and not lifetime disease detection. The target admission is hidden.
MIMIC-IV objective	Assign a calibrated 0–100 confidence to EVERY supplied ICD-10-CM candidate for the exact code being documented in the immediately next, unknown admission. This is temporal diagnosis forecasting, not extraction and not lifetime disease detection. The target admission is hidden.
Ranking rules 1–4	1. Patient-specific pre-target evidence dominates population prevalence and generic literature. 2. Direct and research ranks are imperfect priors, not labels. Direct top-20 membership is nevertheless a meaningful independent forecast signal. Preserve a Direct candidate when it has a plausible patient anchor; demote it only for a concrete contradiction, exact-code mismatch, or clearly stronger competitors. 3. A baseline/patient-history candidate may carry a legacy “acute_speculation” default. Ignore that default label and infer persistence from the history, provenance, and code meaning. 4. A source with support_signal=false provides no promotion. A supported source establishes only a general relation, never that this patient will receive the code next.

Appendix

Table S8: Exact static Evidence-Aware Reranker prompts used by the full ICD-Deepresearch model (legacy artifact label FR; original implementation name: Stage A v2) and shared response contract used for the full-data MIMIC-III and MIMIC-IV evaluations. The two executable templates differ in the explicitly shown ICD-9-CM versus ICD-10-CM system/objective text and prompt-version identifier; the ranking rules, ceilings, candidate fields, and schema are otherwise identical. Runtime history and candidate JSON are represented by explicit dynamic fields. The verbatim

K=20

wording records the archived evaluation configuration.

Stage 1B: Patient Anchor–Literature Bridge–Exact Mapping Judge
System message	Apply a conservative patient-anchor, literature-bridge, and exact-mapping contract. JSON only.
Opening instruction	Final-admission ICD-10-CM diagnosis code add editor using a three-part evidence contract. Start from baseline top20 and only assess GPT_EXPANDED_CANDIDATES . Do not use ground truth or target-admission content.
Role 1: patient anchor	PATIENT ANCHOR comes only from this EHR. It must establish an active/repeated condition, current treatment, durable procedure/device/status, explicit plan, or a concrete acute/lab precursor.
Role 2: literature bridge	LITERATURE BRIDGE comes from candidate-bound E evidence. It verifies the exact persistence, recurrence, treatment consequence, diagnostic threshold, or durable-status relation. General association and code-title pages are insufficient.
Role 3: exact mapping	EXACT MAPPING requires the anchor to match the candidate entity, subtype, site, cause, acuity, and status.
Clinical policy	Durable status may remain true after a documented procedure/device unless removal/reversal is present. Chronic disease needs repeated code or an active treatment/problem clue. Acute events, lab abnormalities, infections, injuries, and precise complication subtypes require direct patient evidence; risk alone never suffices. Literature cannot create a patient diagnosis.

Appendix

Table S9: Exact static MIMIC-IV Iteration 1 add-judge prompt used inside Research Expansion. This is a separate LLM call from the Iteration 1 candidate-formulation call.

Stage 2A: Clinical Relation Query Planner
System message	Generate compact clinical-code discovery queries. JSON only.
Objective	Plan second-pass medical discovery queries for ICD-10-CM diagnosis code. No ground truth, target/post codes, or hidden pretrain tail. Stage 1 has already run with the legacy query method. Stage 2 must use a different retrieval strategy.
Search design	Formulate phrases that resemble real clinical review questions about natural history, disease progression/outcomes, post-procedure persistence or recurrence, reoperation/retreatment rates, or complication incidence/timing/manifestations. Each query tests one patient-anchored clinical hypothesis. Use 4–9 medical terms with one candidate entity, at most one concrete precursor/intervention, and one endpoint. Generate 2–3 separate phrases for distinct search angles; never concatenate multiple conditions, procedures, endpoints, or raw note tokens.
Force-carry and exclusions	Never use next visit, prediction, coding, code definition, guideline, longitudinal, or readmission as filler. Every rejected stage-1 candidate listed below will be force-carried and searched separately; use this plan to discover additional candidates and clinical relations. Allow no query for unsupported classes.
Prompt examples	post-stroke depression long-term persistence autologous stem cell transplant engraftment complications prosthetic heart valve long-term follow-up cancer anemia chronic disease iron studies cervical disc herniation natural history native coronary atherosclerosis after CABG TURP long-term LUTS recurrence These work because they search relations that clinical literature actually studies: natural history, progression, persistence, recurrence or reoperation, and complication incidence or timing. Each phrase contains one clinical entity, at most one relevant intervention, and one endpoint. Split distinct intents into separate queries.
Structured output	{"queries":[{ "query":"4--9 term phrase", "query_variants":["alternative 4--9 term phrase"], "focus":"candidate entity", "patient_anchor":"concrete EHR clue", "clinical_relation":"short falsifiable relation", "evidence_needed":"specific endpoint, threshold, persistence, or consequence", "source_type":"procedure_status \| chronic_persistence \| recurrence \| treatment_consequence \| diagnostic_threshold \| planned_care" }]}

Appendix

Table S10: Exact static MIMIC-IV Iteration 2 query-planning prompt (archived Stage 2), segmented for readability. The short-query examples are prompt patterns, not case-specific rules.

Stage 2B: Evidence-Conditioned Exact-Code Mapper
System message	Map compact clinical entities to exact ICD candidates. JSON only.
Task	Second-pass ICD-10-CM diagnosis code candidate mapper. Do NOT use ground truth, target/post codes, or hidden pretrain top21–100. Use patient history plus second-pass search snippets to propose exact normalized code candidates NOT already in CURRENT_CODES . Valid formats include ICD_... . You may reuse first-pass generated candidates that were not accepted.
Per-candidate fields	For every candidate provide: (1) a concrete patient_anchor from this EHR, (2) temporal_class , (3) a short falsifiable transition_hypothesis , and (4) 2–3 validation_queries . Every query must be a 4–9 term phrase containing one candidate entity, at most one precursor/intervention, and one endpoint. Split distinct intents; do not concatenate note tokens. Never use next visit, prediction, coding, guideline, longitudinal, or readmission as filler. Generate direct entity mappings before supported complications/status expansions. Avoid pure risk-factor speculation and broad catch-all codes.
Shared example block	The prompt inserts the exact SHORT_QUERY_EXAMPLES block reproduced in Table S10 , from “Short-query examples (patterns, not patient-specific rules):” through “Split distinct intents into separate queries.”
Mapping types	direct_entity \| specific_variant \| complication \| status \| history \| procedure_consequence \| medication_effect \| recurring_lab_procedure \| risk_association .
Structured output	{"candidates":[{ "code":"ICD_...", "term":"...", "mapping_type":"...", "temporal_class":"...", "entity":"short clinical entity", "patient_anchor":"concrete EHR clue", "ongoing_clue":true, "transition_hypothesis":"short medical relation", "validation_queries":["4--9 term phrase", "alternative phrase"], "source_hint":"query_term \| query_evidence \| patient_history \| procedure_context \| first_pass_candidate", "rationale":"short" }]}

Appendix

Table S11: Exact static MIMIC-IV Iteration 2 candidate-mapping prompt (archived Stage 2). This agent sees the query plan and retrieved text, but every proposed code must still bind to the patient’s pre-target record.

Stage 2C: Default Three-Part Add Judge
System message	Judge patient anchor, literature bridge, and exact clinical-code mapping separately. JSON only.
Opening instruction	“Second-stage” (initial call) or “Evidence-repair” (repair call) clinical-code add editor using a three-part evidence contract. Assess only the supplied candidates. Do not use ground truth, target/post codes, or hidden pretrain tail.
Patient anchor	PATIENT ANCHOR comes only from this EHR: repeated code, active diagnosis/treatment, durable procedure/device/status, explicit plan, or direct acute/lab event.
Literature bridge	LITERATURE BRIDGE comes from candidate-bound E or QEV sources and verifies the claimed medical persistence, recurrence, treatment consequence, diagnostic threshold, or durable-status relation. It does not need to prove that this individual patient will be coded next.
Exact mapping	EXACT MAPPING requires the patient anchor to match the candidate entity, subtype, site, cause, acuity, and status.
Clinical policy	Durable procedure/device status can persist unless removal is documented. Chronic disease needs repetition or an active treatment/problem clue. Acute events, infections, injuries, laboratory abnormalities, and precise complications require a direct patient event, threshold, or explicit plan; general risk is insufficient. Literature cannot create a patient diagnosis.

Appendix

Table S12: Exact static default MIMIC-IV Iteration 2 judge prompt (archived Stage 2). The same function is called for the initial pass and the targeted repair pass; only the opening prefix and previous-decision payload differ.

Stage 2D: Untruncated Verdict-Completion Judge
System message	Judge every supplied clinical-code candidate. Return one verdict per code. JSON only.
Core instruction	You are the legacy second-stage clinical-code add judge. Assess every supplied candidate independently. Do not omit a candidate and do not rank candidates against one another. Patient history is primary evidence. Literature may verify code meaning, persistence, recurrence, treatment consequence, or a bridge from the patient context to the candidate, but literature cannot create a patient diagnosis.
Acceptance rule	Add chronic or persistent conditions only with an explicit prior diagnosis, repeated code, active treatment, durable status/procedure, or other concrete ongoing clue. Add acute events, laboratory abnormalities, injuries, infections, external causes, or precise complications only with a direct current event, lab/treatment context, or strong recurrent pattern. A prior code is a clue but may support a persistent chronic condition when the candidate and patient history match exactly. Reject a related but different subtype. DNR alone does not establish a palliative-care encounter.
Structured output	{"add":[{ "code":"ICD_...", "direct_support":true, "support_type":"persistent_current\|procedure_supported\| active_context\|prior_only\|speculative", "evidence_ids":["E1"], "query_evidence_ids":["QEV_E1"], "rationale":"brief" }]}
Interface limitation	Unlike Table S12 , this exact archived response does not contain accept , confidence , literature_bridge_supported , or exact_mapping_supported . No unreported field mapping is assumed; a uniform-gate release must rejudge these fixed proposals with the default three-part contract.
Dynamic payload	Current codes; up to 8,000 characters of patient history; and every candidate paired with its dictionary title, candidate-bound evidence, and semantically matched discovery evidence. Missing verdicts are retried per ICD until every candidate has one output record.

Appendix

Table S13: Exact static MIMIC-IV verdict-completion prompt used when a long Stage 2 response (Iteration 2 in the method) omitted candidate verdicts. It is a repair/completion call, not the default Stage 2 policy.

MIMIC-III ICD-9 Verdict Completion and Fixed-Candidate Fallback
Batch system message	Judge patient anchor, literature bridge, and exact ICD-9 mapping separately. JSON only.
Batch opening	You are a strict future-admission ICD-9 prediction judge using a three-part evidence contract. Everything in PATIENT HISTORY happened before the unknown target admission. Judge each role independently.
Three exact roles	1. PATIENT ANCHOR comes only from this EHR. It must be a repeated code, active diagnosis/treatment, durable procedure/device/status, explicit plan, or direct acute/lab precursor. 2. LITERATURE BRIDGE comes from candidate-bound Reader evidence. It verifies the claimed general medical persistence, recurrence, treatment consequence, diagnostic threshold, or durable-status relation. It does not need to prove that this individual patient will be coded next. 3. EXACT MAPPING requires the patient anchor to match the candidate entity, subtype, site, cause, acuity, and status.
Batch policy	Durable procedure/device status can persist unless removal is documented. Chronic disease can be supported by either repetition or a concrete active treatment/problem clue. Acute symptoms, infection, injury, lab abnormalities, and precise complications require direct patient evidence; general risk is insufficient. Literature cannot create a patient diagnosis. Return one record for every assessed candidate. JSON only.
Batch contract	add[code, accept, confidence, support_type, patient_anchor_present, patient_anchor_type, literature_bridge_supported, exact_mapping_supported, evidence_ids, rationale] ; dynamic sections are CURRENT CODES , CANDIDATES , the last 8,500 history characters, and CANDIDATE-BOUND READER EVIDENCE .
Fallback system message	Return one explicit accept or reject verdict for the fixed ICD-9 candidate. JSON only.

Appendix

Table S14: Exact static MIMIC-III verdict-completion prompts. The batch call retries omitted codes; any code still missing after those retries receives the fixed-candidate fallback call.

Research Confidence Ranking and Candidate-Union Fusion
System message	You are a calibrated next-visit ICD-10-CM diagnosis ranking model. Score candidates independently, then calibrate the ranking jointly. JSON only.
MIMIC-IV instruction	Rank every supplied ICD-10-CM candidate for the patient’s unknown final visit. The history below is the only patient-specific information available before the target visit. Predict which exact diagnoses are likely to be coded at that next visit. A past code can recur, persist, resolve, or be irrelevant. General retrieved evidence may support a medical transition but does not prove that it will occur in this patient. Use one calibrated 0–100 probability scale across all candidates. Reward active chronic disease, durable status, repeated diagnoses, plausible complications, and coherent care transitions. Penalize resolved events, unsupported specificity, distant associations, and diagnoses supported only by a generic code-description page.
MIMIC-IV output	{"eov_threshold":50, "scores":[{"code":"ICD_...", "confidence":0}]} Return one patient-specific eov_threshold (0–100). The complete variable-length prediction contains candidates with confidence $\geq$ eov_threshold . Return JSON only, include every supplied code exactly once, and introduce no new code.
MIMIC-III system	You are a calibrated next-admission ICD-9 ranking model. Evaluate candidates independently, then calibrate them jointly. JSON only.
MIMIC-III instruction	Score every candidate ICD-9-CM diagnosis for the unknown final admission. The PATIENT HISTORY contains only information available before the target admission. Do not infer that a historical diagnosis must recur. Estimate the probability that the exact candidate code, at its stated specificity, will be assigned during the next admission. Distinguish persistent active disease, durable status, recurrent disease, likely complications, isolated past events, and speculative associations. Retrieved sources support general medical transitions but do not prove this patient’s future diagnosis. Use one consistent 0–100 scale across all candidates: 90–100 near-certain; 70–89 likely; 45–69 plausible but uncertain; 20–44 weak; 0–19 unlikely.
MIMIC-III output	Also return one patient-specific eov_threshold from 0–100. A candidate with confidence $\geq$ eov_threshold is in the variable-length diagnosis prediction set before EOV. The threshold should balance missing likely diagnoses against unsupported additions. Return JSON only. Include every supplied code exactly once and do not introduce new codes: {"eov_threshold":0, "scores":[{"code":"ICD_...", "confidence":0, "include_before_eov":false}]}

Appendix

Table S15: Exact static MIMIC-IV and MIMIC-III research-confidence prompts plus the deterministic Union+RRF selector replacement reported as ICD-Deepresearch w/o Evidence-Aware Reranker. Each confidence agent scores every dictionary-valid candidate; RRF combines the resulting research rank with the independent Direct GPT-5 rank.

Independent Direct Forecast Prompt
System message	Independently forecast ICD-10-CM diagnoses from the supplied pre-target patient history. Return JSON only.
Opening and restriction	Forecast the ICD-10-CM diagnosis codes most likely to be assigned during this patient’s unknown next/final admission. Use ONLY the observed pre-target patient history below and your own medical/coding knowledge. No future encounter information is available. You have no candidate list, baseline predictions, retrieved sources, external tools, or outputs from another system.
Prediction requirements	Independently generate 20 to 30 unique ICD-10-CM diagnosis codes. Do not output ICD-9-CM, CPT, procedure codes, or prose in the code field. Rank them from most to least likely for the next/final admission. Give each code a confidence from 0 to 100. Confidence must be non-increasing down the list and should reflect uncertainty rather than forcing every item to look likely. Forecast future coding rather than copying every historical code. Balance persistent diseases/statuses, recurrent conditions, likely acute complications, and procedure/treatment consequences. Preserve exact ICD-10-CM specificity. Do not invent patient facts or assume access to the held-out admission.
Structured output	{"ranked":[{ "code":"ICD-10-CM code", "confidence":0 }]}
Dynamic payload	Observed pre-target patient history: <row.pre_text> . On validation failure the exact suffix is: “Prior structural errors: <feedback> . Regenerate the complete ranking without any external candidate list.”
MIMIC-III resolution	The same executable template resolves codeSystem to “ICD-9-CM” and otherCodeSystem to “ICD-10-CM”; all other static sentences and the structural retry suffix are unchanged.

Appendix

Table S16: Exact static Direct GPT forecasting prompt used for Direct Forecast Only and as the independent direct branch of the RRF selector replacement. The model name changes by arm; no retrieval tool, candidate pool, baseline prediction, dictionary content, or other system output is supplied.

GPT-5 + Web Search (Standalone)
System message	Independently produce an exhaustive, unbounded ICD-10-CM forecast from pre-target history. Use Web Search for general medical and coding evidence, maximize recall, and return JSON only.
Opening and restriction	Forecast every distinct ICD-10-CM diagnosis code that could plausibly be assigned during the unknown target encounter represented by the final empty icd_list_begin marker. Use ONLY the observed pre-target patient history below, your medical and coding knowledge, and the provided Web Search tool. No future encounter information is available. You have no baseline predictions, candidate list, dictionary entries, or outputs from another system.
Search policy	You MUST use Web Search before finalizing the ranking: Decide the number of searches yourself from the complexity and uncertainty of this patient’s trajectory; there is no minimum or maximum search-count target beyond using the tool at least once. Search each clinically distinct unresolved trajectory when doing so could change candidate inclusion, code specificity, confidence, or rank. Continue searching while another focused query is likely to add decision-relevant evidence; stop when additional searches would only repeat information already found. Search for diagnosis progression, common next-encounter diagnoses, treatment/complication relations, and exact ICD-10-CM terminology when useful. Do not include patient identifiers or long excerpts from the patient history in a query. Use retrieved pages as general medical or coding evidence, not as hidden patient facts.
Unbounded objective	The objective is exhaustive recall. There is deliberately NO prediction-count budget: Do not target or stop at 20, 30, or any other preset number. Generate as many distinct, medically defensible codes as the trajectory can support, including lower-confidence possibilities. Continue until further codes would be unrelated to this patient or would merely duplicate a more appropriate code already listed.
Ranking/output	1. Rank all generated codes from most to least likely for the unknown target encounter. 2. Assign confidence from 0 to 100 in non-increasing order. Low-confidence candidates are allowed and should remain low. 3. Cover the full trajectory, including persistent and recurrent diagnoses, durable status/history codes, treatment consequences, plausible acute developments, comorbidities, and follow-up-related diagnoses. 4. Preserve exact ICD-10-CM specificity when justified. Do not invent hidden patient facts. 5. Do not output ICD-9-CM, CPT, ICD-10-PCS procedure codes, or prose in the code field. 6. Return JSON only: {"ranked":[{"code":"ICD-10-CM code","confidence":0}]} .
Tool and retry	OpenAI Responses API tool: web_search_preview , tool_choice="required" , search_context_size="low" . The exact retry suffix is: “Prior structural errors: <feedback> . Regenerate the complete exhaustive list without imposing a count limit.” The MIMIC-III resolution substitutes the ICD-9-CM target sentence and its exact opposite-vocabulary exclusion.

Appendix

Table S17: Exact static GPT-5 + Web Search (Standalone) baseline prompt. Unlike ICD-Deepresearch, this agent chooses its own search count and directly produces an unbounded code ranking from OpenAI Web Search results.

Direct Locked-Prediction Explanation Writer
System message	Explain locked next-admission ICD-10-CM predictions using only the supplied patient history. Return JSON only.
User instruction	Explain the following locked ICD-10-CM predictions for the patient’s unknown next/final admission. Use ONLY the observed pre-target patient history and your own medical/coding knowledge. The prediction set is already locked: do not add, remove, replace, or reorder codes. No gold answer, candidate pool, baseline prediction, retrieval result, external source, or other system output is available.
Medical output	For every code, write one concise English medical rationale linking observed history to possible next-admission coding, plus one concise uncertainty statement. Do not claim that the future diagnosis is known. Do not mention models, prompts, experiments, candidates, or evaluation.
Exact JSON	{"explanations":[{ "code":"locked code", "rationale":"medical rationale", "uncertainty":"specific limitation" }]}
Dynamic payload/retry	Ten locked predictions per parallel explanation chunk and the full observed pre-target history. Validation requires one explanation per locked code. The exact retry suffix is: “Prior structural errors: <feedback> . Return corrected complete JSON.”
MIMIC-III resolution	The same executable template replaces “ICD-10-CM” with “ICD-9-CM” in the system, opening, and output-schema context; the remaining static prose is unchanged.

Appendix

Table S18: Exact static Direct-model explanation prompt, distinct from the evidence-linked explanation writer.

Locked-Prediction Explanation Agent
System message	Write evidence-linked medical explanations for locked next-admission ICD-10-CM forecasts. Return JSON only.
Prediction lock	Write concise English medical explanations for the 20 locked ICD-10-CM forecasts for this patient’s unknown next/final admission. The prediction set and ranking are fixed. Do not add, remove, replace, or reorder codes. Use only the observed pre-target history, the candidate-specific clues, and the supplied evidence ledger. The held-out admission and gold codes are not available.
Medical requirements	1. Explain medicine only. Never mention a model, system, baseline, candidate-generation stage, rank, prompt, experiment, or evaluation. 2. For every exact code, state a patient-specific medical bridge from observed history to possible next-admission documentation. Similar diagnoses may share concise reasoning, but each code must receive its own explanation. 3. Preserve exact disease, organ, acuity, status, device, complication, and laterality. Do not substitute a nearby diagnosis. 4. Chronic/status/history codes may persist when their anchor remains relevant. Acute diagnoses, symptoms, injuries, and abnormalities require recurrence or renewed documentation; do not portray them as certain. 7. Give one medically specific uncertainty statement. Do not invent future symptoms, tests, procedures, treatments, or diagnoses. 8. Return every locked code exactly once as JSON.
Citation contract	5. Use a definition reference only to establish code meaning. Use a relation source only for a claim actually supported by its excerpt. General literature cannot prove this patient’s future event. 6. Cite only reference IDs assigned to that same code. Put bracketed citations such as [1][2] next to the supported claim and return those same integers in citation_ids . Every explanation must cite at least its definition reference.
Structured output	{"explanations":[{ "code":"locked code", "rationale":"medical rationale with [id]", "uncertainty":"specific limitation", "citation_ids":[1] }]}
Dynamic payload	Observed history; locked predictions with code-specific clues and allowed reference IDs; and numbered evidence ledger. Structural validation requires all 20 codes exactly once and rejects cross-code or missing citations. The exact retry suffix is: “Prior structural errors: <feedback> . Return a corrected complete response.”

Appendix

Table S19: Exact static evidence-linked explanation prompt for ICD-Deepresearch (RRF-selected; legacy artifact label AF), segmented for readability. Predictions and order are finalized before this call, so the explanation agent cannot improve prediction metrics; this prompt was not evaluated on the full ICD-Deepresearch outputs produced by the Evidence-Aware Reranker (legacy label FR). Verbatim “locked” language below reflects the archived

K=20

implementation.

Evaluation-Only Shared-Pool Selector
System message	You rank a fixed candidate pool for prospective ICD-10-CM forecasting. Return JSON only.
Opening	Predict exactly 20 unique ICD-10-CM diagnoses for the patient’s unknown next encounter. This is prospective next-encounter forecasting, not coding a future note. The held-out target is unavailable. Select only from the supplied valid candidate pool. Candidate order is deterministically randomized and carries no rank signal.
Requirements	Return exactly 20 different candidate_id values, ordered from highest to lowest confidence. Copy IDs exactly; do not add, duplicate, substitute, or omit a slot. Rank persistent active disease, recurrent conditions, near-term treatment consequences, and strongly anchored follow-up needs above generic medical possibilities. Preserve exact ICD entity, acuity, anatomy, status, device, laterality, and complication. Do not invent patient facts or assume access to the held-out encounter.
Controlled contrast	Ledger arm: “Use the supplied target-blind research ledger critically. A definition or general relation does not by itself prove next-encounter occurrence.” No-ledger arms: “Use only the observed trajectory and candidate code identities; no research ledger is available.”
MIMIC-IV output/payload	Return {"ranked":[{"candidate_id":"C01","confidence":0}]} . The dynamic payload contains the pre-target trajectory and the same randomized pool; only the ledger arm receives candidate-specific research fields. The exact retry suffix is: “Prior response errors: <feedback> . Return corrected JSON.”
MIMIC-III system	You produce an exact-cardinality next-admission ICD-9 ranking. Return JSON only.

Appendix

Table S20: Exact static candidate-controlled shared-pool selector prompts. MIMIC-IV and MIMIC-III use separate executable templates, reproduced here rather than described as a simple vocabulary substitution.

MIMIC-III Shared-Pool Locked-Prediction Explainer
System message	You explain locked next-admission ICD-9 predictions with concise medical reasoning. Return JSON only.
Opening	Write an English medical next-encounter explanation for each of the <chunk size> locked ICD-9-CM predictions below. The predictions and gold answer are unavailable for revision: do not add, remove, substitute, or reorder codes. Explain the forecast from only the observed pre-target record and supplied method-specific information.
Requirements 1–5	1. Return all <chunk size> locked codes exactly once. 2. For each, write one concise patient-specific medical rationale and one concise uncertainty statement. 3. Preserve exact ICD identity and avoid invented patient or future-visit facts. 4. For weak predictions, say specifically why evidence is insufficient instead of making the code sound certain. 5. Do not mention models, systems, prompts, candidates, stages, ranks, experiments, or evaluation.
Method-specific rule	Ledger arm: “6. citation_ids may contain only supplied reference IDs assigned to that exact code. Include at least its definition reference. General literature does not establish patient-specific future occurrence.” No-ledger arms: “6. citation_ids must be empty. Do not invent citations, references, or URLs.”
Output/payload	7. Return JSON only: {"explanations":[{"code":"ICD_...","rationale":"one concise medical sentence","uncertainty":"one concise caveat","citation_ids":[]}]} . Payload headings are OBSERVED PRE-TARGET PATIENT CONTEXT , LOCKED TOP-20 PREDICTIONS , METHOD-SPECIFIC INFORMATION FOR THOSE CODES , and, for the ledger arm, SUPPLIED REFERENCES .
Chunk retry	Two ten-code chunks run in parallel. The exact retry suffix is: “Prior chunk <chunk index> errors: <feedback> . Return corrected complete JSON.”

Appendix

Table S21: Exact static MIMIC-III shared-pool explanation prompt. These post-selection reasons were generated by the

K=20

shared-pool runner but do not enter the candidate-controlled prediction metrics. Verbatim prompt text is preserved below.

Label-Blind Retrieval-Item Utility Judge
System message	You are a blinded clinical retrieval evaluator. Return JSON only.
Task/blinding	Evaluate one retrieved webpage for a next-visit diagnosis prediction system. TASK: Given only the patient’s pre-target history, predict diagnosis codes documented at the immediately next hospital visit. BLINDING: You are not told which retrieval system selected this page. Do not infer quality from the query style. Gold diagnoses and model predictions are intentionally absent.
Exact definitions	credible : URL/domain and title indicate a medically or officially credible source. patient_relevant : the page addresses a diagnosis, mechanism, complication, or follow-up issue grounded in this patient’s history. forecast_useful : it supplies a concrete clinical bridge that could help rank a diagnosis for the immediate next visit. A generic ICD definition alone is NOT forecast-useful. coding_only : useful only for code identity/specificity, without improving which diagnosis is likely next. direct : concrete patient-grounded next-visit bridge; indirect : relevant background or coding help but weak temporal discrimination; none : irrelevant/noisy/unhelpful.
Dynamic payload	Patient history (first 5,000 characters), retrieval query (first 600), page title (first 300), and page URL. The exact final instruction is: “Return a strict, conservative judgment. The reason must be one short sentence.”
JSON fields	credible , patient_relevant , forecast_useful , and coding_only booleans; utility_level in direct\|indirect\|none ; and one-sentence reason .

Appendix

Table S22: Exact static source-metadata proxy judge prompt. The archived response schema uses utility-related field names, but this audit sees only source metadata and patient history, not page bodies, system identity, predictions, or gold labels; its outputs are not source-utility measurements.

Gold-Aware Item-Blind Five-Dimension Explanation Judge
System message	You are a strict blinded clinical forecast-reason adjudicator. Return JSON only.
Opening	You are a blinded clinical adjudicator evaluating individually anonymized reasons for exact Top-20 next-admission ICD-10-CM forecasts. The complete held-out gold code set is supplied only for post-generation evaluation. All generators were target-blind.
Symmetric set rule	Prediction-set evaluation is fixed and symmetric: every false prediction and every missed gold code receives zero reason utility. Do not let fluent prose, confidence, citations, or medical plausibility rescue either error. Score only reasons attached to true-positive predicted codes because only a correct set prediction can earn reason utility.
Item blinding	Each anonymous_reason_item is independently shuffled and may come from any system. The opaque item_id contains no model identity. You are not shown the originating system’s prediction set, TP count, FP count, FN count, rank, or confidence. Do not group items by writing style or infer generator identity. Score each reason on its own content.
Five exact dimensions	Clinical validity: Are the medical relationships, mechanisms, and claims clinically sound and free of substantive contradiction or hallucination? Gold occurrence proves that the code appeared, not that the stated medical mechanism is correct. Patient grounding: Are all patient-specific claims explicitly supported by the observed pre-target trajectory, without importing held-out encounter facts or inventing symptoms, tests, diagnoses, treatments, or plans? Next-encounter relevance: Does the reason explain why this code is likely to be documented in the immediately held-out admission, instead of merely defining the code, restating history, or asserting generic chronic carry-forward? ICD alignment: Does the reason match the exact ICD entity and its level of specificity, rather than a neighboring diagnosis, symptom, complication, procedure, or broader code family? Evidential calibration: Does the reason clearly separate observed patient facts, general medical knowledge or cited relations, and the forecast hypothesis; avoid treating association as patient-specific certainty; and express uncertainty proportionate to the available support? If citations are supplied, judge whether they entail the claim attributed to them. Citation absence alone must not lower this score when the trajectory itself provides adequate support.
Scale/output	Shared rubric: 5 complete, precise, and without a substantive flaw; 4 strong with one minor limitation; 3 plausible but partial or generic; 2 major gaps, weak support, or notable mismatch; 1 materially misleading; 0 absent, unusable, contradicted, or incompatible with the trajectory or exact ICD entity. Score dimensions independently, use the full scale, and do not infer system or model identity. Assess each listed true-positive reason exactly once and return JSON only. The strict response schema requires all five integers and one concise_reason per opaque item ID.

Appendix

Table S23: Exact static prompt for the automated, gold-aware GPT-5 Reason5 semantic audit, segmented for readability. All prediction and explanation outputs are frozen before the complete gold answer is supplied to this evaluation-only judge. Items hide method identity, but the audit is not independent clinical adjudication.

Table S24. Selected qualitative forecasting and explanation traces. Panel A compares ICD-Deepresearch: Direct Forecast Only with ICD-Deepresearch (RRF-selected) on a shared true-positive code; the latter has the legacy label AF/Union+RRF in the saved records. Panels B–C follow productive and failed Research Expansion; Panel D shows how Direct Forecast Only, Research Path Only, and ICD-Deepresearch w/o Evidence-Aware Reranker contribute different true positives. Metrics and gold membership are revealed only in the evaluation rows.
Trace	Function / view	Selected output and interpretation
Panel A. Explanation case, MIMIC-IV row 45 — correcting a fluent ICD-entity error
A0	Observed record	A 32-year-old woman has repeated pregnancy-related records containing uterine leiomyoma (D25.9), maternal care for benign uterine tumor in the second trimester (O34.12), and later third-trimester obstetric care. O34.13 is a held-out true positive; neither explanation generator saw the gold set.
A1	GPT-5 (Direct Forecasting)	“Maternal care for uterine scar from prior surgery was previously noted (O34.12), so a future third-trimester pregnancy could warrant this code if the scar is not specifically attributed to cesarean or is coded under the broader uterine-scar category.” The explanation treats O34.12 as a uterine-scar code rather than maternal care for a benign uterine tumor.
A2	ICD-Deepresearch (RRF-selected)	“She has leiomyoma and previously required maternal care for a benign uterine tumor; if pregnant again into the third trimester with fibroids affecting management, O34.13 would apply [15].” Source [15] resolves to the exact local ICD-10-CM title. The uncertainty statement correctly conditions the forecast on a future third-trimester pregnancy and active management.
A3	Automated gold-aware Reason5 judge	ICD-Deepresearch (RRF-selected) receives clinical validity/patient grounding/next-encounter relevance/ICD alignment/evidential-calibration scores of 5/5/4/5/5; GPT-5 (Direct Forecasting) receives 0/1/1/0/1. After outputs are frozen, the method-blind GPT-5 judge receives the gold set, identifies the scar–tumor conflation, and assigns higher scores to the richer-input ICD-Deepresearch explanation.

Appendix

Table 33

Table S25. Complete explanation outputs for MIMIC-IV row 45. The exact same pre-target record is used by GPT-5 (Direct Forecasting) and ICD-Deepresearch (RRF-selected), while the ICD-Deepresearch writer also receives canonical code information, candidate clues, and ledger fields; the held-out target contains 15 ICD-10-CM codes. Every one of the 20 code-level rationales and uncertainty statements is shown. The ICD-Deepresearch source IDs are expanded in place so that each citation is inspectable.
ID	Predicted code / outcome	Complete explanation output
Panel A. Frozen input and post-hoc target
A0	Pre-target history	Age 32, female. Visit 1: O34.32 , D25.9 , D57.3 , Z3A.26 , Z37.0 , O32.8XX0 , O26.852 , O42.912 , and O34.12 ; procedure 10D00Z1 . Visit 2: O86.12 , O86.89 , R78.81 , and B95.61 . Visit 3: O34.33 , O26.893 , Z3A.36 , O99.013 , and D57.3 .
A1	Held-out gold	O34.211 , Z37.0 , N85.8 , O69.81X0 , O32.6XX0 , O70.1 , O34.13 , D25.1 , D25.2 , O99.03 , D57.3 , O76 , O99.89 , R55 , Z3A.38 . The gold set is disclosed only for this retrospective case audit.
Panel B. GPT-5 (Direct Forecasting): all 20 explanations
D01	O34.33 Maternal care for cervical incompetence, third trimester Post-hoc: FP	Rationale. Prior records show maternal care for scar from previous cesarean (O34.32, O34.33) and a documented cesarean delivery, so a future third-trimester pregnancy could require care for a prior C-section scar. Uncertainty. Future pregnancy and trimester at the next admission are unknown, so this condition may not be applicable. Evidence returned. None (the frozen output contains an empty citation-ID list).

Appendix

Table 34

Comparison axis	GPT-5 (Direct Forecasting)	ICD-Deepresearch (RRF-selected)	What the automated audit shows
Prediction outcome	3/20 hits: O34.13 , D57.3 , Z37.0 . P/R/F1=15.00/20.00/17.14%.	3/20 hits: D57.3 , Z37.0 , O34.13 . P/R/F1=15.00/20.00/17.14%.	Prediction quality is tied in this case; explanation quality must therefore be evaluated separately.
Clinical validity	3.33/5	5.00/5	The automated-score difference coincides with the Direct O34.13 scar–tumor conflation; both configurations score 5/5 on the other two shared hits.
Patient grounding	3.67/5	5.00/5	The richer-input ICD-Deepresearch explanation links O34.13 to observed leiomyoma/ O34.12 ; GPT-5 (Direct Forecasting) links it to an incorrectly inferred uterine-scar entity.
Next-encounter relevance	3.33/5	4.00/5	Both outputs are conditional forecasts; the score difference reflects the specificity of the observed transition rationale.
ICD alignment	3.33/5	5.00/5	GPT-5 (Direct Forecasting) misdescribes O34.33 , O34.13 , B95.61 , and O90.81 ; the 20 ICD-Deepresearch rows preserve title–rationale consistency in this selected case.
Evidential calibration	3.67/5	5.00/5	Code-bound source IDs make the asserted ICD entity and definition traceable.

Appendix

Table S26: Gold-aware automated comparison of the complete row-45 explanation outputs. Reason5 values are means over the three shared true-positive codes ( O34.13 , D57.3 , and Z37.0 ) under the frozen item-blind GPT-5 judge. The semantic audit counts explicit title–rationale contradictions visible in Table S25 ; ICD-Deepresearch (RRF-selected) receives the complete code-bound explanation input.

Table S27. Selected paired retrieval traces from the metadata audit. Within each panel, Research Expansion and GPT-5 + Web Search (Standalone) operate on the same patient row. The rows reproduce recorded queries, source metadata, and metadata-proxy labels.
Case	Retriever / judge	Selected retrieval trace
Panel A. MIMIC-III row 48 — newborn jaundice trajectory
A1	Archived research-path metadata	Query: bilirubin total peak newborn phototherapy . Source: Guidelines for Phototherapy , Stanford Newborn Nursery, https://med.stanford.edu/newborns/professional-education/jaundice-and-phototherapy/guidelines-for-phototherapy.html .
A2	Archived proxy label	Credible=yes; patient-relevant=yes; forecast-useful=yes; coding-only=no; utility=direct. The metadata judge labels the phototherapy-threshold source as a possible near-term bridge from the query, title, URL, and pre-target record.
A3	GPT-5 + Web Search (Standalone) metadata	Query: ICD-9-CM 771.81 septicemia of newborn code definition \| 771.82 urinary tract infection \| V29.0 observation . Source: Reddit Emergency Medicine thread, https://www.reddit.com/r/emergencymedicine/comments/1jx5rdi .
A4	Archived proxy label	Credible=no; patient-relevant=no; forecast-useful=no; coding-only=no; utility=none. The metadata judge labels this forum source as lacking an authoritative next-visit bridge.

Appendix

Table 36

Mechanism	Observed trace	Current validation gap	Required correction
Free-text anchor hallucination	Row 59 invents LDL, HbA1c, medications, BP values, and symptom visits absent from its structured input.	The judge receives the candidate’s anchor statement without an immutable source-span link.	Candidate must return an event ID or exact quoted pre-target span; deterministic validation rejects unresolved anchors before search.
Query–candidate drift	Row 59’s initial plan concerns hepatitis, COPD, migraine, autonomic dysfunction, substance relapse, and constipation; accepted additions later concern lipids, GERD, back pain, diabetes, and hypertension.	Mapping is allowed to “discover a supported consequence,” weakening traceability to the planned hypothesis.	Assign a hypothesis ID and require every candidate, query, source, and verdict to preserve that ID; new hypotheses must trigger a new planning pass.
Definition–forecast conflation	Row 52 true additions and several row 59 false additions cite dictionary pages as their only accepted evidence.	Exact code validation is represented beside clinical evidence and may be read as additive support.	Store definition , patient_anchor , and relation_evidence in separate channels; definitions contribute zero forecast confidence.
Semantic mapping mismatch	R73.9 is described as abnormal liver enzymes, G62.0 as alcoholic neuropathy, and D73.89 as anemia despite conflicting dictionary titles.	Validation checks that a code exists, not that model term, hypothesis, anchor, and dictionary entity agree.	Add bidirectional semantic entailment checks over code title $\leftrightarrow$ candidate term $\leftrightarrow$ patient anchor; reject mismatched site, cause, status, or entity.
Horizon-polarity error	Row 59 accepts acute pharyngitis while its own hypothesis and sources describe a self-limited episode.	The current relation label conflates persistence or recurrence with resolution.	Verifier must label the relation as persistence, recurrence, emergence, resolution, or contraindication and require positive immediate-horizon polarity.
Uncontrolled cardinality	Row 52’s 4/8 Research Round 2 yield improves F1; row 59’s 0/9 Research Round 2 yield necessarily lowers F1.	Candidate acceptance is local; the observed break-even statistic is available only after held-out targets are revealed.	Learn any retention rule on disjoint development data, freeze it before evaluation, and compare it with fixed-budget controls; preserve failed-query and negative-evidence states for ranking.

Appendix

Table S28: Failure mechanisms and corresponding validation requirements derived from the saved case traces.

Explore similar work

Graph-Constrained Policy Learning for Extreme Clinical Code Prediction
Jul 11, 2026Amritpal Singh, Sebastian Torres, Khawar Shakeel +1Hierarchical Classification Constrained Decoding
Division of Computer Science, Mathematics and Science St. John’s University, New York, NY, USA
Primary ICD Category Prediction using LLM-based Probing
Jun 27, 2026Chengyuan Liu, Xinyue Zhang, Yao Li +1Automated Medical Coding Linear Probing
Department of Statistics, Pennsylvania State University, PA, USA · Department of Biostatistics, University of North Carolina at Chapel Hill, NC, USA · Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, NC, USA
Hierarchical Modeling of ICD Codes in EHR Foundation Models
Jun 13, 2026Megha Thukral, Dong Gyun Kang, Rudra Pratap Singh +3Graph Representation Learning Hierarchical Representation Learning
School of Interactive Computing, Georgia Institute of Technology, Atlanta, United States · Optum AI, United States

Jul 11, 2026J/K move · Enter open · S save

Graph-Constrained Policy Learning for Extreme Clinical Code Prediction

Amritpal Singh, Sebastian Torres, Khawar Shakeel, Syed Ahmad Chan Bukhari

Division of Computer Science, Mathematics and Science St. John’s University, New York, NY, USA

Hierarchical Classification Constrained Decoding Multi-Label Classification Automated Medical Coding

Open paper AskPDF arXiv

Clinical code prediction maps unstructured discharge summaries to ICD-10-CM leaf codes in a large, sparse, and deeply hierarchical label space. Most systems treat the task as flat multi-label classification, scoring codes independently and providing limited training signal for rare labels. We propose a graph-constrained traversal policy that formulates ICD prediction as a finite-horizon decision process over a pruned code hierarchy. A single language model descends the graph level by level, selecting valid child nodes until billable leaf codes are reached. This converts extreme multi-label prediction into sparse, hierarchy-aware subset decisions while guaranteeing structurally valid outputs. On MIMIC-IV discharge summaries, our best supervised policy, SFT-1+, achieves 0.709 micro-F1 on a curated 50-code subset and 0.527 micro-F1 on the full 15,761-code space, outperforming flat baselines including CAML, LAAT, and PLM-ICD. In the full setting, SFT-1+ improves over the strongest flat baseline by 0.044 micro-F1 and 0.157 macro-F1, suggesting that graph-constrained decomposition mitigates the rare-code bottleneck. A controlled factorial study evaluates architecture, training algorithm, and data budget. Across both scales, one shared policy matches a three-specialist cascade while avoiding its context-window overflow on 28-32% of full-space test notes. Increasing supervised trajectory data is the only intervention that consistently improves performance, while GRPO reinforcement learning provides no benefit over supervised continuation with matched data. These results show that simple graph-constrained policy learning can outperform more complex flat, cascaded, and reinforcement-learning alternatives for extreme clinical code prediction.

Abstract

Explore similar work

Graph-Constrained Policy Learning for Extreme Clinical Code Prediction

Primary ICD Category Prediction using LLM-based Probing

Hierarchical Modeling of ICD Codes in EHR Foundation Models