Agentic LLM reasoning in pediatric diabetes outcome prediction from multimodal medical records

Daniel Palacios; Guantong Qi; Anna Kadi; Natali Borrego; Sukru Aras; Michael L. Ferm; Elizabeth Kubota-Mishra; Maria J. Redondo

q-bio.QMOct 29, 2026

Agentic LLM reasoning in pediatric diabetes outcome prediction from multimodal medical records

Authors: Daniel Palacios, Guantong Qi, Anna Kadi, Natali Borrego, Sukru Aras, Michael L. Ferm, Elizabeth Kubota-Mishra, Maria J. Redondo,

Organizations: Quantitative and Computational Biosciences, Baylor College of Medicine, Houston, Texas, USA · Department of Pediatrics, Baylor College of Medicine, Houston, Texas, USA · Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital, Houston, Texas, USA · Data Science Center, Texas Children’s Hospital, Houston, Texas, USA · Genetics and Genomics Graduate Program, Baylor College of Medicine, Houston, Texas, USA · School of Medicine, Baylor College of Medicine, Houston, Texas, USA · Department of Epidemiology and Population Health, Baylor College of Medicine, Houston, Texas, USA · Department of Pediatrics, Division of Pediatric Diabetes and Endocrinology, Baylor College of Medicine, Texas Children’s Hospital, Houston, Texas, USA · Division of Pediatric Endocrinology and Diabetes, Department of Pediatrics, NYU Grossman School of Medicine, New York, NY, USA · Children’s Nutrition Research Center, USDA/ARS, Department of Pediatrics, Baylor College of Medicine, Houston, Texas, USA · Immunobiology Center, Baylor College of Medicine, Texas Children’s Hospital, Houston, Texas, USA

Abstract

Large language model (LLM) agents are proposed for clinical prediction but are untested in pediatrics, where cohorts are small and outcomes are uncommon. Using records from 2,489 youth with type 2 diabetes (T2D) and 4,018 with type 1 diabetes (T1D), spanning structured data, note-derived social determinants and, in T1D, continuous glucose monitoring data, we predicted seven T2D outcomes, four also in T1D, at 24 months from 18 months of follow-up. CEDAR (Clinical Evidence-Driven Agent for Risk-prediction), an evidence-citing, self-auditing LLM pipeline, outperformed trained machine learning in both cohorts (pooled area under the receiver operating characteristic curve 0.815 versus 0.656 to 0.700 in T2D and 0.756 versus 0.603 to 0.631 in T1D), most on the sparsely screened outcomes. Only 4 of 70,246 values it cited did not match the chart. Reading only the evidence and reasoning cited by CEDAR, two pediatric endocrinologists reached the same prediction as CEDAR in about nine of ten cases, agreeing with each other on 90.5%.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Outcome	T1D	T2D
Optimal glycemic control	561 / 2544 (22.1%)	512 / 1059 (48.3%)
Hypertension	118 / 2531 (4.7%)	102 / 1144 (8.9%)
Microalbuminuria	58 / 280 (20.7%)	217 / 849 (25.6%)
Dyslipidemia	168 / 684 (24.6%)	396 / 675 (58.7%)
Insulin independence	—	381 / 1038 (36.7%)
Metformin response	—	200 / 597 (33.5%)

Appendix

List of supptables 1 Prediction targets: assessable population and prevalence. For each outcome, the assessable population is the set of patients with a valid input window and a non-null label at the target visit, the same denominator used to construct the held-out test sets. Values are n (prevalence %). These assessable

N

match the annotations in Supplementary Figure 15 (e.g. T1D microalbuminuria

N=280

). Dashes mark outcomes not defined for a cohort (GLP-1RA response, insulin independence, and metformin response are T2D-specific).

Demographics
Characteristic	T1D	T2D
	( $N=4{,}018$ )	( $N=2{,}489$ )
Age at diagnosis (yr)	$8.1\pm 4.1$	$13.6\pm 2.4$
Diabetes duration (yr)	$6.4\pm 4.6$	$6.1\pm 4.2$
Sex, male	2052 (51.1%)	1036 (41.6%)
Race, White	2525 (62.8%)	1575 (63.3%)

Appendix

List of supptables 2 Composition of the analysis cohorts. Demographics and social determinants of health for the Type 1 (T1D) and Type 2 (T2D) diabetes cohorts, as consumed by the prediction tasks. Continuous variables are mean

\pm

SD; binary variables as n (%) on the full-cohort denominator. Full descriptive clinical and epidemiological characterization of each cohort is reported in companion studies of the same dataset (in preparation) and is not reproduced here to keep the two lines of work independent. Missing SDOH values (1–32% per factor in T1D and 13–66% in T2D) are counted as absent, so these percentages are lower bounds.

Measure	$N$	Miss v1	Miss ever	$N$	Miss v1	Miss ever
	T1D			T2D
HbA1c	3802	19.0	5.4	2437	13.1	2.1
Glucose	3812	18.1	5.1	2446	12.7	1.7
BMI z-score	3801	19.6	5.4	2423	14.1	2.7
Total cholesterol	1554	91.2	61.3	1399	70.0	43.8
UACR ratio	1425	97.2	64.5	2095	47.7	15.8

Appendix

List of supptables 3 Feature availability and the surveillance-timing artifact. Ever-observed coverage for representative dynamic measures in each cohort. Miss v1 is the fraction of patients missing the value at the diagnosis visit alone; Miss ever is the fraction with no value in any of the 10 visit windows. The gap between the two columns is the key point. Renal-screening labs (UACR, urine microalbumin) read as 97–99% “missing” at diagnosis in T1D and 48–81% in T2D, but this reflects standard surveillance cadence. They are ordered at later follow-up visits, not at onset, so their true cohort coverage is much higher (e.g. T1D UACR observed in 1,425 patients, 35.5% of the cohort, not the 2.8% implied by visit 1 alone). The models consume the full visit sequence, so ever-observed coverage is the relevant availability. Full per-analyte statistics are given in the companion studies.

(a) Pooled family comparisons (Mann–Whitney $U$ )
Comparison	Cohort	$n$ (group 1 / 2)	$U$	Exact $p$
Cedar vs. ML family	T2D	7 / 56	352	$1.9\times 10^{-4}$
Cedar vs. classical ML	T2D	7 / 21	123	0.007
Cedar vs. temporal ML	T2D	7 / 35	229	$5.4\times 10^{-5}$
Cedar vs. ML family	T1D	4 / 32	115	0.007
Cedar vs. classical ML	T1D	4 / 12	40	0.058

Appendix

List of supptables 4 Exact test statistics for the pooled family comparisons and the multi-model comparison. The main text rounds any

p

-value below 0.001 to

p<0.001

for readability; the exact values are given here. (a) Mann–Whitney

U

tests in which one observation is one model’s ROC-AUC on one outcome, pooled within each family or subgroup;

n

is the number of such model–outcome AUCs per group. In the main comparison Cedar is the full pipeline only (7 observations in T2D, 4 in T1D); the reference rows below use the subgroup definitions of Supplementary Note 4, which also average Cedar ’s two partial ablations. These are descriptive effect sizes rather than formal hypothesis tests, because AUCs computed on overlapping patients and correlated outcomes are not independent (Methods, Section 10). (b) Paired Wilcoxon signed-rank tests of Cedar versus a single LLM call on the same base model, over the 11 cohort–outcome pairs, with Holm–Bonferroni correction across the four base models. Only llama-3-3-70b reaches significance.

Cohort	Outcome	Best ML model	Best ML	Cedar	Difference	Separated
T2D	Optimal glycemic control	XGBoost-Temporal	0.811 [0.764, 0.866]	0.838 [0.790, 0.884]	$+0.026$	no
T2D	Insulin independence	XGBoost-Temporal	0.866 [0.815, 0.910]	0.881 [0.833, 0.918]	$+0.014$	no
T2D	Metformin response	Logistic regression	0.779 [0.721, 0.831]	0.842 [0.785, 0.883]	$+0.063$	no
T2D	GLP-1RA response	Random forest	0.665 [0.556, 0.782]	0.739 [0.652, 0.826]	$+0.074$	no
T2D	Dyslipidemia	Random forest	0.576 [0.499, 0.642]	0.782 [0.731, 0.837]	$+0.206$	yes
T2D	Hypertension	Random forest	0.732 [0.632, 0.820]	0.778 [0.656, 0.891]	$+0.045$	no

Appendix

List of supptables 5 Cedar against the best statistical model on each outcome. For each cohort–outcome pair, the highest-scoring classical or temporal ML model and the full Cedar pipeline, ROC-AUC with 2,000-iteration patient-level bootstrap 95% confidence intervals on the shared held-out set. Separated marks pairs whose two intervals do not overlap, a conservative criterion. Cedar is ahead on 9 of 11 pairs, and the intervals separate only on dyslipidemia, in both cohorts.

Outcome	Single LLM	Cedar +FE	$\Delta$ AUC (paired)	Cost/patient
	AUC	AUC	[95% CI]	(vanilla $\to$ +FE)
GLP-1RA response	0.743	0.761	$+0.018$ [ $-0.030$ , $+0.063$ ]	$0.007$ \to0.115
Hypertension a	0.878	0.873	$-0.005$ [ $-0.055$ , $+0.068$ ]	$0.006$ \to0.106
Dyslipidemia	0.822	0.772	$-0.050$ [ $-0.128$ , $+0.014$ ]	$0.005$ \to0.102

Appendix

List of supptables 6 LLM-guided feature engineering does not improve accuracy over a single LLM (proof-of-concept, T2D). An LLM proposed, per outcome, which trajectory summaries of the available longitudinal features (slopes, deltas, means, variability) would be most predictive. Deterministic code then computed exactly those summaries from the raw visit values and appended them to the record Cedar reasoned over, 8 engineered features per outcome. The same outcome-specific exclusions used elsewhere were applied, so no target-leaking feature could be proposed. Evaluated on the three highest-headroom outcomes, a fixed held-out set of

N=60

patients each, with 2,000-iteration bootstrap confidence intervals. Re-expressing the same raw inputs as engineered trajectory features, which is the representation the temporal machine-learning baselines already receive, did not improve discrimination on any outcome. Every paired AUC difference versus a single LLM has a 95% CI spanning zero, and two of three point estimates are flat-to-negative, at roughly the same per-patient cost as Cedar itself (

0.10–0.12 versus

0.006 for a single call). This is a third convergent null (with self-refinement and reasoner ensembling; Supplementary Figure 4 ), reinforcing that in this regime accuracy is bounded by the available signal rather than by how it is reasoned over or re-encoded.

Outcome	$n$	Orig.	Ref.	$\Delta$	Orig.	Ref.	Orig.	Ref.	Orig. $\rightarrow$ Ref.
		ROC-AUC			MCC		Brier		Flips caught
Hypertension	239	0.787	0.756	$-0.030$	0.092	0.399	0.440	0.094	9/32 $\rightarrow$ 5/32
Microalbuminuria	192	0.854	0.880	$+0.026$	0.400	0.732	0.243	0.098	9/19 $\rightarrow$ 0/19
Dyslipidemia	221	0.790	0.802	$+0.012$	0.111	0.352	0.249	0.201	17/52 $\rightarrow$ 17/52
Optimal glycemic control	241	0.837	0.837	$+0.000$	0.485	0.545	0.174	0.167	6/54 $\rightarrow$ 2/54
Insulin independence	241	0.890	0.884	$-0.006$	0.579	0.596	0.138	0.140	0/45 $\rightarrow$ 0/45

Appendix

List of supptables 7 Refinement 3: Cedar with run-length-conditioned transition rates supplied in the prompt, T2D, 24-month horizon. Both configurations are scored on the identical held-out patients, and every base rate and transition rate quoted to the model is computed on the training folds only (Supplementary Note 2). Original is the configuration used throughout the paper; Refined adds training-fold base rates and transition rates conditioned on state run length. Flips caught is the number of patients whose 24-month state differs from their last observed state that the model predicted correctly, out of all such patients. Discrimination does not change; agreement and calibration improve; the dominant error class gets worse.

		Primary code, mechanism-first
Outcome	Errors	Anchoring	Calibration	Polarity	No signal
Hypertension	4	1	3	0	0
Microalbuminuria	6	2	1	1	2
Dyslipidemia	6	4	2	0	0
Optimal glycemic control	6	5	1	0	0
Insulin independence	6	6	0	0	0

Appendix

List of supptables 8 Cedar error taxonomy, T2D, 24-month horizon. All 40 misclassified cases in the 82-case stratified deck, coded by rule against Cedar ’s own reasoning and structured evidence (Supplementary Note 2). Codes are not mutually exclusive: nine errors carry more than one, so the bottom three rows give both defensible priority orders rather than a single primary code. “Any” counts each error under every code that applies and therefore exceeds 40. Anchoring = the prediction repeats the last observed outcome state and the patient’s state changed by 24 months. Calibration = the cited evidence and its direction are right and only the numeric probability falls on the wrong side of the threshold. Polarity = the direction lean of the cited evidence contradicts the predicted probability given the outcome’s valence. Input artifact = faithful reasoning over a physiologically impossible cited value. No signal = no code applies, i.e. nothing in the model’s own output distinguishes the case from correctly predicted ones. Label noise cannot be assigned without chart review and is absorbed into no signal . Anchoring requires the prior state to be recoverable from the citations, which holds in 70 of 82 cases.

Outcome	Prior	Full	$\Delta$	AUC	MCC	AUC	MCC	MCC †
	Flip ROC-AUC			Persistence		Cedar
Hypertension	0.817	0.841	$+0.024$	0.711	0.347	0.780	0.097	0.436
Microalbuminuria	0.559	0.554	$-0.005$	0.872	0.770	0.846	0.354	0.569
Dyslipidemia	0.540	0.583	$+0.043$	0.747	0.503	0.782	0.097	0.467
Optimal glycemic control	0.552	0.618	$+0.067$	0.777	0.556	0.833	0.472	0.532
Insulin independence	0.591	0.689	$+0.098$	0.804	0.604	0.885	0.565	0.561

Appendix

List of supptables 9 Refinement experiments, T2D, 24-month horizon. All quantities are on the same held-out patients used throughout the paper, and every calibrator, threshold and classifier is fitted on the training folds only (Supplementary Note 2). Flip ROC-AUC refers to a gradient-boosted classifier whose target is the binary indicator that the 24-month outcome state differs from the last observed state. The prior state column is the ROC-AUC available analytically from the transition rates alone, and the full column adds the complete 47-feature

\times

3-visit block. The difference is therefore the discrimination attributable to the observed record rather than to the population fact that recently acquired states are unstable. Persistence is the rule that predicts the last observed outcome state; it is defined only where that state is recoverable, hence

n=221

–241 rather than 250 (

n=66

of 100 for GLP-1RA response, which we do not interpret). Cedar recalibrated applies per-outcome isotonic calibration and a per-outcome threshold; it changes MCC and Brier but not the ordering of patients, so no ROC-AUC in this paper is recalibrated.

Tier 0: published training-free scores (0 labels)
	R (readmit-30)			M-B (death-365)			M-A (in-hospital)
Model	AUC	AUPRC	MCC	AUC	AUPRC	MCC	AUC	AUPRC	MCC
Best published score a	0.615	—	—	0.687	—	—	0.677	—	—
Tier 1: training-free LLM (0 labels)
Vanilla single LLM	0.616	0.341	0.136	0.811	0.472	0.388	0.814	0.391	0.342
Cedar	0.645	0.389	0.185	0.852	0.570	0.425	0.810	0.435	0.353

Appendix

List of supptables 10 External benchmark on MIMIC-IV: Cedar against published training-free scores and supervised machine learning. All models scored on the identical 250-patient held-out set for each task (Supplementary Note 3). R , 30-day readmission at discharge (62 positives); M-B , death within 365 days of discharge (50); M-A , in-hospital death predicted from the first 48 ICU hours (34). Tier 0 and Tier 1 consume no training labels; Tier 2 models were fit on 2,239 patients. MCC is taken at the prevalence-matched threshold. Bold marks the best value in each column. Cedar leads every model on M-B, trails the best supervised model by 0.041 ROC-AUC on R, and loses on M-A by more than ROC-AUC alone indicates.

Condition	History only	Both	Difference	95% CI
Dyslipidemia (clinical context only: 0.726)
All labs, no codes	0.683	0.753	$+0.070$ ∗	[+0.015, +0.134]
Codes only	0.801	0.815	$+0.014$	[-0.038, +0.070]
Visit 1	0.719	0.780	$+0.062$ ∗	[+0.007, +0.120]
Visit 2	0.708	0.766	$+0.058$ ∗	[+0.006, +0.119]
Visit 3	0.735	0.789	$+0.054$	[-0.007, +0.117]

Appendix

List of supptables 11 Information ablation by outcome and condition. ROC-AUC of the history-only reader and of the reader that sees both channels, 100 patients per outcome (50 positive, 50 negative), with the difference and its 1,000-resample paired bootstrap 95% confidence interval; ∗ marks intervals that exclude zero. The codes-only condition does not apply to the three HbA1c-defined outcomes. Exploratory: no correction for the roughly 150 comparisons (Supplementary Note 5).

Category	Count	Type	Examples
Labs & Vitals	18	Continuous	HbA1c, glucose, BMI z-score, lipid panel, ALT, AST, BUN, creatinine, C-peptide, UACR, $\beta$ -hydroxybutyrate, SBP/DBP (inpatient & outpatient)
Medications	3	Binary	Insulins, biguanide, GLP-1RA
Conditions	4	Binary	DKA, ketosis, retinopathy, neuropathy
Demographics	9	Mixed	Age at diagnosis, sex, Hispanic ethnicity, race (4 indicators), primary language, diabetes duration
SDOH	13	Mixed	Food insecurity, ACEs, housing instability, transportation, smoking, abuse (3 further binary indicators), insurance, parental education/employment, financial strain, family support
CGM (T1D only)	20	Continuous	Mean glucose, SD, CV, GMI, time-in/above/below-range, hypo episodes, reading counts, monitoring duration, % missing

Appendix

List of supptables 12 Feature categories in the analysis-ready dataset. All features are measured at each of 10 visit windows (6-month intervals, 0–5 years post-diagnosis). The T1D cohort adds a CGM modality not available for T2D.

Outcome	Excluded feature(s)
Optimal glycemic control	HbA1c
Hypertension	all blood-pressure features (SBP/DBP, inpatient & outpatient)
Dyslipidemia	lipid panel (TC, HDL, LDL, triglycerides)
Microalbuminuria	UACR ratio
Insulin independence	insulin (medication)
Metformin response	biguanide (medication), HbA1c

Appendix

List of supptables 13 Features excluded per outcome to prevent label leakage or circularity (Methods, Section 3).

Snippet, exactly as prepended to the prompt	Attribution
Optimal glycemic control
“ADA 2024: Target HbA1c $<$ 7% for youth with T2D; metformin + insulin are first-line therapies.”	ADA Standards of Care [ 20 ]
“ADA 2024: Youth-onset T2D is more aggressive than adult-onset; beta-cell decline is faster.”	ADA Standards of Care [ 20 ]
“ISPAD 2022: Intensive lifestyle intervention should accompany pharmacotherapy in pediatric T2D.”	ISPAD 2022 [ 21 ]
Hypertension
“ADA 2024: Screen BP at every visit; hypertension in T2D youth often co-occurs with obesity.”	ADA Standards of Care [ 20 ]

Appendix

List of supptables 14 Verbatim stage-S0 knowledge-injection text, T2D cohort, with attribution. Cedar ’s S0 stage prepends two to three fixed one-sentence snippets to the S1 prompt, chosen by outcome. Each line below is reproduced exactly as it is inserted into the prompt, including the source tags the model sees, and is attributed in the right-hand column; snippets marked as accepted pathophysiology are author-written summaries that carry no single source. This is a fixed per-outcome knowledge base, not retrieval: no snippet is patient-, cohort- or test-set-specific, and the same text is used for every patient scored on that outcome. The T1D snippet set is a separate, cohort-specific dictionary, given in Supplementary Table 15 . † This snippet understates the published TODAY result, a 51.7% metformin-monotherapy failure rate over a mean follow-up of 3.9 years; it is reproduced here as the model received it (Methods, Section 6).

Snippet, exactly as prepended to the prompt	Attribution
Optimal glycemic control
“ADA 2024: Target HbA1c $<$ 7% for most pediatric T1D patients; individualize based on hypoglycemia risk.”	ADA Standards of Care [ 20 ]
“ISPAD 2022: CGM time-in-range $>$ 70% (70–180 mg/dL) correlates with HbA1c $<$ 7% and improved outcomes.”	ISPAD 2022 [ 21 ]
“Younger age at diagnosis and longer diabetes duration are associated with higher HbA1c trajectories.”	Accepted pathophysiology; no specific citation
Hypertension
“ADA 2024: Screen BP at every visit; hypertension defined as BP $\geq$ 95th percentile for age/height/sex on $\geq$ 3 occasions.”	ADA Standards of Care [ 20 ]

Appendix

List of supptables 15 Verbatim stage-S0 knowledge-injection text, T1D cohort, with attribution. The T1D pipeline uses the same S0 mechanism as T2D (Supplementary Table 14 ) with its own snippet dictionary, reproduced here exactly as inserted into the prompt. Snippets are shown for the four outcomes evaluated in T1D; the dictionary also defines entries for insulin independence, metformin response and GLP-1RA response, which are T2D-specific and are not scored in T1D, so that text is never presented to the model in this cohort. Two differences from the T2D set are substantive rather than cosmetic. The thresholds are T1D-specific, for example UACR screening five years after diagnosis or at puberty rather than at diagnosis, and an LDL lifestyle threshold of 100 rather than 130 mg/dL. The glycemic-control snippets also reference CGM time-in-range, the only place a modality appears in S0.

	Vanilla LLM	Cedar
Prompt	Record + “return a probability”	Outcome-specific guideline snippets (S0) + a citation-forced reasoning schema (S1)
Model calls	1	5 (S0 is prepended context, not a call; S1 grounded CoT $\rightarrow$ S2 verify $\rightarrow$ S3 self-consistency, $K{=}3$ )
Output	Probability + brief free-text rationale	Probability + a structured FEATURE \| VISIT \| VALUE \| DIRECTION \| IMPORTANCE citation table with visit-tokened values
Evidence check	None	S2 auditor verifies every cited triple against the record and flags hallucinations
Probability stability	Single sample	Mean over $K{=}3$ resamples (variance reduction)
Failure mode addressed	—	Missing priors; ungrounded reasoning; hallucinated evidence; unstable probability

Appendix

List of supptables 16 How Cedar differs from a single (“Vanilla”) LLM call. Both run on the same underlying model and see the same record; Cedar adds four disciplined stages, each targeting a distinct failure mode of naive clinical prompting.

Analysis	$N$ per outcome	Rationale
Main benchmark, all families (Figure 2)	50–250 (adaptive)	Sized to the eligible population per outcome to stabilize imbalanced estimates while capping LLM cost.
Modality ablation, all families (Supplementary Figure 11 )	100 (fixed)	A coarser “does adding a modality shift AUC?” question; a fixed matched set caps the $4\times$ per-family LLM call volume.
Faithfulness audit (Figure 3)	10 (per outcome)	Per-citation, per-record manual-grade audit is labor-intensive; $\approx$ 70 records total suffice for grounding rates.
Late fusion & error analysis (Figure 4)	full held-out set	Combiners and error-correlation need the full set; no additional LLM calls beyond the main benchmark.
Feature-engineering PoC (Supplementary Table 6 )	60 (fixed)	Proof-of-concept on the three highest-headroom outcomes; small $N$ to bound the $\sim$ $20\times$ per-patient cost.
Information ablation (Figure 5)	1,600; 100 per outcome	All T2D test patient-outcomes for the complete-history comparison; for the conditions, 50 positive and 50 negative patients per outcome, T1D and T2D pooled, to bound the eight-condition LLM cost (Methods, Section 11).

Appendix

List of supptables 17 Sample size per analysis, and why it differs. Different analyses use different held-out sample sizes, chosen to balance statistical precision against LLM inference cost. This table collects each analysis, its per-outcome

N

, and the rationale, so the varying denominators throughout the paper are transparent. In all cases the same held-out patients are scored across every model family being compared, and uncertainty is quantified by 2,000-iteration patient-level bootstrap.

Cohort	Outcome	Diff. ( Cedar $-$ LLM)	95% CI	Equiv. @.02	@.025	@.03	Classification
T2D	Optimal Glycemic Control	$+0.010$	$-0.004$ to $+0.025$	No	Yes	Yes	Equivalent
T2D	Insulin Independence	$+0.011$	$-0.004$ to $+0.026$	No	Yes	Yes	Equivalent
T2D	Dyslipidemia	$-0.005$	$-0.028$ to $+0.018$	No	No	Yes	Equivalent
T2D	Hypertension	$+0.049$	$+0.016$ to $+0.086$	No	No	No	Favors Cedar
T2D	Metformin Response	$+0.018$	$-0.016$ to $+0.049$	No	No	No	Indeterminate
T2D	GLP-1RA Response	$-0.008$	$-0.056$ to $+0.038$	No	No	No	Indeterminate

Appendix

List of supptables 18 Single-LLM vs. Cedar equivalence (TOST), all 11 cohort–outcome pairs. Two one-sided tests on the paired bootstrap ROC-AUC difference ( Cedar

-

single LLM) were swept across three pre-specified margins rather than one. The choice of margin materially changes how many pairs qualify as equivalent (0 of 11 at

\pm 0.02

, 2 of 11 at

\pm 0.025

, 3 of 11 at

\pm 0.03

). Classification at the

\pm 0.03

margin: equivalent (TOST rejects), favors Cedar (95% CI excludes zero, positive), favors single LLM (CI excludes zero, negative), or indeterminate (CI includes zero, not equivalent). A pair can fail to reach equivalence without favoring either model, which is why 7 of 11 pairs below are indeterminate rather than a statement of no difference. Equivalence is determined by the TOST

p

-value at

\alpha=0.05

, equivalently by the 90% CI lying within the margin, not by the 95% CIs displayed, which are shown for effect-size interpretation. The two criteria can disagree at the boundary.

Metric	Statistical	LLM	Difference	LLM ahead	$p$ (family)	$p$ (best)
ROC-AUC	0.652	0.789	$+0.137$	11/11	0.001	0.005
MCC	0.197	0.287	$+0.090$	10/11	0.014	0.175
Balanced accuracy	0.601	0.644	$+0.044$	10/11	0.014	0.465
$F_{1}$ (positive)	0.425	0.493	$+0.068$	9/11	0.054	0.365

Appendix

List of supptables 19 Statistical versus LLM families under class-imbalance metrics. Each observation is one model’s score on one cohort–outcome pair; family means are paired by cohort–outcome and compared with a Wilcoxon signed-rank test over the 11 pairs. LLM ahead counts the pairs on which the LLM family mean exceeds the statistical family mean. The final column repeats the test on the best model in each family per outcome rather than the family mean, which is the contrast Supplementary Figure 16 e draws. Thresholds were selected on training folds only (Supplementary Note 1); ROC-AUC is threshold-free and is shown for reference. The direction of the effect is the same for every metric, while the best-versus-best contrast is statistically distinguishable only for ROC-AUC.

Metric	Single LLM	Cedar	$p$	Cedar ahead	$p$
	Pooled subgroups			Paired, Cedar vs Vanilla
ROC-AUC	0.782	0.793	0.264	7/11	0.175
MCC	0.275	0.285	0.759	7/11	0.695
Balanced accuracy	0.642	0.640	0.944	4/11	0.922
$F_{1}$ (positive)	0.485	0.486	0.953	4/11	0.922

Appendix

List of supptables 20 A single LLM call versus Cedar under class-imbalance metrics. Pooled columns compare the single-LLM subgroup (37 model–outcome observations) with the Cedar subgroup (33) by Mann–Whitney

U

, using the same subgroup definitions as Supplementary Figure 16 . The paired columns compare the full Cedar pipeline directly against the Vanilla single call on the same 11 cohort–outcome pairs by Wilcoxon signed-rank. No metric distinguishes the two, and on balanced accuracy and

F_{1}

the full pipeline is ahead on fewer than half the pairs, so the equivalence reported in the main text is firmer under these metrics than under ROC-AUC.

Item	Topic	Where reported
1	Title	Title page
2	Abstract	Abstract
3a	Background and rationale	Introduction, paragraphs 1–5
3b	Target population, intended use and users	Introduction; Discussion, paragraph 1
3c	Known health inequalities	Introduction, paragraph 1
4	Objectives	Introduction, the four questions and final paragraph

Appendix

List of supptables 21 TRIPOD+AI reporting checklist. Each item of the TRIPOD+AI statement [ 24 ] with the place in the main text (Methods sections are numbered) or Supplementary Information where it is reported. Items the study does not address are stated as such.

Clinical Outcome Prediction

Clinical outcome prediction uses patient records, imaging, signals, and biomarkers to estimate risks, disease trajectories, treatment response, and survival.

See all 268 papers

Aug 31, 2026

DIASENTINEL: An Auditable Multi-Agent System for Guideline-Grounded Diabetes Risk Screening

Large language models (LLMs) offer promising clinical decision support but remain vulnerable to hallucinated facts, unsupported recommendations, and citation errors. We present DIASENTINEL, a fully on-premise multi-agent system for one-year type 2 diabetes mellitus (T2DM) risk screening and guideline-grounded report generation from electronic health records (EHRs). The system integrates calibrated risk prediction, deterministic clinical signal extraction, Reciprocal Rank Fusion over American Diabetes Association (ADA) guidelines, and a hybrid verification layer combining rule-based checks with LLM entailment. The demonstration provides a real-time batch-screening dashboard and an interactive patient report interface with cited recommendations, verification results, and raw EHR comparison. DIASENTINEL demonstrates a practical framework for reliable, auditable, and privacy-preserving LLM-based clinical decision support.

Yung Wei Shueh, Zhi-Jie Chen, Chia-Hsuan Hsu +9

Far Eastern Memorial Hospital, Taiwan · University of Michigan, USA · Rutgers University, USA +1

Multi-Agent LLM Systems Medical Report Generation LLM Auditing Clinical Decision Support Clinical Risk Prediction

Ask PDF

Jun 23, 2026

T2D-Bench: Evidence-Gated Evaluation of LLM Outputs for Type 2 Diabetes Using a Multi-Layer Clinical-Lifestyle Knowledge Graph

Large language models (LLMs) can produce clinically fluent recommendations for type 2 diabetes while failing to satisfy guideline constraints or explicitly justify lifestyle-related glycemic claims. We present T2D-Bench, a reproducible benchmark and evidence-gated evaluation framework for testing whether LLM outputs satisfy explicit, graph-checkable evidence requirements. T2D-Bench is built on a multi-layer clinical-lifestyle knowledge graph that combines a biomedical spine (UMLS, DrugBank, SIDER), computable ADA Standards of Care rules, and lifestyle knowledge connected through a mechanistic bridge to glycemic laboratory effects. Across 100 structured vignettes spanning diagnosis, medication safety, and adversarial lifestyle conflicts, baseline outputs failed benchmark-defined evidence-path checks in 35% of cases for GPT-4o-mini and 33% for GPT-4o. The evidence gate detects unsupported omissions and uses constrained revision to bring outputs into verifier-level compliance with benchmark-defined evidence requirements. These results show that computable evidence constraints can make unsupported clinical omissions explicit, measurable, and correctable in diabetes-focused LLM outputs.

Saba A. Farahani, Hung Cao, Ramesh Jain +1

University of California, Irvine, Irvine, CA, USA

Type 1 Diabetes Clinical Decision Support Evidence-Grounded Generation Clinical Language Model Evaluation Knowledge Graphs

Ask PDF

Jun 17, 2026

Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis

Large language models (LLMs) can make clinical decision support more accessible by interpreting free-text documentation, but their direct use as diagnostic engines is limited by sensitivity to prompts, information order, and plausible but incorrect outputs. Structured machine-learning models offer more stable risk prediction, yet they require tabular inputs that are difficult to integrate with narrative clinical workflows. We present ClaMPAPP (Clinical Language-assisted Machine-learning Pipeline for Appendicitis), a hybrid system that uses an LLM as an interface rather than as the final decision-maker. ClaMPAPP extracts schema-constrained clinical features from note-like narratives, applies deterministic plausibility checks, and passes validated features to an XGBoost classifier trained on clinical, laboratory, and ultrasound variables. We evaluated ClaMPAPP on two independent pediatric appendicitis cohorts from German hospitals and compared it with end-to-end LLM baselines, including open-source and proprietary models. To preserve ground truth while testing free-text input, narratives were generated from structured electronic health records through template rendering and constrained LLM rewriting, with additional sentence-order permutation to assess positional robustness. ClaMPAPP achieved the strongest overall diagnostic performance in both internal and external validation while minimizing missed appendicitis cases, the key safety concern in acute triage. End-to-end LLMs showed unstable sensitivity-specificity trade-offs and greater degradation under narrative reordering. These results support an LLM-as-interface, ML-as-predictor design that separates natural-language usability from predictive inference and provides a more auditable pathway for clinical decision support.

Soheyl Bateni, Maryam Abdolali

K. N. Toosi University of Technology, Tehran, Iran

Medical Diagnosis Clinical Decision Support Clinical Information Extraction Clinical Prediction Input Order Sensitivity in Language Models

Ask PDF

Abstract

Clinical Outcome Prediction

Explore similar work

DIASENTINEL: An Auditable Multi-Agent System for Guideline-Grounded Diabetes Risk Screening

T2D-Bench: Evidence-Gated Evaluation of LLM Outputs for Type 2 Diabetes Using a Multi-Layer Clinical-Lifestyle Knowledge Graph

Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis