Can we generate portable representations for clinical time series data using LLMs?

Zongliang Ji; Yifei Sun; Andre Amaral; Anna Goldenberg; Rahul G. Krishnan

cs.LGMar 25, 2026

Can we generate portable representations for clinical time series data using LLMs?

Authors: Zongliang Ji, Yifei Sun, Andre Amaral, Anna Goldenberg, Rahul G. Krishnan

Organizations: University of Toronto, Canada · Vector Institute, Canada · Sunnybrook Health Sciences Centre, Canada

Abstract

Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.

Figures & tables

Figure 1: Motivation for Record2Vec. Numeric imputation loses clinical semantics and limits portability; human handoffs preserve meaning but are costly and variable. LLMs can create handoff-style summaries that retain semantics and provide portable inputs for forecasting and classification.

Figure 2: Methods to generate medical-record representations. Top to bottom: imputation pipeline; self-supervised TS representation (TSDE); TS foundation model (TimesFM); and Record2Vec: LLM summary followed by text embedding.

Dataset $\rightarrow$	HiRID					MIMIC					PPICU					Wins
Tasks $\rightarrow$	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab
Method $\downarrow$	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall
Mean	0.040	0.378	0.914	0.878	0.857	0.035	0.447	0.847	0.838	0.886	0.028	0.528	0.842	0.834	0.847	0
Right shift	0.041	0.342	0.923	0.884	0.858	0.037	0.409	0.886	0.841	0.884	0.031	0.490	0.868	0.831	0.846	1
Interpolation	0.435	0.370	0.918	0.879	0.853	0.103	0.430	0.873	0.844	0.893	0.133	0.493	0.857	0.837	0.847	0
TSDE	0.029	0.411	0.923	0.901	0.902	0.030	0.406	0.915	0.888	0.899	0.053	0.485	0.890	0.899	0.870	1

Table 1: In-distribution Result (RQ1). Best (per column) in bold ; second-best underlined . Wins counts the number of bests per method across all 15 downstream tasks.

Dataset $\rightarrow$	HiRID $\rightarrow$ PPICU					MIMIC $\rightarrow$ PPICU					Wins
Tasks $\rightarrow$	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab
Method $\downarrow$	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall
Mean	0.306	1.09	0.50	0.42	0.77	4.839	0.85	0.50	0.30	0.81	0
Right shift	0.417	1.03	0.50	0.53	0.88	6.344	0.76	0.50	0.23	0.82	0
Interpolation	24.80	0.91	0.50	0.34	0.81	13.41	0.71	0.50	0.21	0.80	0
TSDE	0.209	0.73	0.49	0.91	0.91	0.269	0.76	0.51	0.90	0.90	0

Table 2: Transfer Learning Result (RQ2). Best per column in bold ; second-best underlined . Wins counts tied bests.

Figure 3: Rank distributions for No-summary vs. three LLM variants across 15 in-distribution tasks (left) and 30 cross-site transfer tasks (right). Methods are ranked based on performance across five downstream tasks: Forecast (MSE), LoS (MAE), Mortality (AUROC), Drug (Recall), and Lab (Recall). See the Appendix M for the detailed values.

Figure 4: Rank distributions for four prompt variants with Gemini 2.0-flash across 15 in-distribution tasks (left) and 30 cross-site transfer tasks (right). Lower ranks indicate better performance. Methods are ranked based on performance across five downstream tasks: Forecast (MSE), LoS (MAE), Mortality (AUROC), Drug (Recall), and Lab (Recall). See the Appendix M for the detailed values.

Figure 5: Few-shot finetuning with 16 labeled target samples for mortality prediction (RQ5) shown across six transfer settings. All tasks are reported the same metric as previous section: Forecast: masked mse, LOS: mae, Mortality: AUROC. The first row is the result of Hirid $\rightarrow$ Ppicu and the second row is Mimic $\rightarrow$ Ppicu. Reference lines: blue = best in-distribution upper bound, orange = best pre-finetune result, black = best finetuned baseline. Record2Vec surpassed baselines with a large gap, reaching comparable performance to in-distribution.

Figure 6: In-distribution privacy prediction on Gender across three ICU datasets. Each bar is a different method. The dashed horizontal line in each panel marks the least leaked classical baseline among {TimesFM, TSDE, Mean, Interpolation, Right shift} for that metric/dataset. Our method reached comparable and mostly reduced results in terms of privacy leakage. Age results have less than 0.5% gap and can be inferred from Table 4

Figure 7: Information gained vs. lost with Record2Vec (heatmaps; in-distribution and cross-site transfer). Each cell shows the task-aligned relative change (%) of Record2Vec over representation vectors. Overall, Record2Vec yields the largest gains for drug , forecast , and mortality , including cross-site transfer, while demographic signals ( age , gender ) are attenuated or unchanged.

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Embedding	Silhouette
HiRID	Raw time-series	0.372
HiRID	CoT embedding	0.452
HiRID	ICD embedding	0.408
HiRID	zero_shot embedding	0.638
HiRID	Trend embedding	0.540
MIMIC	Raw time-series	0.332

Appendix

Table 3: Silhouette scores for different embeddings across datasets (higher is better).

Dataset $\rightarrow$	HiRID		MIMIC		PPICU
Tasks $\rightarrow$	Age	Gender	Age	Gender	Age	Gender
Method $\downarrow$	MAE	AUROC	MAE	AUROC	MAE	AUROC
Mean	0.805017	0.754615	0.803168	0.678225	0.76247	0.707425
Right shift	0.803392	0.742055	0.798042	0.694878	0.767785	0.726552
Interpolation	0.800455	0.726648	0.799755	0.670795	0.76095	0.720293
TSDE	0.798725	0.662075	0.797682	0.733638	0.76362	0.764377

Appendix

Table 4: RQ6 (Privacy): Age and gender prediction. Age is evaluated with MAE; Gender with AUROC.

	count	total	mean	std	max
File A ( Raw Text )	46,818	285,894,402	6,106.51	3,328.77	13,233.00
File B ( ICD Summarization )	47,119	11,027,274	234.03	60.52	567.00

Appendix

Table 5: Per-string tokenization summary (Qwen3-Embedding-8B; no special tokens).

Model	Latency (per sample)	GPU time (per batch)	Token counts	Cost (per 1k samples)
MedGemma + Qwen	$0.784+0.026\,\mathrm{s}$	$3959.41+119.61\,\mathrm{ms}$	$234.03~\text{tks/sample}$	–
Llama + Qwen	$0.25406+0.023\,\mathrm{s}$	$1299.4+117.24\,\mathrm{ms}$	$254.17~\text{tks/sample}$	–
Gemini + Qwen	$0.28+0.036\,\mathrm{s}$	–	$293.44~\text{tks/sample}$	$0.7\,\$ \text{/1k samples}$
Qwen (no summarization)	$2.9675\,\mathrm{s}$	$23608.14\,\mathrm{ms}$	$6106.56~\text{tks/sample}$	–

Appendix

Table 6: Latency, GPU time, token usage, and approximate cost of different summarization/inference pipelines.

Dataset $\rightarrow$	HiRID $\rightarrow$ PPICU			MIMIC $\rightarrow$ PPICU			Wins
Task $\rightarrow$	Forecast	LoS	Mortality	Forecast	LoS	Mortality
Method $\downarrow$	MSE $\downarrow$	MAE $\downarrow$	AUROC $\uparrow$	MSE $\downarrow$	MAE $\downarrow$	AUROC $\uparrow$
Mean	$0.133\pm 0.001$	$2.678\pm 0.229$	$0.600\pm 0.007$	$0.183\pm 0.003$	$1.171\pm 0.284$	$0.565\pm 0.050$	0
Right shift	$0.136\pm 0.002$	$1.803\pm 0.041$	$0.621\pm 0.036$	$0.181\pm 0.003$	$0.662\pm 0.056$	$0.606\pm 0.036$	0
Interpolation	$0.139\pm 0.001$	$1.999\pm 0.123$	$0.605\pm 0.033$	$0.183\pm 0.003$	$0.972\pm 0.090$	$0.563\pm 0.028$	0
TFM	$0.144\pm 0.005$	$0.896\pm 0.042$	$0.686\pm 0.003$	$0.163\pm 0.001$	$1.067\pm 0.028$	$0.683\pm 0.002$	0

Appendix

Table 7: Transfer performance from HiRID/MIMIC to PPICU. Values are mean

\pm

std. Best per column in bold ; second-best underlined . Wins count the number of best results per method.

Task	Small PPICU	Few-shot Finetune (Record2Vec)
Mortality	0.6843	0.72
LoS	1.03	0.49
Forecast	0.62	0.183
Feature	0.901	0.97
Lab	0.94	0.97

Appendix

Table 8: Performance comparison between models trained on a small PPICU subset vs. few-shot finetuned models.

	LSTM	GRU	Transformer	TCN
H $\rightarrow$ P Baseline	0.640	0.687	0.579	0.719
H $\rightarrow$ P Record2Vec	0.69788	0.6838	0.679	0.684
M $\rightarrow$ P Baseline	0.5235	0.5476	0.564	0.719
M $\rightarrow$ P Record2Vec	0.722	0.685	0.682	0.695

Appendix

Table 9: Model performance for HiRID

\rightarrow

PPICU (H

\rightarrow

P) and MIMIC

\rightarrow

PPICU (M

\rightarrow

P), comparing baseline vs. Record2Vec representations.

Ablations	Forecast	LoS	Mort	Drug	Lab
Qwen3-mean-l2	0.021	0.347	0.90	0.911	0.931
Qwen3-mean-none	0.027	0.371	0.88	0.871	0.919
Qwen3-cls-l2	0.024	0.530	0.89	0.900	0.940
Qwen3-last-l2	0.022	0.423	0.89	0.920	0.923
gte-Qwen2-instruct	0.028	0.480	0.88	0.886	0.918
llama-embed-nemotron	0.021	0.379	0.89	0.9097	0.929

Appendix

Table 10: Ablations on PPICU (in-distribution).

Ablations	Forecast	LoS	Mort	Drug	Lab
Qwen3-mean-l2	0.183	0.69	0.72	0.97	0.97
Qwen3-mean-none	0.209	0.66	0.71	0.94	0.96
Qwen3-cls-l2	0.230	0.78	0.50	0.995	0.98
Qwen3-last-l2	0.190	0.78	0.73	0.94	0.96
gte-Qwen2-instruct	0.250	0.82	0.50	0.93	0.96
llama-embed-nemotron	0.190	0.72	0.713	0.96	0.998

Appendix

Table 11: Ablations for HiRID

\rightarrow

PPICU transfer.

	HIRID					MIMIC					PPICU
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.021	0.3538	0.9	0.899	0.931	0.027	0.328	0.82	0.886	0.947	0.0217	0.376	0.64	0.9308	0.923
llama 3.1	0.0214	0.381	0.8797	0.899	0.925	0.0288	0.406	0.8131	0.886	0.938	0.029	0.3752	0.63	0.937	0.9271
medgemma	0.0237	0.3682	0.83	0.911	0.925	0.0299	0.4005	0.7788	0.8946	0.9432	0.029	0.3626	0.6346	0.925	0.936
gemini	0.028	0.347	0.8302	0.9064	0.9307	0.03	0.3328	0.77	0.903	0.942	0.017	0.358	0.6377	0.9308	0.9328

Appendix

Table 12: Summarization methods performance comparison across 15 in-distribution tasks (Figure 3 left subfigure)

	H $\rightarrow$ P					M $\rightarrow$ P					H $\rightarrow$ M
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.21	0.98	0.66	0.92	0.96	0.263	0.77	0.7	0.92	0.92	0.1284	0.693	0.73	0.881	0.832
llama 3.1	0.1908	0.71	0.735	0.9243	0.9718	0.249	0.5522	0.7097	0.9415	0.9285	0.134	0.6464	0.82	0.878	0.851
medgemma	0.21	0.6981	0.74	0.97	0.98	0.239	0.5627	0.7145	0.9428	0.95	0.0922	0.537	0.81	0.891	0.83
gemini	0.183	0.7017	0.7001	0.92	0.9737	0.258	0.49	0.73	0.95	0.9372	0.089	0.6661	0.8044	0.852	0.842

Appendix

Table 13: Summarization methods performance comparison across 30 transfer learning tasks (Part 1: Transfer Pairs 1-3)

	P $\rightarrow$ M					M $\rightarrow$ H					P $\rightarrow$ H
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.15971	0.518	0.75	0.913	0.8669	0.135	0.5805	0.8277	0.816	0.8857	0.0954	0.494	0.76	0.892	0.7895
llama 3.1	0.141	0.443	0.752	0.877	0.8669	0.1741	0.572	0.83	0.903	0.9085	0.096	0.4393	0.7657	0.835	0.734
medgemma	0.1281	0.4493	0.81	0.88	0.9054	0.135	0.5805	0.8277	0.8653	0.877	0.089	0.4682	0.7725	0.8637	0.8042
gemini	0.127	0.4886	0.7942	0.901	0.919	0.167	0.5864	0.8037	0.8239	0.916	0.083	0.429	0.81	0.8627	0.812

Appendix

Table 14: Summarization methods performance comparison across 30 transfer learning tasks (Part 2: Transfer Pairs 4-6)

	HIRID					MIMIC					PPICU
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
zero-shot	0.0244	0.354	0.8501	0.911	0.931	0.0282	0.3951	0.77	0.8894	0.9431	0.017	0.376	0.6367	0.9346	0.923
CoT	0.0268	0.381	0.8311	0.9078	0.925	0.0294	0.328	0.8101	0.886	0.938	0.029	0.358	0.6387	0.937	0.936
ICD	0.021	0.3517	0.83	0.899	0.9305	0.027	0.3844	0.8018	0.903	0.947	0.0284	0.3645	0.64	0.925	0.9234
Trend	0.028	0.347	0.9	0.9042	0.9263	0.03	0.406	0.82	0.8881	0.9469	0.0249	0.3713	0.63	0.9324	0.9308

Appendix

Table 15: Prompting methods performance comparison across 15 in-distribution tasks (Figure 4 left subfigure)

	H $\rightarrow$ P					M $\rightarrow$ P					H $\rightarrow$ M
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
zero-shot	0.183	0.6994	0.6888	0.9359	0.9758	0.2536	0.57	0.7045	0.949	0.95	0.1082	0.6467	0.73	0.8878	0.83
CoT	0.1996	0.69	0.66	0.92	0.9667	0.2461	0.49	0.7106	0.92	0.9375	0.089	0.693	0.7328	0.868	0.832
ICD	0.21	0.6989	0.6916	0.933	0.98	0.239	0.5199	0.73	0.926	0.9498	0.134	0.6564	0.82	0.891	0.851
Trend	0.1927	0.71	0.74	0.97	0.96	0.258	0.5595	0.7	0.95	0.92 — 0.1248	0.537	0.7629	0.852	0.847

Appendix

Table 16: Prompting methods performance comparison across 15 transfer learning tasks (Figure 4 right subfigure)

Dataset	Lab features	Vital features	Binary features
HiRID	ALP, ALT, AST, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Chloride, CreatineKinase, Creatinine, Glucose, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, AirwayPressurePeak, DiastolicBloodPressure, FiO2, FluidBalance, GCS, HeartRate, MeanBloodPressure, PEEP, RespiratoryRate, SAS, Saturation, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput, ICPMonitor, Ventilation	Analgesia, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Aspirin, CaReplacement, Dialysis, Diuretics, ICPMonitor, Insulin, KReplacement, LiverToxicDrug, MgReplacement, Neuroleptics, Paralysis, Saline, Sedation, Steroids, TPN, Transfusions, Vasopressors, Ventilation
MIMIC	ALP, ALT, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Creatinine, CreatinineKinase, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, DiastolicBloodPressure, FiO2, GCS, HeartRate, ICDSC, MeanBloodPressure, MinuteVentilation, PEEP, RespiratoryRate, SAS, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput	Analgesia, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Antipsychotics, CTScan, CaReplacement, Dialysis, Diuretics, EnteralNutrition, ICPMonitor, KReplacement, MRI, MgReplacement, PPI, Paralysis, TPN, Transfusions, UltraSound, Vasopressors, Ventilation, Xray
PPICU	ALP, ALT, AST, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Chloride, Creatinine, CreatinineKinase, GGT, Glucose, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, AirwayPressureIP, AirwayPressurePeak, DiastolicBloodPressure, FiO2, FluidInput, FluidOutput, GCS, HeartRate, ICDSC, MeanBloodPressure, MinuteVentilation, PAVSupport, PC, PEEP, PlateauPressure, RespiratoryRate, SAS, Saturation, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput	Analgesia, Antiarrhythmic, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Aspirin, Dialysis, Diuretics, EKG, EVD, ICPMonitor, InhaledVasodilator, Insulin, Isoproterenol, MgReplacement, Neuroleptics, OsmoticTherapy, PPI, PS, Paralysis, Sedation, Steroids, TPN, Transfusions, Vasopressors

Appendix

Table 17: Feature groupings for converting ICU data into hierarchical textual representations. Each feature is prefixed with its group label (e.g., lab_Hemoglobin , vital_HeartRate , bin_Vasopressors ) before tokenization.

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.799\,\pm\,4.12\mathrm{e}-4$	$1\,\pm\,1.893\mathrm{e}-4$	$0.802\,\pm\,2.517\mathrm{e}-5$	$1\,\pm\,1.732\mathrm{e}-5$	$0.808\,\pm\,1.323\mathrm{e}-3$	$1\,\pm\,3.87\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.799\,\pm\,5.4\mathrm{e}-4$	$1\,\pm\,1.888\mathrm{e}-4$	$0.802\,\pm\,6.557\mathrm{e}-5$	$1\,\pm\,7.572\mathrm{e}-5$	$0.811\,\pm\,1.514\mathrm{e}-3$	$1.001\,\pm\,1.177\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.808\,\pm\,1.848\mathrm{e}-3$	$1.018\,\pm\,3.714\mathrm{e}-3$	$0.806\,\pm\,1.544\mathrm{e}-3$	$1.01\,\pm\,4.083\mathrm{e}-3$	$0.818\,\pm\,5.124\mathrm{e}-3$	$1.015\,\pm\,7.344\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.8\,\pm\,1.198\mathrm{e}-3$	$1\,\pm\,2.955\mathrm{e}-4$	$0.802\,\pm\,1.229\mathrm{e}-4$	$1\,\pm\,1.528\mathrm{e}-5$	$0.809\,\pm\,2.309\mathrm{e}-5$	$1\,\pm\,1\mathrm{e}-5$
Llama-3.1 CoT lstm	$0.799\,\pm\,4.594\mathrm{e}-4$	$1\,\pm\,1.09\mathrm{e}-4$	$0.802\,\pm\,8.139\mathrm{e}-5$	$1\,\pm\,3.948\mathrm{e}-5$	$0.808\,\pm\,7.551\mathrm{e}-4$	$1\,\pm\,1.603\mathrm{e}-4$

Appendix

Table 18: In-distribution results - Age. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
No_sum_Trend lstm	$0.799\,\pm\,1.177\mathrm{e}-3$	$1\,\pm\,2.004\mathrm{e}-4$	$0.802\,\pm\,5.188\mathrm{e}-5$	$1\,\pm\,1.5\mathrm{e}-5$	$0.808\,\pm\,6.139\mathrm{e}-4$	$1\,\pm\,2.061\mathrm{e}-4$
No_sum_Trend mlp	$0.801\,\pm\,1.357\mathrm{e}-3$	$1.001\,\pm\,6.168\mathrm{e}-4$	$0.802\,\pm\,2.016\mathrm{e}-4$	$1\,\pm\,1.005\mathrm{e}-4$	$0.81\,\pm\,1.434\mathrm{e}-3$	$1\,\pm\,5.804\mathrm{e}-4$
No_sum_Trend patchtsmixer	$0.803\,\pm\,2.482\mathrm{e}-3$	$1.006\,\pm\,2.681\mathrm{e}-3$	$0.805\,\pm\,1.96\mathrm{e}-3$	$1.004\,\pm\,7.381\mathrm{e}-4$	$0.812\,\pm\,1.637\mathrm{e}-3$	$1.005\,\pm\,1.892\mathrm{e}-3$
No_sum_Trend timemixer	$0.8\,\pm\,6.989\mathrm{e}-4$	$1\,\pm\,1.409\mathrm{e}-4$	$0.802\,\pm\,4.933\mathrm{e}-5$	$1\,\pm\,5.774\mathrm{e}-6$	$0.809\,\pm\,1.931\mathrm{e}-4$	$1\,\pm\,5.888\mathrm{e}-5$
medgemma CoT +TFM lstm	$0.798\,\pm\,3.301\mathrm{e}-4$	$1\,\pm\,5.014\mathrm{e}-4$	$0.802\,\pm\,1.617\mathrm{e}-4$	$0.999\,\pm\,1.401\mathrm{e}-4$	$0.808\,\pm\,1.51\mathrm{e}-4$	$1\,\pm\,2.768\mathrm{e}-4$

Appendix

Table 19: In-distribution results - Age. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.808\,\pm\,1.305\mathrm{e}-3$	$1\,\pm\,4.309\mathrm{e}-4$	$0.806\,\pm\,1.069\mathrm{e}-4$	$1\,\pm\,6.028\mathrm{e}-5$
Llama-3.1 CoT +TFM mlp	$0.809\,\pm\,7.101\mathrm{e}-4$	$1.001\,\pm\,6.509\mathrm{e}-4$	$0.806\,\pm\,1.457\mathrm{e}-4$	$1\,\pm\,3.215\mathrm{e}-5$
Llama-3.1 CoT +TFM patchtsmixer	$0.82\,\pm\,2.642\mathrm{e}-3$	$1.019\,\pm\,4.362\mathrm{e}-3$	$0.81\,\pm\,2.535\mathrm{e}-3$	$1.01\,\pm\,6.56\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.809\,\pm\,1.014\mathrm{e}-3$	$1\,\pm\,2.955\mathrm{e}-4$	$0.806\,\pm\,1.986\mathrm{e}-4$	$1\,\pm\,1\mathrm{e}-5$
Llama-3.1 CoT lstm	$0.809\,\pm\,2.399\mathrm{e}-4$	$1\,\pm\,9.274\mathrm{e}-5$	$0.806\,\pm\,1.097\mathrm{e}-4$	$1\,\pm\,2.944\mathrm{e}-5$

Appendix

Table 20: Cross-site transfer results - Age. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
No_sum_Trend lstm	$0.808\,\pm\,1.044\mathrm{e}-3$	$1\,\pm\,3.807\mathrm{e}-4$	$0.806\,\pm\,1.065\mathrm{e}-4$	$1\,\pm\,1.88\mathrm{e}-4$
No_sum_Trend mlp	$0.81\,\pm\,1.264\mathrm{e}-3$	$1\,\pm\,7.27\mathrm{e}-4$	$0.806\,\pm\,3.889\mathrm{e}-4$	$1\,\pm\,2.887\mathrm{e}-5$
No_sum_Trend patchtsmixer	$0.814\,\pm\,3.215\mathrm{e}-3$	$1.007\,\pm\,3.595\mathrm{e}-3$	$0.808\,\pm\,4.422\mathrm{e}-3$	$1.005\,\pm\,3.252\mathrm{e}-3$
No_sum_Trend timemixer	$0.809\,\pm\,5.934\mathrm{e}-4$	$1\,\pm\,1.409\mathrm{e}-4$	$0.806\,\pm\,7.5\mathrm{e}-5$	$1\,\pm\,5\mathrm{e}-6$
medgemma CoT +TFM lstm	$0.807\,\pm\,6.886\mathrm{e}-4$	$1\,\pm\,3.897\mathrm{e}-4$	$0.806\,\pm\,2.281\mathrm{e}-4$	$1\,\pm\,1.756\mathrm{e}-4$

Appendix

Table 21: Cross-site transfer results - Age. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.682\,\pm\,8.894\mathrm{e}-3$	$0.555\,\pm\,1.246\mathrm{e}-2$	$0.655\,\pm\,1.33\mathrm{e}-2$	$0.585\,\pm\,1.425\mathrm{e}-2$	$0.693\,\pm\,7.3\mathrm{e}-3$	$0.573\,\pm\,1.212\mathrm{e}-2$
Llama-3.1 CoT +TFM mlp	$0.72\,\pm\,7.983\mathrm{e}-3$	$0.6\,\pm\,1.015\mathrm{e}-2$	$0.705\,\pm\,1.309\mathrm{e}-2$	$0.64\,\pm\,1.339\mathrm{e}-2$	$0.725\,\pm\,1.677\mathrm{e}-2$	$0.608\,\pm\,2.07\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.704\,\pm\,3.667\mathrm{e}-4$	$0.58\,\pm\,1.341\mathrm{e}-3$	$0.702\,\pm\,8.468\mathrm{e}-4$	$0.63\,\pm\,2\mathrm{e}-4$	$0.714\,\pm\,5.538\mathrm{e}-3$	$0.595\,\pm\,6.329\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.707\,\pm\,2.153\mathrm{e}-3$	$0.588\,\pm\,2.084\mathrm{e}-3$	$0.696\,\pm\,1.456\mathrm{e}-2$	$0.631\,\pm\,1.694\mathrm{e}-2$	$0.7\,\pm\,6.089\mathrm{e}-3$	$0.584\,\pm\,5.492\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.702\,\pm\,3.827\mathrm{e}-3$	$0.579\,\pm\,5.909\mathrm{e}-3$	$0.685\,\pm\,2.532\mathrm{e}-2$	$0.612\,\pm\,2.763\mathrm{e}-2$	$0.696\,\pm\,6.709\mathrm{e}-3$	$0.573\,\pm\,1.042\mathrm{e}-2$

Appendix

Table 22: In-distribution results - Gender. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
No_sum_Trend lstm	$0.74\,\pm\,2.23\mathrm{e}-3$	$0.63\,\pm\,3.426\mathrm{e}-3$	$0.722\,\pm\,1.427\mathrm{e}-2$	$0.663\,\pm\,1.613\mathrm{e}-2$	$0.778\,\pm\,2.76\mathrm{e}-3$	$0.681\,\pm\,3.846\mathrm{e}-3$
No_sum_Trend mlp	$0.755\,\pm\,3.027\mathrm{e}-3$	$0.641\,\pm\,3.93\mathrm{e}-3$	$0.753\,\pm\,1.008\mathrm{e}-2$	$0.698\,\pm\,1.139\mathrm{e}-2$	$0.787\,\pm\,9.173\mathrm{e}-3$	$0.686\,\pm\,1.275\mathrm{e}-2$
No_sum_Trend patchtsmixer	$0.749\,\pm\,4.704\mathrm{e}-3$	$0.631\,\pm\,5.715\mathrm{e}-3$	$0.754\,\pm\,6.242\mathrm{e}-3$	$0.695\,\pm\,7.373\mathrm{e}-3$	$0.79\,\pm\,1.108\mathrm{e}-2$	$0.684\,\pm\,1.554\mathrm{e}-2$
No_sum_Trend timemixer	$0.751\,\pm\,2.802\mathrm{e}-3$	$0.632\,\pm\,3.398\mathrm{e}-3$	$0.736\,\pm\,1.755\mathrm{e}-2$	$0.675\,\pm\,1.889\mathrm{e}-2$	$0.78\,\pm\,2.308\mathrm{e}-3$	$0.674\,\pm\,2.988\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.701\,\pm\,9.768\mathrm{e}-3$	$0.582\,\pm\,1.444\mathrm{e}-2$	$0.659\,\pm\,4.775\mathrm{e}-3$	$0.598\,\pm\,4.392\mathrm{e}-3$	$0.724\,\pm\,1.238\mathrm{e}-2$	$0.615\,\pm\,2.059\mathrm{e}-2$

Appendix

Table 23: In-distribution results - Gender. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.664\,\pm\,6.426\mathrm{e}-3$	$0.528\,\pm\,5.865\mathrm{e}-3$	$0.673\,\pm\,2.82\mathrm{e}-3$	$0.539\,\pm\,4.609\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.673\,\pm\,3.117\mathrm{e}-3$	$0.535\,\pm\,3.658\mathrm{e}-3$	$0.674\,\pm\,2.093\mathrm{e}-3$	$0.54\,\pm\,4.741\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.68\,\pm\,1.007\mathrm{e}-3$	$0.548\,\pm\,1.27\mathrm{e}-3$	$0.678\,\pm\,4.151\mathrm{e}-4$	$0.546\,\pm\,1.174\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.677\,\pm\,3.143\mathrm{e}-3$	$0.544\,\pm\,6.236\mathrm{e}-3$	$0.675\,\pm\,1.276\mathrm{e}-3$	$0.542\,\pm\,1.627\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.674\,\pm\,5.913\mathrm{e}-4$	$0.542\,\pm\,2.421\mathrm{e}-3$	$0.675\,\pm\,3.617\mathrm{e}-4$	$0.543\,\pm\,8.516\mathrm{e}-4$

Appendix

Table 24: Cross-site transfer results - Gender. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
No_sum_Trend lstm	$0.686\,\pm\,2.715\mathrm{e}-3$	$0.561\,\pm\,3.173\mathrm{e}-3$	$0.712\,\pm\,2.536\mathrm{e}-3$	$0.588\,\pm\,2.901\mathrm{e}-3$
No_sum_Trend mlp	$0.687\,\pm\,1.638\mathrm{e}-3$	$0.563\,\pm\,1.828\mathrm{e}-3$	$0.707\,\pm\,3.987\mathrm{e}-3$	$0.588\,\pm\,3.721\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.691\,\pm\,1.504\mathrm{e}-3$	$0.566\,\pm\,1.569\mathrm{e}-3$	$0.702\,\pm\,2.505\mathrm{e}-3$	$0.584\,\pm\,2.975\mathrm{e}-3$
No_sum_Trend timemixer	$0.694\,\pm\,1.373\mathrm{e}-3$	$0.568\,\pm\,1.335\mathrm{e}-3$	$0.702\,\pm\,1.888\mathrm{e}-3$	$0.582\,\pm\,3.587\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.674\,\pm\,3.333\mathrm{e}-3$	$0.542\,\pm\,4.159\mathrm{e}-3$	$0.68\,\pm\,9.923\mathrm{e}-4$	$0.55\,\pm\,1.624\mathrm{e}-3$

Appendix

Table 25: Cross-site transfer results - Gender. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.759\,\pm\,1.504\mathrm{e}-3$	$0.848\,\pm\,5.911\mathrm{e}-3$	$0.76\,\pm\,4.839\mathrm{e}-3$	$0.836\,\pm\,2.494\mathrm{e}-3$	$0.808\,\pm\,2.843\mathrm{e}-3$	$0.917\,\pm\,5.773\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.771\,\pm\,2.913\mathrm{e}-3$	$0.847\,\pm\,3.027\mathrm{e}-3$	$0.77\,\pm\,7.323\mathrm{e}-3$	$0.831\,\pm\,4.135\mathrm{e}-3$	$0.814\,\pm\,3.384\mathrm{e}-3$	$0.915\,\pm\,2.589\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.736\,\pm\,1.973\mathrm{e}-2$	$0.859\,\pm\,1.498\mathrm{e}-2$	$0.753\,\pm\,7.457\mathrm{e}-3$	$0.82\,\pm\,6.964\mathrm{e}-3$	$0.798\,\pm\,8.902\mathrm{e}-3$	$0.921\,\pm\,4.925\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.747\,\pm\,6.439\mathrm{e}-3$	$0.854\,\pm\,1.833\mathrm{e}-3$	$0.737\,\pm\,1.381\mathrm{e}-2$	$0.811\,\pm\,1.056\mathrm{e}-2$	$0.805\,\pm\,8.986\mathrm{e}-3$	$0.925\,\pm\,2.113\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.733\,\pm\,9.055\mathrm{e}-3$	$0.855\,\pm\,3.436\mathrm{e}-3$	$0.692\,\pm\,6.113\mathrm{e}-3$	$0.815\,\pm\,4.863\mathrm{e}-3$	$0.793\,\pm\,3.488\mathrm{e}-3$	$0.935\,\pm\,4.021\mathrm{e}-3$

Appendix

Table 26: In-distribution results - Drug. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
medgemma CoT +TFM patchtsmixer	$0.816\,\pm\,1.606\mathrm{e}-2$	$0.876\,\pm\,6.866\mathrm{e}-3$	$0.811\,\pm\,6.325\mathrm{e}-3$	$0.867\,\pm\,3.918\mathrm{e}-3$	$0.833\,\pm\,1.357\mathrm{e}-2$	$0.912\,\pm\,5.332\mathrm{e}-3$
medgemma CoT +TFM timemixer	$0.824\,\pm\,4.624\mathrm{e}-3$	$0.887\,\pm\,2.053\mathrm{e}-3$	$0.805\,\pm\,6.915\mathrm{e}-3$	$0.873\,\pm\,6.623\mathrm{e}-3$	$0.841\,\pm\,8.596\mathrm{e}-3$	$0.904\,\pm\,7.508\mathrm{e}-3$
medgemma CoT lstm	$0.821\,\pm\,8.278\mathrm{e}-3$	$0.88\,\pm\,5.981\mathrm{e}-3$	$0.787\,\pm\,1.078\mathrm{e}-2$	$0.855\,\pm\,2.705\mathrm{e}-3$	$0.84\,\pm\,7.288\mathrm{e}-3$	$0.919\,\pm\,4.913\mathrm{e}-3$
medgemma CoT mlp	$0.824\,\pm\,7.467\mathrm{e}-3$	$0.884\,\pm\,2.531\mathrm{e}-3$	$0.788\,\pm\,1.038\mathrm{e}-2$	$0.849\,\pm\,4.009\mathrm{e}-3$	$0.841\,\pm\,6.244\mathrm{e}-3$	$0.918\,\pm\,3.274\mathrm{e}-3$
medgemma CoT patchtsmixer	$0.816\,\pm\,6.976\mathrm{e}-3$	$0.882\,\pm\,1.833\mathrm{e}-3$	$0.773\,\pm\,8.922\mathrm{e}-3$	$0.848\,\pm\,5.822\mathrm{e}-3$	$0.828\,\pm\,4.332\mathrm{e}-3$	$0.919\,\pm\,3.551\mathrm{e}-3$

Appendix

Table 27: In-distribution results - Drug. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.769\,\pm\,1.565\mathrm{e}-3$	$0.931\,\pm\,1.886\mathrm{e}-3$	$0.777\,\pm\,2.04\mathrm{e}-3$	$0.931\,\pm\,1.593\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.763\,\pm\,1.471\mathrm{e}-3$	$0.933\,\pm\,1.939\mathrm{e}-3$	$0.774\,\pm\,1.628\mathrm{e}-3$	$0.938\,\pm\,4.576\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.75\,\pm\,2.781\mathrm{e}-3$	$0.945\,\pm\,4.113\mathrm{e}-3$	$0.767\,\pm\,3.535\mathrm{e}-3$	$0.932\,\pm\,1.907\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.764\,\pm\,3.39\mathrm{e}-3$	$0.937\,\pm\,5.739\mathrm{e}-3$	$0.772\,\pm\,1.664\mathrm{e}-3$	$0.936\,\pm\,6.964\mathrm{e}-4$
Llama-3.1 CoT lstm	$0.761\,\pm\,3.771\mathrm{e}-3$	$0.944\,\pm\,3.279\mathrm{e}-3$	$0.766\,\pm\,4.793\mathrm{e}-4$	$0.95\,\pm\,7.115\mathrm{e}-3$

Appendix

Table 28: Cross-site transfer results - Drug. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
medgemma CoT +TFM patchtsmixer	$0.772\,\pm\,2.573\mathrm{e}-3$	$0.907\,\pm\,3.85\mathrm{e}-3$	$0.784\,\pm\,9.672\mathrm{e}-4$	$0.916\,\pm\,9.366\mathrm{e}-4$
medgemma CoT +TFM timemixer	$0.783\,\pm\,1.245\mathrm{e}-3$	$0.912\,\pm\,4.037\mathrm{e}-3$	$0.793\,\pm\,2.516\mathrm{e}-4$	$0.91\,\pm\,2.613\mathrm{e}-3$
medgemma CoT lstm	$0.788\,\pm\,2.26\mathrm{e}-3$	$0.924\,\pm\,1.781\mathrm{e}-3$	$0.797\,\pm\,6.506\mathrm{e}-4$	$0.923\,\pm\,5.205\mathrm{e}-3$
medgemma CoT mlp	$0.79\,\pm\,6.558\mathrm{e}-4$	$0.924\,\pm\,3.799\mathrm{e}-3$	$0.795\,\pm\,6.553\mathrm{e}-4$	$0.926\,\pm\,7.525\mathrm{e}-4$
medgemma CoT patchtsmixer	$0.791\,\pm\,1.586\mathrm{e}-3$	$0.922\,\pm\,8.402\mathrm{e}-4$	$0.793\,\pm\,1.714\mathrm{e}-3$	$0.924\,\pm\,3.783\mathrm{e}-3$

Appendix

Table 29: Cross-site transfer results - Drug. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.872\,\pm\,2.001\mathrm{e}-3$	$0.92\,\pm\,8.798\mathrm{e}-4$	$0.895\,\pm\,2.53\mathrm{e}-3$	$0.936\,\pm\,1.722\mathrm{e}-3$	$0.847\,\pm\,7.912\mathrm{e}-4$	$0.917\,\pm\,6.809\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.881\,\pm\,2.75\mathrm{e}-3$	$0.926\,\pm\,8.832\mathrm{e}-4$	$0.903\,\pm\,1.691\mathrm{e}-3$	$0.94\,\pm\,4.133\mathrm{e}-4$	$0.857\,\pm\,1.235\mathrm{e}-3$	$0.925\,\pm\,6.144\mathrm{e}-4$
Llama-3.1 CoT +TFM patchtsmixer	$0.857\,\pm\,6.165\mathrm{e}-3$	$0.919\,\pm\,5.505\mathrm{e}-4$	$0.892\,\pm\,1.797\mathrm{e}-3$	$0.938\,\pm\,2.665\mathrm{e}-4$	$0.839\,\pm\,7.46\mathrm{e}-3$	$0.922\,\pm\,1.361\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.856\,\pm\,6.02\mathrm{e}-3$	$0.923\,\pm\,1.165\mathrm{e}-3$	$0.87\,\pm\,1.593\mathrm{e}-3$	$0.935\,\pm\,8.472\mathrm{e}-4$	$0.811\,\pm\,3.494\mathrm{e}-3$	$0.928\,\pm\,1.361\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.846\,\pm\,0\mathrm{e}0$	$0.928\,\pm\,0\mathrm{e}0$	$0.881\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$	$0.831\,\pm\,0\mathrm{e}0$	$0.931\,\pm\,0\mathrm{e}0$

Appendix

Table 30: In-distribution results - Lab. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
No_sum_Trend lstm	$0.878\,\pm\,2.658\mathrm{e}-4$	$0.928\,\pm\,6.463\mathrm{e}-4$	$0.906\,\pm\,2.312\mathrm{e}-4$	$0.943\,\pm\,6.671\mathrm{e}-4$	$0.875\,\pm\,1.3\mathrm{e}-4$	$0.932\,\pm\,1.242\mathrm{e}-4$
No_sum_Trend mlp	$0.895\,\pm\,0\mathrm{e}0$	$0.933\,\pm\,0\mathrm{e}0$	$0.914\,\pm\,0\mathrm{e}0$	$0.942\,\pm\,0\mathrm{e}0$	$0.883\,\pm\,0\mathrm{e}0$	$0.933\,\pm\,0\mathrm{e}0$
No_sum_Trend patchtsmixer	$0.879\,\pm\,7.515\mathrm{e}-4$	$0.929\,\pm\,1.331\mathrm{e}-3$	$0.905\,\pm\,2.403\mathrm{e}-4$	$0.942\,\pm\,7.504\mathrm{e}-4$	$0.875\,\pm\,1.472\mathrm{e}-3$	$0.936\,\pm\,2.444\mathrm{e}-4$
No_sum_Trend timemixer	$0.873\,\pm\,3.395\mathrm{e}-4$	$0.93\,\pm\,4.888\mathrm{e}-4$	$0.896\,\pm\,8.228\mathrm{e}-4$	$0.942\,\pm\,6.2\mathrm{e}-4$	$0.864\,\pm\,3.47\mathrm{e}-4$	$0.926\,\pm\,1.854\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.889\,\pm\,5.13\mathrm{e}-3$	$0.926\,\pm\,4.828\mathrm{e}-4$	$0.904\,\pm\,2.248\mathrm{e}-3$	$0.936\,\pm\,1.293\mathrm{e}-3$	$0.861\,\pm\,2.565\mathrm{e}-3$	$0.916\,\pm\,1.212\mathrm{e}-3$

Appendix

Table 31: In-distribution results - Lab. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.774\,\pm\,6.907\mathrm{e}-4$	$0.959\,\pm\,3.078\mathrm{e}-3$	$0.766\,\pm\,1.933\mathrm{e}-3$	$0.95\,\pm\,5.956\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.78\,\pm\,2.159\mathrm{e}-3$	$0.956\,\pm\,2.175\mathrm{e}-3$	$0.761\,\pm\,1.398\mathrm{e}-3$	$0.956\,\pm\,1.442\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.777\,\pm\,2.362\mathrm{e}-3$	$0.959\,\pm\,4.087\mathrm{e}-3$	$0.755\,\pm\,1.652\mathrm{e}-3$	$0.961\,\pm\,2.694\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.773\,\pm\,1.304\mathrm{e}-3$	$0.962\,\pm\,2.365\mathrm{e}-3$	$0.775\,\pm\,3.476\mathrm{e}-3$	$0.94\,\pm\,5.293\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.786\,\pm\,0\mathrm{e}0$	$0.954\,\pm\,0\mathrm{e}0$	$0.775\,\pm\,0\mathrm{e}0$	$0.952\,\pm\,0\mathrm{e}0$

Appendix

Table 32: Cross-site transfer results - Lab. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
No_sum_Trend lstm	$0.805\,\pm\,8.298\mathrm{e}-4$	$0.944\,\pm\,1.776\mathrm{e}-3$	$0.78\,\pm\,3.98\mathrm{e}-4$	$0.953\,\pm\,5.442\mathrm{e}-4$
No_sum_Trend mlp	$0.8\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$	$0.779\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$
No_sum_Trend patchtsmixer	$0.807\,\pm\,7.472\mathrm{e}-4$	$0.95\,\pm\,9.666\mathrm{e}-4$	$0.779\,\pm\,2.562\mathrm{e}-3$	$0.948\,\pm\,1.084\mathrm{e}-3$
No_sum_Trend timemixer	$0.808\,\pm\,1.079\mathrm{e}-3$	$0.941\,\pm\,4.553\mathrm{e}-4$	$0.789\,\pm\,7.168\mathrm{e}-4$	$0.937\,\pm\,1.581\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.779\,\pm\,3.345\mathrm{e}-3$	$0.955\,\pm\,5.752\mathrm{e}-3$	$0.775\,\pm\,1.652\mathrm{e}-3$	$0.94\,\pm\,2.151\mathrm{e}-3$

Appendix

Table 33: Cross-site transfer results - Lab. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse	masked_mae	masked_mse
Llama-3.1 CoT +TFM lstm	$0.073\,\pm\,1.4\mathrm{e}-3$	$0.014\,\pm\,3.704\mathrm{e}-4$	$0.093\,\pm\,8.415\mathrm{e}-4$	$0.018\,\pm\,2.152\mathrm{e}-4$	$0.074\,\pm\,8.799\mathrm{e}-4$	$0.015\,\pm\,2.152\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.078\,\pm\,2.696\mathrm{e}-4$	$0.013\,\pm\,7.234\mathrm{e}-5$	$0.096\,\pm\,3.704\mathrm{e}-4$	$0.018\,\pm\,1.193\mathrm{e}-4$	$0.08\,\pm\,8.415\mathrm{e}-4$	$0.015\,\pm\,2.801\mathrm{e}-4$
Llama-3.1 CoT +TFM patchtsmixer	$0.125\,\pm\,7.892\mathrm{e}-4$	$0.028\,\pm\,3.819\mathrm{e}-4$	$0.139\,\pm\,9.019\mathrm{e}-4$	$0.033\,\pm\,4.1\mathrm{e}-4$	$0.455\,\pm\,7.315\mathrm{e}-2$	$0.339\,\pm\,9.426\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.077\,\pm\,7.422\mathrm{e}-4$	$0.013\,\pm\,2.196\mathrm{e}-4$	$0.094\,\pm\,3.646\mathrm{e}-4$	$0.018\,\pm\,2.452\mathrm{e}-4$	$0.109\,\pm\,6.269\mathrm{e}-4$	$0.021\,\pm\,2.065\mathrm{e}-4$
Llama-3.1 CoT lstm	$0.075\,\pm\,4.255\mathrm{e}-4$	$0.015\,\pm\,1.569\mathrm{e}-4$	$0.094\,\pm\,4.84\mathrm{e}-4$	$0.019\,\pm\,1.787\mathrm{e}-4$	$0.077\,\pm\,6.466\mathrm{e}-4$	$0.017\,\pm\,1.758\mathrm{e}-4$

Appendix

Table 34: In-distribution results - Forecast. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse	masked_mae	masked_mse
No_sum_Trend lstm	$0.068\,\pm\,1.317\mathrm{e}-3$	$0.012\,\pm\,3.116\mathrm{e}-4$	$0.085\,\pm\,4.437\mathrm{e}-4$	$0.016\,\pm\,1.175\mathrm{e}-4$	$0.066\,\pm\,5.187\mathrm{e}-4$	$0.013\,\pm\,1.835\mathrm{e}-4$
No_sum_Trend mlp	$0.068\,\pm\,6.357\mathrm{e}-4$	$0.011\,\pm\,1.617\mathrm{e}-4$	$0.087\,\pm\,1.297\mathrm{e}-3$	$0.016\,\pm\,3.724\mathrm{e}-4$	$0.067\,\pm\,8.032\mathrm{e}-4$	$0.012\,\pm\,2.634\mathrm{e}-4$
No_sum_Trend patchtsmixer	$0.111\,\pm\,8.596\mathrm{e}-4$	$0.022\,\pm\,2.076\mathrm{e}-4$	$0.126\,\pm\,1.205\mathrm{e}-3$	$0.028\,\pm\,5.758\mathrm{e}-4$	$0.089\,\pm\,6.671\mathrm{e}-3$	$0.017\,\pm\,1.798\mathrm{e}-3$
No_sum_Trend timemixer	$0.072\,\pm\,2.463\mathrm{e}-3$	$0.012\,\pm\,5.846\mathrm{e}-4$	$0.088\,\pm\,1.163\mathrm{e}-3$	$0.016\,\pm\,3.008\mathrm{e}-4$	$0.079\,\pm\,1.129\mathrm{e}-3$	$0.014\,\pm\,3.756\mathrm{e}-4$
medgemma CoT +TFM lstm	$0.069\,\pm\,2.873\mathrm{e}-4$	$0.012\,\pm\,5.686\mathrm{e}-5$	$0.088\,\pm\,6.381\mathrm{e}-4$	$0.016\,\pm\,1.65\mathrm{e}-4$	$0.069\,\pm\,1.11\mathrm{e}-3$	$0.013\,\pm\,3.317\mathrm{e}-4$

Appendix

Table 35: In-distribution results - Forecast. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse
Llama-3.1 CoT +TFM lstm	$0.233\,\pm\,1.151\mathrm{e}-2$	$0.12\,\pm\,8.28\mathrm{e}-3$	$0.263\,\pm\,5.289\mathrm{e}-4$	$0.151\,\pm\,2.991\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.257\,\pm\,1.285\mathrm{e}-3$	$0.138\,\pm\,2.024\mathrm{e}-3$	$0.266\,\pm\,8.977\mathrm{e}-4$	$0.155\,\pm\,1.573\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.309\,\pm\,1.042\mathrm{e}-2$	$0.209\,\pm\,2.272\mathrm{e}-2$	$0.36\,\pm\,6.736\mathrm{e}-3$	$0.267\,\pm\,1.614\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.264\,\pm\,5.936\mathrm{e}-3$	$0.141\,\pm\,4.233\mathrm{e}-3$	$0.27\,\pm\,1.086\mathrm{e}-3$	$0.155\,\pm\,1.749\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.249\,\pm\,3.484\mathrm{e}-3$	$0.14\,\pm\,4.978\mathrm{e}-3$	$0.265\,\pm\,1.101\mathrm{e}-3$	$0.159\,\pm\,1.279\mathrm{e}-3$

Appendix

Table 36: Cross-site transfer results - Forecast. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse
No_sum_Trend lstm	$0.249\,\pm\,2.623\mathrm{e}-3$	$0.143\,\pm\,2.817\mathrm{e}-3$	$0.271\,\pm\,1.111\mathrm{e}-3$	$0.163\,\pm\,1.111\mathrm{e}-3$
No_sum_Trend mlp	$0.253\,\pm\,2.225\mathrm{e}-3$	$0.145\,\pm\,2.026\mathrm{e}-3$	$0.284\,\pm\,2.136\mathrm{e}-3$	$0.175\,\pm\,1.803\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.293\,\pm\,5.256\mathrm{e}-3$	$0.19\,\pm\,1.041\mathrm{e}-2$	$0.345\,\pm\,9.259\mathrm{e}-3$	$0.243\,\pm\,1.885\mathrm{e}-2$
No_sum_Trend timemixer	$0.253\,\pm\,2.217\mathrm{e}-3$	$0.144\,\pm\,2.074\mathrm{e}-3$	$0.277\,\pm\,2.751\mathrm{e}-3$	$0.169\,\pm\,3.138\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.24\,\pm\,6.816\mathrm{e}-3$	$0.125\,\pm\,8.342\mathrm{e}-3$	$0.269\,\pm\,7.465\mathrm{e}-4$	$0.157\,\pm\,3.317\mathrm{e}-3$

Appendix

Table 37: Cross-site transfer results - Forecast. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.302\,\pm\,1.48\mathrm{e}-2$	$0.379\,\pm\,3.635\mathrm{e}-2$	$0.322\,\pm\,8.197\mathrm{e}-3$	$0.567\,\pm\,4.639\mathrm{e}-3$	$0.351\,\pm\,4.62\mathrm{e}-3$	$0.937\,\pm\,8.33\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.304\,\pm\,9.901\mathrm{e}-3$	$0.372\,\pm\,1.076\mathrm{e}-2$	$0.304\,\pm\,3.403\mathrm{e}-3$	$0.505\,\pm\,1.562\mathrm{e}-2$	$0.343\,\pm\,9.457\mathrm{e}-3$	$0.922\,\pm\,1.08\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.524\,\pm\,2.093\mathrm{e}-2$	$0.691\,\pm\,3.479\mathrm{e}-2$	$0.47\,\pm\,2.87\mathrm{e}-2$	$0.766\,\pm\,1.297\mathrm{e}-1$	$0.416\,\pm\,9.236\mathrm{e}-3$	$1.011\,\pm\,3.858\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.348\,\pm\,3.617\mathrm{e}-3$	$0.38\,\pm\,1.789\mathrm{e}-3$	$0.338\,\pm\,1.84\mathrm{e}-2$	$0.519\,\pm\,3.783\mathrm{e}-2$	$0.337\,\pm\,5.955\mathrm{e}-3$	$0.924\,\pm\,7.734\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.378\,\pm\,1.686\mathrm{e}-2$	$0.527\,\pm\,1.747\mathrm{e}-2$	$0.396\,\pm\,1.467\mathrm{e}-2$	$0.692\,\pm\,9.725\mathrm{e}-3$	$0.358\,\pm\,6.408\mathrm{e}-3$	$0.958\,\pm\,7.056\mathrm{e}-3$

Appendix

Table 38: In-distribution results - Los. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
No_sum_Trend lstm	$0.301\,\pm\,6.829\mathrm{e}-3$	$0.399\,\pm\,1.76\mathrm{e}-2$	$0.283\,\pm\,5.464\mathrm{e}-3$	$0.538\,\pm\,7.988\mathrm{e}-3$	$0.339\,\pm\,5.337\mathrm{e}-3$	$0.92\,\pm\,6.172\mathrm{e}-3$
No_sum_Trend mlp	$0.3\,\pm\,9.631\mathrm{e}-3$	$0.386\,\pm\,2.206\mathrm{e}-2$	$0.294\,\pm\,1.381\mathrm{e}-2$	$0.539\,\pm\,1.816\mathrm{e}-2$	$0.343\,\pm\,1.163\mathrm{e}-2$	$0.929\,\pm\,2.489\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.348\,\pm\,3.213\mathrm{e}-3$	$0.428\,\pm\,8.989\mathrm{e}-3$	$0.329\,\pm\,9.072\mathrm{e}-3$	$0.563\,\pm\,6.203\mathrm{e}-3$	$0.37\,\pm\,1.121\mathrm{e}-2$	$0.93\,\pm\,1.805\mathrm{e}-2$
No_sum_Trend timemixer	$0.363\,\pm\,1.044\mathrm{e}-2$	$0.457\,\pm\,1.057\mathrm{e}-2$	$0.326\,\pm\,1.872\mathrm{e}-2$	$0.575\,\pm\,8.601\mathrm{e}-3$	$0.325\,\pm\,9.516\mathrm{e}-3$	$0.916\,\pm\,6.562\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.312\,\pm\,6.866\mathrm{e}-3$	$0.405\,\pm\,2.334\mathrm{e}-2$	$0.312\,\pm\,1.294\mathrm{e}-2$	$0.569\,\pm\,2.844\mathrm{e}-2$	$0.346\,\pm\,3.115\mathrm{e}-3$	$0.929\,\pm\,4.153\mathrm{e}-4$

Appendix

Table 39: In-distribution results - Los. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.343\,\pm\,6.473\mathrm{e}-3$	$1.017\,\pm\,4.164\mathrm{e}-3$	$0.347\,\pm\,1.144\mathrm{e}-3$	$0.988\,\pm\,2.541\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.373\,\pm\,3.751\mathrm{e}-3$	$1.047\,\pm\,9.349\mathrm{e}-3$	$0.351\,\pm\,2.08\mathrm{e}-3$	$1.014\,\pm\,1.484\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.516\,\pm\,1.446\mathrm{e}-2$	$1.113\,\pm\,1.521\mathrm{e}-2$	$0.489\,\pm\,1.7\mathrm{e}-2$	$1.091\,\pm\,1.069\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.425\,\pm\,6.04\mathrm{e}-3$	$1.043\,\pm\,3.039\mathrm{e}-3$	$0.392\,\pm\,2.021\mathrm{e}-2$	$1.015\,\pm\,1.283\mathrm{e}-2$
Llama-3.1 CoT lstm	$0.795\,\pm\,3.91\mathrm{e}-2$	$1.544\,\pm\,5.866\mathrm{e}-2$	$0.558\,\pm\,2.002\mathrm{e}-2$	$1.267\,\pm\,5.593\mathrm{e}-2$

Appendix

Table 40: Cross-site transfer results - Los. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
No_sum_Trend lstm	$0.704\,\pm\,8.339\mathrm{e}-2$	$1.389\,\pm\,1.546\mathrm{e}-1$	$0.742\,\pm\,4.071\mathrm{e}-2$	$1.461\,\pm\,9.049\mathrm{e}-2$
No_sum_Trend mlp	$0.754\,\pm\,3.342\mathrm{e}-2$	$1.496\,\pm\,4.605\mathrm{e}-2$	$0.794\,\pm\,3.188\mathrm{e}-2$	$1.568\,\pm\,6.687\mathrm{e}-2$
No_sum_Trend patchtsmixer	$0.853\,\pm\,3.164\mathrm{e}-2$	$1.606\,\pm\,6.786\mathrm{e}-2$	$0.776\,\pm\,6.463\mathrm{e}-2$	$1.468\,\pm\,1.277\mathrm{e}-1$
No_sum_Trend timemixer	$0.802\,\pm\,5.126\mathrm{e}-2$	$1.562\,\pm\,1.056\mathrm{e}-1$	$0.753\,\pm\,3.791\mathrm{e}-2$	$1.451\,\pm\,8.137\mathrm{e}-2$
medgemma CoT +TFM lstm	$0.369\,\pm\,5.866\mathrm{e}-3$	$1.007\,\pm\,5.832\mathrm{e}-3$	$0.352\,\pm\,6.795\mathrm{e}-3$	$0.963\,\pm\,3.323\mathrm{e}-3$

Appendix

Table 41: Cross-site transfer results - Los. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.434\,\pm\,4.047\mathrm{e}-2$	$0.888\,\pm\,1.253\mathrm{e}-2$	$0.539\,\pm\,1.934\mathrm{e}-2$	$0.851\,\pm\,8.073\mathrm{e}-3$	$0.573\,\pm\,2.554\mathrm{e}-2$	$0.796\,\pm\,1.254\mathrm{e}-2$
Llama-3.1 CoT +TFM mlp	$0.443\,\pm\,2.504\mathrm{e}-2$	$0.89\,\pm\,1.262\mathrm{e}-2$	$0.514\,\pm\,2.709\mathrm{e}-2$	$0.847\,\pm\,1.626\mathrm{e}-2$	$0.546\,\pm\,4.932\mathrm{e}-2$	$0.789\,\pm\,3.076\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.351\,\pm\,4.992\mathrm{e}-3$	$0.847\,\pm\,2.807\mathrm{e}-3$	$0.412\,\pm\,4.796\mathrm{e}-3$	$0.776\,\pm\,4.092\mathrm{e}-3$	$0.247\,\pm\,1.763\mathrm{e}-3$	$0.506\,\pm\,1.556\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.343\,\pm\,1.168\mathrm{e}-2$	$0.847\,\pm\,4.997\mathrm{e}-3$	$0.488\,\pm\,4.005\mathrm{e}-3$	$0.834\,\pm\,2.875\mathrm{e}-3$	$0.537\,\pm\,3.13\mathrm{e}-2$	$0.788\,\pm\,1.751\mathrm{e}-2$
Llama-3.1 CoT lstm	$0.341\,\pm\,9.298\mathrm{e}-3$	$0.843\,\pm\,5.597\mathrm{e}-3$	$0.451\,\pm\,5.876\mathrm{e}-3$	$0.815\,\pm\,3.777\mathrm{e}-3$	$0.457\,\pm\,2.569\mathrm{e}-2$	$0.731\,\pm\,1.79\mathrm{e}-2$

Appendix

Table 42: In-distribution results - Mort. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
medgemma ICD +TFM lstm	$0.582\,\pm\,6.357\mathrm{e}-3$	$0.927\,\pm\,2.878\mathrm{e}-3$	$0.572\,\pm\,1.236\mathrm{e}-2$	$0.87\,\pm\,5.52\mathrm{e}-3$	$0.66\,\pm\,2.533\mathrm{e}-2$	$0.844\,\pm\,1.297\mathrm{e}-2$
medgemma ICD +TFM mlp	$0.582\,\pm\,2.505\mathrm{e}-2$	$0.929\,\pm\,8.287\mathrm{e}-3$	$0.571\,\pm\,4.68\mathrm{e}-3$	$0.874\,\pm\,4.162\mathrm{e}-4$	$0.634\,\pm\,1.234\mathrm{e}-2$	$0.837\,\pm\,7.092\mathrm{e}-3$
medgemma ICD +TFM patchtsmixer	$0.467\,\pm\,5.098\mathrm{e}-3$	$0.899\,\pm\,2.98\mathrm{e}-3$	$0.489\,\pm\,2.237\mathrm{e}-2$	$0.821\,\pm\,1.109\mathrm{e}-2$	$0.582\,\pm\,1.334\mathrm{e}-2$	$0.786\,\pm\,8.672\mathrm{e}-3$
medgemma ICD +TFM timemixer	$0.453\,\pm\,6.038\mathrm{e}-2$	$0.894\,\pm\,1.792\mathrm{e}-2$	$0.526\,\pm\,1.474\mathrm{e}-2$	$0.853\,\pm\,7.344\mathrm{e}-3$	$0.609\,\pm\,1.786\mathrm{e}-2$	$0.827\,\pm\,1.059\mathrm{e}-2$
medgemma ICD lstm	$0.506\,\pm\,2.012\mathrm{e}-2$	$0.906\,\pm\,7.309\mathrm{e}-3$	$0.547\,\pm\,1.449\mathrm{e}-2$	$0.858\,\pm\,7.275\mathrm{e}-3$	$0.594\,\pm\,8.807\mathrm{e}-3$	$0.81\,\pm\,5.606\mathrm{e}-3$

Appendix

Table 43: In-distribution results - Mort. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.365\,\pm\,5.073\mathrm{e}-3$	$0.656\,\pm\,2.368\mathrm{e}-3$	$0.392\,\pm\,2.422\mathrm{e}-3$	$0.681\,\pm\,9.8\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.373\,\pm\,8.213\mathrm{e}-4$	$0.664\,\pm\,1.107\mathrm{e}-3$	$0.383\,\pm\,7.763\mathrm{e}-3$	$0.67\,\pm\,6.792\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.342\,\pm\,2.198\mathrm{e}-3$	$0.625\,\pm\,1.408\mathrm{e}-3$	$0.361\,\pm\,4.654\mathrm{e}-3$	$0.633\,\pm\,5.435\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.369\,\pm\,4.226\mathrm{e}-3$	$0.662\,\pm\,3.855\mathrm{e}-3$	$0.376\,\pm\,2.861\mathrm{e}-3$	$0.665\,\pm\,2.893\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.362\,\pm\,3.588\mathrm{e}-3$	$0.653\,\pm\,1.988\mathrm{e}-3$	$0.365\,\pm\,1.458\mathrm{e}-3$	$0.652\,\pm\,1.682\mathrm{e}-3$

Appendix

Table 44: Cross-site transfer results - Mort. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
medgemma ICD +TFM lstm	$0.459\,\pm\,2.197\mathrm{e}-3$	$0.715\,\pm\,1.475\mathrm{e}-3$	$0.489\,\pm\,4.21\mathrm{e}-3$	$0.741\,\pm\,2.302\mathrm{e}-3$
medgemma ICD +TFM mlp	$0.46\,\pm\,7.566\mathrm{e}-3$	$0.717\,\pm\,3.012\mathrm{e}-3$	$0.484\,\pm\,9.579\mathrm{e}-3$	$0.736\,\pm\,6.33\mathrm{e}-3$
medgemma ICD +TFM patchtsmixer	$0.408\,\pm\,2.124\mathrm{e}-3$	$0.675\,\pm\,2.087\mathrm{e}-3$	$0.449\,\pm\,2.893\mathrm{e}-3$	$0.7\,\pm\,5.408\mathrm{e}-4$
medgemma ICD +TFM timemixer	$0.438\,\pm\,1.577\mathrm{e}-2$	$0.712\,\pm\,4.393\mathrm{e}-3$	$0.482\,\pm\,3.808\mathrm{e}-3$	$0.735\,\pm\,6.012\mathrm{e}-4$
medgemma ICD lstm	$0.455\,\pm\,7.277\mathrm{e}-3$	$0.715\,\pm\,1.228\mathrm{e}-3$	$0.49\,\pm\,4.828\mathrm{e}-3$	$0.741\,\pm\,2.204\mathrm{e}-3$

Appendix

Table 45: Cross-site transfer results - Mort. (part 2/2)

Explore similar work

Aug 24, 2026

Future Querying: Can LLMs Serve as Implicit Medical World Models?

Traditional clinical prediction models rely on task-specific pipelines and curated, structured data, which scale poorly and underutilize unstructured text. To address this, we introduce future querying, a paradigm that probes whether large language models (LLMs) can function as implicit medical world models by evaluating their ability to answer time-indexed clinical queries about a patient's future. Our framework operates on unstructured clinical documentation using endpoint-agnostic training, enabling a single model to answer diverse clinical queries over patient trajectories without manual feature engineering or task-specific retraining. We show that small, locally fine-tuned open-weight models can match or approach larger proprietary systems, making the framework suitable for privacy-preserving, on-premise deployment. Evaluated on a new synthetic medical reports dataset and real ICU notes from the MIMIC-IV dataset, our results provide encouraging evidence that LLMs can capture aspects of clinical dynamics.

Siri Willems, James Butterworth, Lore Goetschalckx +4

imec, AI-labs⋆⋆, Paris, France

Medical LLMs Clinical Prediction Longitudinal EHR Modeling Clinical Outcome Prediction

Ask PDF

May 12, 2026

Training Large Language Models to Predict Clinical Events

Longitudinal clinical notes contain rich evidence of how patients evolve over time, but converting this signal into training supervision for clinical prediction remains challenging. We extend Foresight Learning to clinical prediction by converting time-ordered MIMIC-III notes into examples consisting of past patient context, a natural-language question about a possible future event, and a label resolved from later documentation. This process yields 6,900 prediction examples from 702 admissions across medications, procedures, organ support, microbiology, and mortality. A small LoRA adapter trained on these examples improves over the prompted base model, reducing expected calibration error from 0.1269 to 0.0398 and Brier score from 0.199 to 0.145, while slightly outperforming GPT-5 point estimates on held-out questions. The approach enables reusable clinical prediction supervision from longitudinal notes without hand-engineered structured features or endpoint-specific classifiers.

Benjamin Turtel, Paul Wilczewski, Kris Skotheim

Lightning Rod Labs

Healthcare Language Model Calibration Clinical Prediction Longitudinal EHR Modeling Electronic Health Records Medical LLMs+1

Ask PDF

Jun 1, 2026

ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning

Large language models (LLMs) exhibit strong natural-language reasoning abilities for clinical decision support, but struggle to effectively model structured longitudinal electronic health records (EHRs). In contrast, EHR foundation models can learn predictive patient representations, yet lack interpretable language-based reasoning. To bridge this gap, we propose ChatHealthAI, a multimodal reasoning framework that aligns structured EHR representations from a pretrained EHR foundation model with the semantic space of a frozen LLM through a task-aware resampler. By integrating longitudinal patient representations with refined clinical event descriptions, ChatHealthAI enables clinically grounded natural-language reasoning while maintaining accurate patient prediction. We evaluated ChatHealthAI on three clinical predictive tasks from the EHRSHOT benchmark. Results show that ChatHealthAI improves reasoning quality and interpretability while preserving competitive predictive performance. These findings highlight the potential of integrating EHR foundation models with pretrained LLMs for interpretable clinical prediction.

Bo-Hong Wang, Baicheng Peng, Ruilin Wang +3

School of Computer Science, McGill University, Montreal, QC, Canada · Mila - Quebec AI Institute, Montreal, QC, Canada

LLM Grounding LLM Alignment Clinical Outcome Prediction Clinical Reasoning Longitudinal EHR Modeling

Ask PDF