Can we generate portable representations for clinical time series data using LLMs?

Zongliang Ji; Yifei Sun; Andre Amaral; Anna Goldenberg; Rahul G. Krishnan

cs.LGMar 25, 2026

Can we generate portable representations for clinical time series data using LLMs?

Authors: Zongliang Ji, Yifei Sun, Andre Amaral, Anna Goldenberg, Rahul G. Krishnan

Organizations: University of Toronto, Canada · Vector Institute, Canada · Sunnybrook Health Sciences Centre, Canada

Abstract

Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.

Figures & tables

Figure 1: Motivation for Record2Vec. Numeric imputation loses clinical semantics and limits portability; human handoffs preserve meaning but are costly and variable. LLMs can create handoff-style summaries that retain semantics and provide portable inputs for forecasting and classification.

Figure 2: Methods to generate medical-record representations. Top to bottom: imputation pipeline; self-supervised TS representation (TSDE); TS foundation model (TimesFM); and Record2Vec: LLM summary followed by text embedding.

Dataset $\rightarrow$	HiRID					MIMIC					PPICU					Wins
Tasks $\rightarrow$	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab
Method $\downarrow$	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall
Mean	0.040	0.378	0.914	0.878	0.857	0.035	0.447	0.847	0.838	0.886	0.028	0.528	0.842	0.834	0.847	0
Right shift	0.041	0.342	0.923	0.884	0.858	0.037	0.409	0.886	0.841	0.884	0.031	0.490	0.868	0.831	0.846	1
Interpolation	0.435	0.370	0.918	0.879	0.853	0.103	0.430	0.873	0.844	0.893	0.133	0.493	0.857	0.837	0.847	0
TSDE	0.029	0.411	0.923	0.901	0.902	0.030	0.406	0.915	0.888	0.899	0.053	0.485	0.890	0.899	0.870	1

Table 1: In-distribution Result (RQ1). Best (per column) in bold ; second-best underlined . Wins counts the number of bests per method across all 15 downstream tasks.

Dataset $\rightarrow$	HiRID $\rightarrow$ PPICU					MIMIC $\rightarrow$ PPICU					Wins
Tasks $\rightarrow$	Forecast	LoS	Mort	Drug	Lab	Forecast	LoS	Mort	Drug	Lab
Method $\downarrow$	MSE	MAE	AUROC	Recall	Recall	MSE	MAE	AUROC	Recall	Recall
Mean	0.306	1.09	0.50	0.42	0.77	4.839	0.85	0.50	0.30	0.81	0
Right shift	0.417	1.03	0.50	0.53	0.88	6.344	0.76	0.50	0.23	0.82	0
Interpolation	24.80	0.91	0.50	0.34	0.81	13.41	0.71	0.50	0.21	0.80	0
TSDE	0.209	0.73	0.49	0.91	0.91	0.269	0.76	0.51	0.90	0.90	0

Table 2: Transfer Learning Result (RQ2). Best per column in bold ; second-best underlined . Wins counts tied bests.

Figure 3: Rank distributions for No-summary vs. three LLM variants across 15 in-distribution tasks (left) and 30 cross-site transfer tasks (right). Methods are ranked based on performance across five downstream tasks: Forecast (MSE), LoS (MAE), Mortality (AUROC), Drug (Recall), and Lab (Recall). See the Appendix M for the detailed values.

Figure 4: Rank distributions for four prompt variants with Gemini 2.0-flash across 15 in-distribution tasks (left) and 30 cross-site transfer tasks (right). Lower ranks indicate better performance. Methods are ranked based on performance across five downstream tasks: Forecast (MSE), LoS (MAE), Mortality (AUROC), Drug (Recall), and Lab (Recall). See the Appendix M for the detailed values.

Figure 5: Few-shot finetuning with 16 labeled target samples for mortality prediction (RQ5) shown across six transfer settings. All tasks are reported the same metric as previous section: Forecast: masked mse, LOS: mae, Mortality: AUROC. The first row is the result of Hirid $\rightarrow$ Ppicu and the second row is Mimic $\rightarrow$ Ppicu. Reference lines: blue = best in-distribution upper bound, orange = best pre-finetune result, black = best finetuned baseline. Record2Vec surpassed baselines with a large gap, reaching comparable performance to in-distribution.

Figure 6: In-distribution privacy prediction on Gender across three ICU datasets. Each bar is a different method. The dashed horizontal line in each panel marks the least leaked classical baseline among {TimesFM, TSDE, Mean, Interpolation, Right shift} for that metric/dataset. Our method reached comparable and mostly reduced results in terms of privacy leakage. Age results have less than 0.5% gap and can be inferred from Table 4

Figure 7: Information gained vs. lost with Record2Vec (heatmaps; in-distribution and cross-site transfer). Each cell shows the task-aligned relative change (%) of Record2Vec over representation vectors. Overall, Record2Vec yields the largest gains for drug , forecast , and mortality , including cross-site transfer, while demographic signals ( age , gender ) are attenuated or unchanged.

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Embedding	Silhouette
HiRID	Raw time-series	0.372
HiRID	CoT embedding	0.452
HiRID	ICD embedding	0.408
HiRID	zero_shot embedding	0.638
HiRID	Trend embedding	0.540
MIMIC	Raw time-series	0.332

Appendix

Table 3: Silhouette scores for different embeddings across datasets (higher is better).

Dataset $\rightarrow$	HiRID		MIMIC		PPICU
Tasks $\rightarrow$	Age	Gender	Age	Gender	Age	Gender
Method $\downarrow$	MAE	AUROC	MAE	AUROC	MAE	AUROC
Mean	0.805017	0.754615	0.803168	0.678225	0.76247	0.707425
Right shift	0.803392	0.742055	0.798042	0.694878	0.767785	0.726552
Interpolation	0.800455	0.726648	0.799755	0.670795	0.76095	0.720293
TSDE	0.798725	0.662075	0.797682	0.733638	0.76362	0.764377

Appendix

Table 4: RQ6 (Privacy): Age and gender prediction. Age is evaluated with MAE; Gender with AUROC.

	count	total	mean	std	max
File A ( Raw Text )	46,818	285,894,402	6,106.51	3,328.77	13,233.00
File B ( ICD Summarization )	47,119	11,027,274	234.03	60.52	567.00

Appendix

Table 5: Per-string tokenization summary (Qwen3-Embedding-8B; no special tokens).

Model	Latency (per sample)	GPU time (per batch)	Token counts	Cost (per 1k samples)
MedGemma + Qwen	$0.784+0.026\,\mathrm{s}$	$3959.41+119.61\,\mathrm{ms}$	$234.03~\text{tks/sample}$	–
Llama + Qwen	$0.25406+0.023\,\mathrm{s}$	$1299.4+117.24\,\mathrm{ms}$	$254.17~\text{tks/sample}$	–
Gemini + Qwen	$0.28+0.036\,\mathrm{s}$	–	$293.44~\text{tks/sample}$	$0.7\,\$ \text{/1k samples}$
Qwen (no summarization)	$2.9675\,\mathrm{s}$	$23608.14\,\mathrm{ms}$	$6106.56~\text{tks/sample}$	–

Appendix

Table 6: Latency, GPU time, token usage, and approximate cost of different summarization/inference pipelines.

Dataset $\rightarrow$	HiRID $\rightarrow$ PPICU			MIMIC $\rightarrow$ PPICU			Wins
Task $\rightarrow$	Forecast	LoS	Mortality	Forecast	LoS	Mortality
Method $\downarrow$	MSE $\downarrow$	MAE $\downarrow$	AUROC $\uparrow$	MSE $\downarrow$	MAE $\downarrow$	AUROC $\uparrow$
Mean	$0.133\pm 0.001$	$2.678\pm 0.229$	$0.600\pm 0.007$	$0.183\pm 0.003$	$1.171\pm 0.284$	$0.565\pm 0.050$	0
Right shift	$0.136\pm 0.002$	$1.803\pm 0.041$	$0.621\pm 0.036$	$0.181\pm 0.003$	$0.662\pm 0.056$	$0.606\pm 0.036$	0
Interpolation	$0.139\pm 0.001$	$1.999\pm 0.123$	$0.605\pm 0.033$	$0.183\pm 0.003$	$0.972\pm 0.090$	$0.563\pm 0.028$	0
TFM	$0.144\pm 0.005$	$0.896\pm 0.042$	$0.686\pm 0.003$	$0.163\pm 0.001$	$1.067\pm 0.028$	$0.683\pm 0.002$	0

Appendix

Table 7: Transfer performance from HiRID/MIMIC to PPICU. Values are mean

\pm

std. Best per column in bold ; second-best underlined . Wins count the number of best results per method.

Task	Small PPICU	Few-shot Finetune (Record2Vec)
Mortality	0.6843	0.72
LoS	1.03	0.49
Forecast	0.62	0.183
Feature	0.901	0.97
Lab	0.94	0.97

Appendix

Table 8: Performance comparison between models trained on a small PPICU subset vs. few-shot finetuned models.

	LSTM	GRU	Transformer	TCN
H $\rightarrow$ P Baseline	0.640	0.687	0.579	0.719
H $\rightarrow$ P Record2Vec	0.69788	0.6838	0.679	0.684
M $\rightarrow$ P Baseline	0.5235	0.5476	0.564	0.719
M $\rightarrow$ P Record2Vec	0.722	0.685	0.682	0.695

Appendix

Table 9: Model performance for HiRID

\rightarrow

PPICU (H

\rightarrow

P) and MIMIC

\rightarrow

PPICU (M

\rightarrow

P), comparing baseline vs. Record2Vec representations.

Ablations	Forecast	LoS	Mort	Drug	Lab
Qwen3-mean-l2	0.021	0.347	0.90	0.911	0.931
Qwen3-mean-none	0.027	0.371	0.88	0.871	0.919
Qwen3-cls-l2	0.024	0.530	0.89	0.900	0.940
Qwen3-last-l2	0.022	0.423	0.89	0.920	0.923
gte-Qwen2-instruct	0.028	0.480	0.88	0.886	0.918
llama-embed-nemotron	0.021	0.379	0.89	0.9097	0.929

Appendix

Table 10: Ablations on PPICU (in-distribution).

Ablations	Forecast	LoS	Mort	Drug	Lab
Qwen3-mean-l2	0.183	0.69	0.72	0.97	0.97
Qwen3-mean-none	0.209	0.66	0.71	0.94	0.96
Qwen3-cls-l2	0.230	0.78	0.50	0.995	0.98
Qwen3-last-l2	0.190	0.78	0.73	0.94	0.96
gte-Qwen2-instruct	0.250	0.82	0.50	0.93	0.96
llama-embed-nemotron	0.190	0.72	0.713	0.96	0.998

Appendix

Table 11: Ablations for HiRID

\rightarrow

PPICU transfer.

	HIRID					MIMIC					PPICU
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.021	0.3538	0.9	0.899	0.931	0.027	0.328	0.82	0.886	0.947	0.0217	0.376	0.64	0.9308	0.923
llama 3.1	0.0214	0.381	0.8797	0.899	0.925	0.0288	0.406	0.8131	0.886	0.938	0.029	0.3752	0.63	0.937	0.9271
medgemma	0.0237	0.3682	0.83	0.911	0.925	0.0299	0.4005	0.7788	0.8946	0.9432	0.029	0.3626	0.6346	0.925	0.936
gemini	0.028	0.347	0.8302	0.9064	0.9307	0.03	0.3328	0.77	0.903	0.942	0.017	0.358	0.6377	0.9308	0.9328

Appendix

Table 12: Summarization methods performance comparison across 15 in-distribution tasks (Figure 3 left subfigure)

	H $\rightarrow$ P					M $\rightarrow$ P					H $\rightarrow$ M
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.21	0.98	0.66	0.92	0.96	0.263	0.77	0.7	0.92	0.92	0.1284	0.693	0.73	0.881	0.832
llama 3.1	0.1908	0.71	0.735	0.9243	0.9718	0.249	0.5522	0.7097	0.9415	0.9285	0.134	0.6464	0.82	0.878	0.851
medgemma	0.21	0.6981	0.74	0.97	0.98	0.239	0.5627	0.7145	0.9428	0.95	0.0922	0.537	0.81	0.891	0.83
gemini	0.183	0.7017	0.7001	0.92	0.9737	0.258	0.49	0.73	0.95	0.9372	0.089	0.6661	0.8044	0.852	0.842

Appendix

Table 13: Summarization methods performance comparison across 30 transfer learning tasks (Part 1: Transfer Pairs 1-3)

	P $\rightarrow$ M					M $\rightarrow$ H					P $\rightarrow$ H
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
no-summary	0.15971	0.518	0.75	0.913	0.8669	0.135	0.5805	0.8277	0.816	0.8857	0.0954	0.494	0.76	0.892	0.7895
llama 3.1	0.141	0.443	0.752	0.877	0.8669	0.1741	0.572	0.83	0.903	0.9085	0.096	0.4393	0.7657	0.835	0.734
medgemma	0.1281	0.4493	0.81	0.88	0.9054	0.135	0.5805	0.8277	0.8653	0.877	0.089	0.4682	0.7725	0.8637	0.8042
gemini	0.127	0.4886	0.7942	0.901	0.919	0.167	0.5864	0.8037	0.8239	0.916	0.083	0.429	0.81	0.8627	0.812

Appendix

Table 14: Summarization methods performance comparison across 30 transfer learning tasks (Part 2: Transfer Pairs 4-6)

	HIRID					MIMIC					PPICU
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
zero-shot	0.0244	0.354	0.8501	0.911	0.931	0.0282	0.3951	0.77	0.8894	0.9431	0.017	0.376	0.6367	0.9346	0.923
CoT	0.0268	0.381	0.8311	0.9078	0.925	0.0294	0.328	0.8101	0.886	0.938	0.029	0.358	0.6387	0.937	0.936
ICD	0.021	0.3517	0.83	0.899	0.9305	0.027	0.3844	0.8018	0.903	0.947	0.0284	0.3645	0.64	0.925	0.9234
Trend	0.028	0.347	0.9	0.9042	0.9263	0.03	0.406	0.82	0.8881	0.9469	0.0249	0.3713	0.63	0.9324	0.9308

Appendix

Table 15: Prompting methods performance comparison across 15 in-distribution tasks (Figure 4 left subfigure)

	H $\rightarrow$ P					M $\rightarrow$ P					H $\rightarrow$ M
Model	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab	Forecast	LOS	Mort	Drug	Lab
zero-shot	0.183	0.6994	0.6888	0.9359	0.9758	0.2536	0.57	0.7045	0.949	0.95	0.1082	0.6467	0.73	0.8878	0.83
CoT	0.1996	0.69	0.66	0.92	0.9667	0.2461	0.49	0.7106	0.92	0.9375	0.089	0.693	0.7328	0.868	0.832
ICD	0.21	0.6989	0.6916	0.933	0.98	0.239	0.5199	0.73	0.926	0.9498	0.134	0.6564	0.82	0.891	0.851
Trend	0.1927	0.71	0.74	0.97	0.96	0.258	0.5595	0.7	0.95	0.92 — 0.1248	0.537	0.7629	0.852	0.847

Appendix

Table 16: Prompting methods performance comparison across 15 transfer learning tasks (Figure 4 right subfigure)

Dataset	Lab features	Vital features	Binary features
HiRID	ALP, ALT, AST, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Chloride, CreatineKinase, Creatinine, Glucose, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, AirwayPressurePeak, DiastolicBloodPressure, FiO2, FluidBalance, GCS, HeartRate, MeanBloodPressure, PEEP, RespiratoryRate, SAS, Saturation, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput, ICPMonitor, Ventilation	Analgesia, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Aspirin, CaReplacement, Dialysis, Diuretics, ICPMonitor, Insulin, KReplacement, LiverToxicDrug, MgReplacement, Neuroleptics, Paralysis, Saline, Sedation, Steroids, TPN, Transfusions, Vasopressors, Ventilation
MIMIC	ALP, ALT, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Creatinine, CreatinineKinase, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, DiastolicBloodPressure, FiO2, GCS, HeartRate, ICDSC, MeanBloodPressure, MinuteVentilation, PEEP, RespiratoryRate, SAS, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput	Analgesia, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Antipsychotics, CTScan, CaReplacement, Dialysis, Diuretics, EnteralNutrition, ICPMonitor, KReplacement, MRI, MgReplacement, PPI, Paralysis, TPN, Transfusions, UltraSound, Vasopressors, Ventilation, Xray
PPICU	ALP, ALT, AST, Bicarbonate, Bilirubin, BloodUreaNitrogen, Calcium, Chloride, Creatinine, CreatinineKinase, GGT, Glucose, Hemoglobin, INR, Lactate, Magnesium, PaCO2, PaO2, Phosphate, Platelets, Potassium, Sodium, Troponin, WBC, ph	AirwayPressure, AirwayPressureIP, AirwayPressurePeak, DiastolicBloodPressure, FiO2, FluidInput, FluidOutput, GCS, HeartRate, ICDSC, MeanBloodPressure, MinuteVentilation, PAVSupport, PC, PEEP, PlateauPressure, RespiratoryRate, SAS, Saturation, SystolicBloodPressure, Temperature, TidalVolume, UrineOutput	Analgesia, Antiarrhythmic, Antiarrhythmics, Antibiotics, Anticoagulants, Antiepileptics, Antihypertensives, Aspirin, Dialysis, Diuretics, EKG, EVD, ICPMonitor, InhaledVasodilator, Insulin, Isoproterenol, MgReplacement, Neuroleptics, OsmoticTherapy, PPI, PS, Paralysis, Sedation, Steroids, TPN, Transfusions, Vasopressors

Appendix

Table 17: Feature groupings for converting ICU data into hierarchical textual representations. Each feature is prefixed with its group label (e.g., lab_Hemoglobin , vital_HeartRate , bin_Vasopressors ) before tokenization.

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.799\,\pm\,4.12\mathrm{e}-4$	$1\,\pm\,1.893\mathrm{e}-4$	$0.802\,\pm\,2.517\mathrm{e}-5$	$1\,\pm\,1.732\mathrm{e}-5$	$0.808\,\pm\,1.323\mathrm{e}-3$	$1\,\pm\,3.87\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.799\,\pm\,5.4\mathrm{e}-4$	$1\,\pm\,1.888\mathrm{e}-4$	$0.802\,\pm\,6.557\mathrm{e}-5$	$1\,\pm\,7.572\mathrm{e}-5$	$0.811\,\pm\,1.514\mathrm{e}-3$	$1.001\,\pm\,1.177\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.808\,\pm\,1.848\mathrm{e}-3$	$1.018\,\pm\,3.714\mathrm{e}-3$	$0.806\,\pm\,1.544\mathrm{e}-3$	$1.01\,\pm\,4.083\mathrm{e}-3$	$0.818\,\pm\,5.124\mathrm{e}-3$	$1.015\,\pm\,7.344\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.8\,\pm\,1.198\mathrm{e}-3$	$1\,\pm\,2.955\mathrm{e}-4$	$0.802\,\pm\,1.229\mathrm{e}-4$	$1\,\pm\,1.528\mathrm{e}-5$	$0.809\,\pm\,2.309\mathrm{e}-5$	$1\,\pm\,1\mathrm{e}-5$
Llama-3.1 CoT lstm	$0.799\,\pm\,4.594\mathrm{e}-4$	$1\,\pm\,1.09\mathrm{e}-4$	$0.802\,\pm\,8.139\mathrm{e}-5$	$1\,\pm\,3.948\mathrm{e}-5$	$0.808\,\pm\,7.551\mathrm{e}-4$	$1\,\pm\,1.603\mathrm{e}-4$

Appendix

Table 18: In-distribution results - Age. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
No_sum_Trend lstm	$0.799\,\pm\,1.177\mathrm{e}-3$	$1\,\pm\,2.004\mathrm{e}-4$	$0.802\,\pm\,5.188\mathrm{e}-5$	$1\,\pm\,1.5\mathrm{e}-5$	$0.808\,\pm\,6.139\mathrm{e}-4$	$1\,\pm\,2.061\mathrm{e}-4$
No_sum_Trend mlp	$0.801\,\pm\,1.357\mathrm{e}-3$	$1.001\,\pm\,6.168\mathrm{e}-4$	$0.802\,\pm\,2.016\mathrm{e}-4$	$1\,\pm\,1.005\mathrm{e}-4$	$0.81\,\pm\,1.434\mathrm{e}-3$	$1\,\pm\,5.804\mathrm{e}-4$
No_sum_Trend patchtsmixer	$0.803\,\pm\,2.482\mathrm{e}-3$	$1.006\,\pm\,2.681\mathrm{e}-3$	$0.805\,\pm\,1.96\mathrm{e}-3$	$1.004\,\pm\,7.381\mathrm{e}-4$	$0.812\,\pm\,1.637\mathrm{e}-3$	$1.005\,\pm\,1.892\mathrm{e}-3$
No_sum_Trend timemixer	$0.8\,\pm\,6.989\mathrm{e}-4$	$1\,\pm\,1.409\mathrm{e}-4$	$0.802\,\pm\,4.933\mathrm{e}-5$	$1\,\pm\,5.774\mathrm{e}-6$	$0.809\,\pm\,1.931\mathrm{e}-4$	$1\,\pm\,5.888\mathrm{e}-5$
medgemma CoT +TFM lstm	$0.798\,\pm\,3.301\mathrm{e}-4$	$1\,\pm\,5.014\mathrm{e}-4$	$0.802\,\pm\,1.617\mathrm{e}-4$	$0.999\,\pm\,1.401\mathrm{e}-4$	$0.808\,\pm\,1.51\mathrm{e}-4$	$1\,\pm\,2.768\mathrm{e}-4$

Appendix

Table 19: In-distribution results - Age. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.808\,\pm\,1.305\mathrm{e}-3$	$1\,\pm\,4.309\mathrm{e}-4$	$0.806\,\pm\,1.069\mathrm{e}-4$	$1\,\pm\,6.028\mathrm{e}-5$
Llama-3.1 CoT +TFM mlp	$0.809\,\pm\,7.101\mathrm{e}-4$	$1.001\,\pm\,6.509\mathrm{e}-4$	$0.806\,\pm\,1.457\mathrm{e}-4$	$1\,\pm\,3.215\mathrm{e}-5$
Llama-3.1 CoT +TFM patchtsmixer	$0.82\,\pm\,2.642\mathrm{e}-3$	$1.019\,\pm\,4.362\mathrm{e}-3$	$0.81\,\pm\,2.535\mathrm{e}-3$	$1.01\,\pm\,6.56\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.809\,\pm\,1.014\mathrm{e}-3$	$1\,\pm\,2.955\mathrm{e}-4$	$0.806\,\pm\,1.986\mathrm{e}-4$	$1\,\pm\,1\mathrm{e}-5$
Llama-3.1 CoT lstm	$0.809\,\pm\,2.399\mathrm{e}-4$	$1\,\pm\,9.274\mathrm{e}-5$	$0.806\,\pm\,1.097\mathrm{e}-4$	$1\,\pm\,2.944\mathrm{e}-5$

Appendix

Table 20: Cross-site transfer results - Age. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
No_sum_Trend lstm	$0.808\,\pm\,1.044\mathrm{e}-3$	$1\,\pm\,3.807\mathrm{e}-4$	$0.806\,\pm\,1.065\mathrm{e}-4$	$1\,\pm\,1.88\mathrm{e}-4$
No_sum_Trend mlp	$0.81\,\pm\,1.264\mathrm{e}-3$	$1\,\pm\,7.27\mathrm{e}-4$	$0.806\,\pm\,3.889\mathrm{e}-4$	$1\,\pm\,2.887\mathrm{e}-5$
No_sum_Trend patchtsmixer	$0.814\,\pm\,3.215\mathrm{e}-3$	$1.007\,\pm\,3.595\mathrm{e}-3$	$0.808\,\pm\,4.422\mathrm{e}-3$	$1.005\,\pm\,3.252\mathrm{e}-3$
No_sum_Trend timemixer	$0.809\,\pm\,5.934\mathrm{e}-4$	$1\,\pm\,1.409\mathrm{e}-4$	$0.806\,\pm\,7.5\mathrm{e}-5$	$1\,\pm\,5\mathrm{e}-6$
medgemma CoT +TFM lstm	$0.807\,\pm\,6.886\mathrm{e}-4$	$1\,\pm\,3.897\mathrm{e}-4$	$0.806\,\pm\,2.281\mathrm{e}-4$	$1\,\pm\,1.756\mathrm{e}-4$

Appendix

Table 21: Cross-site transfer results - Age. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.682\,\pm\,8.894\mathrm{e}-3$	$0.555\,\pm\,1.246\mathrm{e}-2$	$0.655\,\pm\,1.33\mathrm{e}-2$	$0.585\,\pm\,1.425\mathrm{e}-2$	$0.693\,\pm\,7.3\mathrm{e}-3$	$0.573\,\pm\,1.212\mathrm{e}-2$
Llama-3.1 CoT +TFM mlp	$0.72\,\pm\,7.983\mathrm{e}-3$	$0.6\,\pm\,1.015\mathrm{e}-2$	$0.705\,\pm\,1.309\mathrm{e}-2$	$0.64\,\pm\,1.339\mathrm{e}-2$	$0.725\,\pm\,1.677\mathrm{e}-2$	$0.608\,\pm\,2.07\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.704\,\pm\,3.667\mathrm{e}-4$	$0.58\,\pm\,1.341\mathrm{e}-3$	$0.702\,\pm\,8.468\mathrm{e}-4$	$0.63\,\pm\,2\mathrm{e}-4$	$0.714\,\pm\,5.538\mathrm{e}-3$	$0.595\,\pm\,6.329\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.707\,\pm\,2.153\mathrm{e}-3$	$0.588\,\pm\,2.084\mathrm{e}-3$	$0.696\,\pm\,1.456\mathrm{e}-2$	$0.631\,\pm\,1.694\mathrm{e}-2$	$0.7\,\pm\,6.089\mathrm{e}-3$	$0.584\,\pm\,5.492\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.702\,\pm\,3.827\mathrm{e}-3$	$0.579\,\pm\,5.909\mathrm{e}-3$	$0.685\,\pm\,2.532\mathrm{e}-2$	$0.612\,\pm\,2.763\mathrm{e}-2$	$0.696\,\pm\,6.709\mathrm{e}-3$	$0.573\,\pm\,1.042\mathrm{e}-2$

Appendix

Table 22: In-distribution results - Gender. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
No_sum_Trend lstm	$0.74\,\pm\,2.23\mathrm{e}-3$	$0.63\,\pm\,3.426\mathrm{e}-3$	$0.722\,\pm\,1.427\mathrm{e}-2$	$0.663\,\pm\,1.613\mathrm{e}-2$	$0.778\,\pm\,2.76\mathrm{e}-3$	$0.681\,\pm\,3.846\mathrm{e}-3$
No_sum_Trend mlp	$0.755\,\pm\,3.027\mathrm{e}-3$	$0.641\,\pm\,3.93\mathrm{e}-3$	$0.753\,\pm\,1.008\mathrm{e}-2$	$0.698\,\pm\,1.139\mathrm{e}-2$	$0.787\,\pm\,9.173\mathrm{e}-3$	$0.686\,\pm\,1.275\mathrm{e}-2$
No_sum_Trend patchtsmixer	$0.749\,\pm\,4.704\mathrm{e}-3$	$0.631\,\pm\,5.715\mathrm{e}-3$	$0.754\,\pm\,6.242\mathrm{e}-3$	$0.695\,\pm\,7.373\mathrm{e}-3$	$0.79\,\pm\,1.108\mathrm{e}-2$	$0.684\,\pm\,1.554\mathrm{e}-2$
No_sum_Trend timemixer	$0.751\,\pm\,2.802\mathrm{e}-3$	$0.632\,\pm\,3.398\mathrm{e}-3$	$0.736\,\pm\,1.755\mathrm{e}-2$	$0.675\,\pm\,1.889\mathrm{e}-2$	$0.78\,\pm\,2.308\mathrm{e}-3$	$0.674\,\pm\,2.988\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.701\,\pm\,9.768\mathrm{e}-3$	$0.582\,\pm\,1.444\mathrm{e}-2$	$0.659\,\pm\,4.775\mathrm{e}-3$	$0.598\,\pm\,4.392\mathrm{e}-3$	$0.724\,\pm\,1.238\mathrm{e}-2$	$0.615\,\pm\,2.059\mathrm{e}-2$

Appendix

Table 23: In-distribution results - Gender. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.664\,\pm\,6.426\mathrm{e}-3$	$0.528\,\pm\,5.865\mathrm{e}-3$	$0.673\,\pm\,2.82\mathrm{e}-3$	$0.539\,\pm\,4.609\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.673\,\pm\,3.117\mathrm{e}-3$	$0.535\,\pm\,3.658\mathrm{e}-3$	$0.674\,\pm\,2.093\mathrm{e}-3$	$0.54\,\pm\,4.741\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.68\,\pm\,1.007\mathrm{e}-3$	$0.548\,\pm\,1.27\mathrm{e}-3$	$0.678\,\pm\,4.151\mathrm{e}-4$	$0.546\,\pm\,1.174\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.677\,\pm\,3.143\mathrm{e}-3$	$0.544\,\pm\,6.236\mathrm{e}-3$	$0.675\,\pm\,1.276\mathrm{e}-3$	$0.542\,\pm\,1.627\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.674\,\pm\,5.913\mathrm{e}-4$	$0.542\,\pm\,2.421\mathrm{e}-3$	$0.675\,\pm\,3.617\mathrm{e}-4$	$0.543\,\pm\,8.516\mathrm{e}-4$

Appendix

Table 24: Cross-site transfer results - Gender. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
No_sum_Trend lstm	$0.686\,\pm\,2.715\mathrm{e}-3$	$0.561\,\pm\,3.173\mathrm{e}-3$	$0.712\,\pm\,2.536\mathrm{e}-3$	$0.588\,\pm\,2.901\mathrm{e}-3$
No_sum_Trend mlp	$0.687\,\pm\,1.638\mathrm{e}-3$	$0.563\,\pm\,1.828\mathrm{e}-3$	$0.707\,\pm\,3.987\mathrm{e}-3$	$0.588\,\pm\,3.721\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.691\,\pm\,1.504\mathrm{e}-3$	$0.566\,\pm\,1.569\mathrm{e}-3$	$0.702\,\pm\,2.505\mathrm{e}-3$	$0.584\,\pm\,2.975\mathrm{e}-3$
No_sum_Trend timemixer	$0.694\,\pm\,1.373\mathrm{e}-3$	$0.568\,\pm\,1.335\mathrm{e}-3$	$0.702\,\pm\,1.888\mathrm{e}-3$	$0.582\,\pm\,3.587\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.674\,\pm\,3.333\mathrm{e}-3$	$0.542\,\pm\,4.159\mathrm{e}-3$	$0.68\,\pm\,9.923\mathrm{e}-4$	$0.55\,\pm\,1.624\mathrm{e}-3$

Appendix

Table 25: Cross-site transfer results - Gender. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.759\,\pm\,1.504\mathrm{e}-3$	$0.848\,\pm\,5.911\mathrm{e}-3$	$0.76\,\pm\,4.839\mathrm{e}-3$	$0.836\,\pm\,2.494\mathrm{e}-3$	$0.808\,\pm\,2.843\mathrm{e}-3$	$0.917\,\pm\,5.773\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.771\,\pm\,2.913\mathrm{e}-3$	$0.847\,\pm\,3.027\mathrm{e}-3$	$0.77\,\pm\,7.323\mathrm{e}-3$	$0.831\,\pm\,4.135\mathrm{e}-3$	$0.814\,\pm\,3.384\mathrm{e}-3$	$0.915\,\pm\,2.589\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.736\,\pm\,1.973\mathrm{e}-2$	$0.859\,\pm\,1.498\mathrm{e}-2$	$0.753\,\pm\,7.457\mathrm{e}-3$	$0.82\,\pm\,6.964\mathrm{e}-3$	$0.798\,\pm\,8.902\mathrm{e}-3$	$0.921\,\pm\,4.925\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.747\,\pm\,6.439\mathrm{e}-3$	$0.854\,\pm\,1.833\mathrm{e}-3$	$0.737\,\pm\,1.381\mathrm{e}-2$	$0.811\,\pm\,1.056\mathrm{e}-2$	$0.805\,\pm\,8.986\mathrm{e}-3$	$0.925\,\pm\,2.113\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.733\,\pm\,9.055\mathrm{e}-3$	$0.855\,\pm\,3.436\mathrm{e}-3$	$0.692\,\pm\,6.113\mathrm{e}-3$	$0.815\,\pm\,4.863\mathrm{e}-3$	$0.793\,\pm\,3.488\mathrm{e}-3$	$0.935\,\pm\,4.021\mathrm{e}-3$

Appendix

Table 26: In-distribution results - Drug. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
medgemma CoT +TFM patchtsmixer	$0.816\,\pm\,1.606\mathrm{e}-2$	$0.876\,\pm\,6.866\mathrm{e}-3$	$0.811\,\pm\,6.325\mathrm{e}-3$	$0.867\,\pm\,3.918\mathrm{e}-3$	$0.833\,\pm\,1.357\mathrm{e}-2$	$0.912\,\pm\,5.332\mathrm{e}-3$
medgemma CoT +TFM timemixer	$0.824\,\pm\,4.624\mathrm{e}-3$	$0.887\,\pm\,2.053\mathrm{e}-3$	$0.805\,\pm\,6.915\mathrm{e}-3$	$0.873\,\pm\,6.623\mathrm{e}-3$	$0.841\,\pm\,8.596\mathrm{e}-3$	$0.904\,\pm\,7.508\mathrm{e}-3$
medgemma CoT lstm	$0.821\,\pm\,8.278\mathrm{e}-3$	$0.88\,\pm\,5.981\mathrm{e}-3$	$0.787\,\pm\,1.078\mathrm{e}-2$	$0.855\,\pm\,2.705\mathrm{e}-3$	$0.84\,\pm\,7.288\mathrm{e}-3$	$0.919\,\pm\,4.913\mathrm{e}-3$
medgemma CoT mlp	$0.824\,\pm\,7.467\mathrm{e}-3$	$0.884\,\pm\,2.531\mathrm{e}-3$	$0.788\,\pm\,1.038\mathrm{e}-2$	$0.849\,\pm\,4.009\mathrm{e}-3$	$0.841\,\pm\,6.244\mathrm{e}-3$	$0.918\,\pm\,3.274\mathrm{e}-3$
medgemma CoT patchtsmixer	$0.816\,\pm\,6.976\mathrm{e}-3$	$0.882\,\pm\,1.833\mathrm{e}-3$	$0.773\,\pm\,8.922\mathrm{e}-3$	$0.848\,\pm\,5.822\mathrm{e}-3$	$0.828\,\pm\,4.332\mathrm{e}-3$	$0.919\,\pm\,3.551\mathrm{e}-3$

Appendix

Table 27: In-distribution results - Drug. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.769\,\pm\,1.565\mathrm{e}-3$	$0.931\,\pm\,1.886\mathrm{e}-3$	$0.777\,\pm\,2.04\mathrm{e}-3$	$0.931\,\pm\,1.593\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.763\,\pm\,1.471\mathrm{e}-3$	$0.933\,\pm\,1.939\mathrm{e}-3$	$0.774\,\pm\,1.628\mathrm{e}-3$	$0.938\,\pm\,4.576\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.75\,\pm\,2.781\mathrm{e}-3$	$0.945\,\pm\,4.113\mathrm{e}-3$	$0.767\,\pm\,3.535\mathrm{e}-3$	$0.932\,\pm\,1.907\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.764\,\pm\,3.39\mathrm{e}-3$	$0.937\,\pm\,5.739\mathrm{e}-3$	$0.772\,\pm\,1.664\mathrm{e}-3$	$0.936\,\pm\,6.964\mathrm{e}-4$
Llama-3.1 CoT lstm	$0.761\,\pm\,3.771\mathrm{e}-3$	$0.944\,\pm\,3.279\mathrm{e}-3$	$0.766\,\pm\,4.793\mathrm{e}-4$	$0.95\,\pm\,7.115\mathrm{e}-3$

Appendix

Table 28: Cross-site transfer results - Drug. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
medgemma CoT +TFM patchtsmixer	$0.772\,\pm\,2.573\mathrm{e}-3$	$0.907\,\pm\,3.85\mathrm{e}-3$	$0.784\,\pm\,9.672\mathrm{e}-4$	$0.916\,\pm\,9.366\mathrm{e}-4$
medgemma CoT +TFM timemixer	$0.783\,\pm\,1.245\mathrm{e}-3$	$0.912\,\pm\,4.037\mathrm{e}-3$	$0.793\,\pm\,2.516\mathrm{e}-4$	$0.91\,\pm\,2.613\mathrm{e}-3$
medgemma CoT lstm	$0.788\,\pm\,2.26\mathrm{e}-3$	$0.924\,\pm\,1.781\mathrm{e}-3$	$0.797\,\pm\,6.506\mathrm{e}-4$	$0.923\,\pm\,5.205\mathrm{e}-3$
medgemma CoT mlp	$0.79\,\pm\,6.558\mathrm{e}-4$	$0.924\,\pm\,3.799\mathrm{e}-3$	$0.795\,\pm\,6.553\mathrm{e}-4$	$0.926\,\pm\,7.525\mathrm{e}-4$
medgemma CoT patchtsmixer	$0.791\,\pm\,1.586\mathrm{e}-3$	$0.922\,\pm\,8.402\mathrm{e}-4$	$0.793\,\pm\,1.714\mathrm{e}-3$	$0.924\,\pm\,3.783\mathrm{e}-3$

Appendix

Table 29: Cross-site transfer results - Drug. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.872\,\pm\,2.001\mathrm{e}-3$	$0.92\,\pm\,8.798\mathrm{e}-4$	$0.895\,\pm\,2.53\mathrm{e}-3$	$0.936\,\pm\,1.722\mathrm{e}-3$	$0.847\,\pm\,7.912\mathrm{e}-4$	$0.917\,\pm\,6.809\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.881\,\pm\,2.75\mathrm{e}-3$	$0.926\,\pm\,8.832\mathrm{e}-4$	$0.903\,\pm\,1.691\mathrm{e}-3$	$0.94\,\pm\,4.133\mathrm{e}-4$	$0.857\,\pm\,1.235\mathrm{e}-3$	$0.925\,\pm\,6.144\mathrm{e}-4$
Llama-3.1 CoT +TFM patchtsmixer	$0.857\,\pm\,6.165\mathrm{e}-3$	$0.919\,\pm\,5.505\mathrm{e}-4$	$0.892\,\pm\,1.797\mathrm{e}-3$	$0.938\,\pm\,2.665\mathrm{e}-4$	$0.839\,\pm\,7.46\mathrm{e}-3$	$0.922\,\pm\,1.361\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.856\,\pm\,6.02\mathrm{e}-3$	$0.923\,\pm\,1.165\mathrm{e}-3$	$0.87\,\pm\,1.593\mathrm{e}-3$	$0.935\,\pm\,8.472\mathrm{e}-4$	$0.811\,\pm\,3.494\mathrm{e}-3$	$0.928\,\pm\,1.361\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.846\,\pm\,0\mathrm{e}0$	$0.928\,\pm\,0\mathrm{e}0$	$0.881\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$	$0.831\,\pm\,0\mathrm{e}0$	$0.931\,\pm\,0\mathrm{e}0$

Appendix

Table 30: In-distribution results - Lab. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro	f1_micro	recall_micro
No_sum_Trend lstm	$0.878\,\pm\,2.658\mathrm{e}-4$	$0.928\,\pm\,6.463\mathrm{e}-4$	$0.906\,\pm\,2.312\mathrm{e}-4$	$0.943\,\pm\,6.671\mathrm{e}-4$	$0.875\,\pm\,1.3\mathrm{e}-4$	$0.932\,\pm\,1.242\mathrm{e}-4$
No_sum_Trend mlp	$0.895\,\pm\,0\mathrm{e}0$	$0.933\,\pm\,0\mathrm{e}0$	$0.914\,\pm\,0\mathrm{e}0$	$0.942\,\pm\,0\mathrm{e}0$	$0.883\,\pm\,0\mathrm{e}0$	$0.933\,\pm\,0\mathrm{e}0$
No_sum_Trend patchtsmixer	$0.879\,\pm\,7.515\mathrm{e}-4$	$0.929\,\pm\,1.331\mathrm{e}-3$	$0.905\,\pm\,2.403\mathrm{e}-4$	$0.942\,\pm\,7.504\mathrm{e}-4$	$0.875\,\pm\,1.472\mathrm{e}-3$	$0.936\,\pm\,2.444\mathrm{e}-4$
No_sum_Trend timemixer	$0.873\,\pm\,3.395\mathrm{e}-4$	$0.93\,\pm\,4.888\mathrm{e}-4$	$0.896\,\pm\,8.228\mathrm{e}-4$	$0.942\,\pm\,6.2\mathrm{e}-4$	$0.864\,\pm\,3.47\mathrm{e}-4$	$0.926\,\pm\,1.854\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.889\,\pm\,5.13\mathrm{e}-3$	$0.926\,\pm\,4.828\mathrm{e}-4$	$0.904\,\pm\,2.248\mathrm{e}-3$	$0.936\,\pm\,1.293\mathrm{e}-3$	$0.861\,\pm\,2.565\mathrm{e}-3$	$0.916\,\pm\,1.212\mathrm{e}-3$

Appendix

Table 31: In-distribution results - Lab. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
Llama-3.1 CoT +TFM lstm	$0.774\,\pm\,6.907\mathrm{e}-4$	$0.959\,\pm\,3.078\mathrm{e}-3$	$0.766\,\pm\,1.933\mathrm{e}-3$	$0.95\,\pm\,5.956\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.78\,\pm\,2.159\mathrm{e}-3$	$0.956\,\pm\,2.175\mathrm{e}-3$	$0.761\,\pm\,1.398\mathrm{e}-3$	$0.956\,\pm\,1.442\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.777\,\pm\,2.362\mathrm{e}-3$	$0.959\,\pm\,4.087\mathrm{e}-3$	$0.755\,\pm\,1.652\mathrm{e}-3$	$0.961\,\pm\,2.694\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.773\,\pm\,1.304\mathrm{e}-3$	$0.962\,\pm\,2.365\mathrm{e}-3$	$0.775\,\pm\,3.476\mathrm{e}-3$	$0.94\,\pm\,5.293\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.786\,\pm\,0\mathrm{e}0$	$0.954\,\pm\,0\mathrm{e}0$	$0.775\,\pm\,0\mathrm{e}0$	$0.952\,\pm\,0\mathrm{e}0$

Appendix

Table 32: Cross-site transfer results - Lab. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	f1_micro	recall_micro	f1_micro	recall_micro
No_sum_Trend lstm	$0.805\,\pm\,8.298\mathrm{e}-4$	$0.944\,\pm\,1.776\mathrm{e}-3$	$0.78\,\pm\,3.98\mathrm{e}-4$	$0.953\,\pm\,5.442\mathrm{e}-4$
No_sum_Trend mlp	$0.8\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$	$0.779\,\pm\,0\mathrm{e}0$	$0.946\,\pm\,0\mathrm{e}0$
No_sum_Trend patchtsmixer	$0.807\,\pm\,7.472\mathrm{e}-4$	$0.95\,\pm\,9.666\mathrm{e}-4$	$0.779\,\pm\,2.562\mathrm{e}-3$	$0.948\,\pm\,1.084\mathrm{e}-3$
No_sum_Trend timemixer	$0.808\,\pm\,1.079\mathrm{e}-3$	$0.941\,\pm\,4.553\mathrm{e}-4$	$0.789\,\pm\,7.168\mathrm{e}-4$	$0.937\,\pm\,1.581\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.779\,\pm\,3.345\mathrm{e}-3$	$0.955\,\pm\,5.752\mathrm{e}-3$	$0.775\,\pm\,1.652\mathrm{e}-3$	$0.94\,\pm\,2.151\mathrm{e}-3$

Appendix

Table 33: Cross-site transfer results - Lab. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse	masked_mae	masked_mse
Llama-3.1 CoT +TFM lstm	$0.073\,\pm\,1.4\mathrm{e}-3$	$0.014\,\pm\,3.704\mathrm{e}-4$	$0.093\,\pm\,8.415\mathrm{e}-4$	$0.018\,\pm\,2.152\mathrm{e}-4$	$0.074\,\pm\,8.799\mathrm{e}-4$	$0.015\,\pm\,2.152\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.078\,\pm\,2.696\mathrm{e}-4$	$0.013\,\pm\,7.234\mathrm{e}-5$	$0.096\,\pm\,3.704\mathrm{e}-4$	$0.018\,\pm\,1.193\mathrm{e}-4$	$0.08\,\pm\,8.415\mathrm{e}-4$	$0.015\,\pm\,2.801\mathrm{e}-4$
Llama-3.1 CoT +TFM patchtsmixer	$0.125\,\pm\,7.892\mathrm{e}-4$	$0.028\,\pm\,3.819\mathrm{e}-4$	$0.139\,\pm\,9.019\mathrm{e}-4$	$0.033\,\pm\,4.1\mathrm{e}-4$	$0.455\,\pm\,7.315\mathrm{e}-2$	$0.339\,\pm\,9.426\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.077\,\pm\,7.422\mathrm{e}-4$	$0.013\,\pm\,2.196\mathrm{e}-4$	$0.094\,\pm\,3.646\mathrm{e}-4$	$0.018\,\pm\,2.452\mathrm{e}-4$	$0.109\,\pm\,6.269\mathrm{e}-4$	$0.021\,\pm\,2.065\mathrm{e}-4$
Llama-3.1 CoT lstm	$0.075\,\pm\,4.255\mathrm{e}-4$	$0.015\,\pm\,1.569\mathrm{e}-4$	$0.094\,\pm\,4.84\mathrm{e}-4$	$0.019\,\pm\,1.787\mathrm{e}-4$	$0.077\,\pm\,6.466\mathrm{e}-4$	$0.017\,\pm\,1.758\mathrm{e}-4$

Appendix

Table 34: In-distribution results - Forecast. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse	masked_mae	masked_mse
No_sum_Trend lstm	$0.068\,\pm\,1.317\mathrm{e}-3$	$0.012\,\pm\,3.116\mathrm{e}-4$	$0.085\,\pm\,4.437\mathrm{e}-4$	$0.016\,\pm\,1.175\mathrm{e}-4$	$0.066\,\pm\,5.187\mathrm{e}-4$	$0.013\,\pm\,1.835\mathrm{e}-4$
No_sum_Trend mlp	$0.068\,\pm\,6.357\mathrm{e}-4$	$0.011\,\pm\,1.617\mathrm{e}-4$	$0.087\,\pm\,1.297\mathrm{e}-3$	$0.016\,\pm\,3.724\mathrm{e}-4$	$0.067\,\pm\,8.032\mathrm{e}-4$	$0.012\,\pm\,2.634\mathrm{e}-4$
No_sum_Trend patchtsmixer	$0.111\,\pm\,8.596\mathrm{e}-4$	$0.022\,\pm\,2.076\mathrm{e}-4$	$0.126\,\pm\,1.205\mathrm{e}-3$	$0.028\,\pm\,5.758\mathrm{e}-4$	$0.089\,\pm\,6.671\mathrm{e}-3$	$0.017\,\pm\,1.798\mathrm{e}-3$
No_sum_Trend timemixer	$0.072\,\pm\,2.463\mathrm{e}-3$	$0.012\,\pm\,5.846\mathrm{e}-4$	$0.088\,\pm\,1.163\mathrm{e}-3$	$0.016\,\pm\,3.008\mathrm{e}-4$	$0.079\,\pm\,1.129\mathrm{e}-3$	$0.014\,\pm\,3.756\mathrm{e}-4$
medgemma CoT +TFM lstm	$0.069\,\pm\,2.873\mathrm{e}-4$	$0.012\,\pm\,5.686\mathrm{e}-5$	$0.088\,\pm\,6.381\mathrm{e}-4$	$0.016\,\pm\,1.65\mathrm{e}-4$	$0.069\,\pm\,1.11\mathrm{e}-3$	$0.013\,\pm\,3.317\mathrm{e}-4$

Appendix

Table 35: In-distribution results - Forecast. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse
Llama-3.1 CoT +TFM lstm	$0.233\,\pm\,1.151\mathrm{e}-2$	$0.12\,\pm\,8.28\mathrm{e}-3$	$0.263\,\pm\,5.289\mathrm{e}-4$	$0.151\,\pm\,2.991\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.257\,\pm\,1.285\mathrm{e}-3$	$0.138\,\pm\,2.024\mathrm{e}-3$	$0.266\,\pm\,8.977\mathrm{e}-4$	$0.155\,\pm\,1.573\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.309\,\pm\,1.042\mathrm{e}-2$	$0.209\,\pm\,2.272\mathrm{e}-2$	$0.36\,\pm\,6.736\mathrm{e}-3$	$0.267\,\pm\,1.614\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.264\,\pm\,5.936\mathrm{e}-3$	$0.141\,\pm\,4.233\mathrm{e}-3$	$0.27\,\pm\,1.086\mathrm{e}-3$	$0.155\,\pm\,1.749\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.249\,\pm\,3.484\mathrm{e}-3$	$0.14\,\pm\,4.978\mathrm{e}-3$	$0.265\,\pm\,1.101\mathrm{e}-3$	$0.159\,\pm\,1.279\mathrm{e}-3$

Appendix

Table 36: Cross-site transfer results - Forecast. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	masked_mae	masked_mse	masked_mae	masked_mse
No_sum_Trend lstm	$0.249\,\pm\,2.623\mathrm{e}-3$	$0.143\,\pm\,2.817\mathrm{e}-3$	$0.271\,\pm\,1.111\mathrm{e}-3$	$0.163\,\pm\,1.111\mathrm{e}-3$
No_sum_Trend mlp	$0.253\,\pm\,2.225\mathrm{e}-3$	$0.145\,\pm\,2.026\mathrm{e}-3$	$0.284\,\pm\,2.136\mathrm{e}-3$	$0.175\,\pm\,1.803\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.293\,\pm\,5.256\mathrm{e}-3$	$0.19\,\pm\,1.041\mathrm{e}-2$	$0.345\,\pm\,9.259\mathrm{e}-3$	$0.243\,\pm\,1.885\mathrm{e}-2$
No_sum_Trend timemixer	$0.253\,\pm\,2.217\mathrm{e}-3$	$0.144\,\pm\,2.074\mathrm{e}-3$	$0.277\,\pm\,2.751\mathrm{e}-3$	$0.169\,\pm\,3.138\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.24\,\pm\,6.816\mathrm{e}-3$	$0.125\,\pm\,8.342\mathrm{e}-3$	$0.269\,\pm\,7.465\mathrm{e}-4$	$0.157\,\pm\,3.317\mathrm{e}-3$

Appendix

Table 37: Cross-site transfer results - Forecast. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.302\,\pm\,1.48\mathrm{e}-2$	$0.379\,\pm\,3.635\mathrm{e}-2$	$0.322\,\pm\,8.197\mathrm{e}-3$	$0.567\,\pm\,4.639\mathrm{e}-3$	$0.351\,\pm\,4.62\mathrm{e}-3$	$0.937\,\pm\,8.33\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.304\,\pm\,9.901\mathrm{e}-3$	$0.372\,\pm\,1.076\mathrm{e}-2$	$0.304\,\pm\,3.403\mathrm{e}-3$	$0.505\,\pm\,1.562\mathrm{e}-2$	$0.343\,\pm\,9.457\mathrm{e}-3$	$0.922\,\pm\,1.08\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.524\,\pm\,2.093\mathrm{e}-2$	$0.691\,\pm\,3.479\mathrm{e}-2$	$0.47\,\pm\,2.87\mathrm{e}-2$	$0.766\,\pm\,1.297\mathrm{e}-1$	$0.416\,\pm\,9.236\mathrm{e}-3$	$1.011\,\pm\,3.858\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.348\,\pm\,3.617\mathrm{e}-3$	$0.38\,\pm\,1.789\mathrm{e}-3$	$0.338\,\pm\,1.84\mathrm{e}-2$	$0.519\,\pm\,3.783\mathrm{e}-2$	$0.337\,\pm\,5.955\mathrm{e}-3$	$0.924\,\pm\,7.734\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.378\,\pm\,1.686\mathrm{e}-2$	$0.527\,\pm\,1.747\mathrm{e}-2$	$0.396\,\pm\,1.467\mathrm{e}-2$	$0.692\,\pm\,9.725\mathrm{e}-3$	$0.358\,\pm\,6.408\mathrm{e}-3$	$0.958\,\pm\,7.056\mathrm{e}-3$

Appendix

Table 38: In-distribution results - Los. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	mae	mse	mae	mse	mae	mse
No_sum_Trend lstm	$0.301\,\pm\,6.829\mathrm{e}-3$	$0.399\,\pm\,1.76\mathrm{e}-2$	$0.283\,\pm\,5.464\mathrm{e}-3$	$0.538\,\pm\,7.988\mathrm{e}-3$	$0.339\,\pm\,5.337\mathrm{e}-3$	$0.92\,\pm\,6.172\mathrm{e}-3$
No_sum_Trend mlp	$0.3\,\pm\,9.631\mathrm{e}-3$	$0.386\,\pm\,2.206\mathrm{e}-2$	$0.294\,\pm\,1.381\mathrm{e}-2$	$0.539\,\pm\,1.816\mathrm{e}-2$	$0.343\,\pm\,1.163\mathrm{e}-2$	$0.929\,\pm\,2.489\mathrm{e}-3$
No_sum_Trend patchtsmixer	$0.348\,\pm\,3.213\mathrm{e}-3$	$0.428\,\pm\,8.989\mathrm{e}-3$	$0.329\,\pm\,9.072\mathrm{e}-3$	$0.563\,\pm\,6.203\mathrm{e}-3$	$0.37\,\pm\,1.121\mathrm{e}-2$	$0.93\,\pm\,1.805\mathrm{e}-2$
No_sum_Trend timemixer	$0.363\,\pm\,1.044\mathrm{e}-2$	$0.457\,\pm\,1.057\mathrm{e}-2$	$0.326\,\pm\,1.872\mathrm{e}-2$	$0.575\,\pm\,8.601\mathrm{e}-3$	$0.325\,\pm\,9.516\mathrm{e}-3$	$0.916\,\pm\,6.562\mathrm{e}-3$
medgemma CoT +TFM lstm	$0.312\,\pm\,6.866\mathrm{e}-3$	$0.405\,\pm\,2.334\mathrm{e}-2$	$0.312\,\pm\,1.294\mathrm{e}-2$	$0.569\,\pm\,2.844\mathrm{e}-2$	$0.346\,\pm\,3.115\mathrm{e}-3$	$0.929\,\pm\,4.153\mathrm{e}-4$

Appendix

Table 39: In-distribution results - Los. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
Llama-3.1 CoT +TFM lstm	$0.343\,\pm\,6.473\mathrm{e}-3$	$1.017\,\pm\,4.164\mathrm{e}-3$	$0.347\,\pm\,1.144\mathrm{e}-3$	$0.988\,\pm\,2.541\mathrm{e}-3$
Llama-3.1 CoT +TFM mlp	$0.373\,\pm\,3.751\mathrm{e}-3$	$1.047\,\pm\,9.349\mathrm{e}-3$	$0.351\,\pm\,2.08\mathrm{e}-3$	$1.014\,\pm\,1.484\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.516\,\pm\,1.446\mathrm{e}-2$	$1.113\,\pm\,1.521\mathrm{e}-2$	$0.489\,\pm\,1.7\mathrm{e}-2$	$1.091\,\pm\,1.069\mathrm{e}-2$
Llama-3.1 CoT +TFM timemixer	$0.425\,\pm\,6.04\mathrm{e}-3$	$1.043\,\pm\,3.039\mathrm{e}-3$	$0.392\,\pm\,2.021\mathrm{e}-2$	$1.015\,\pm\,1.283\mathrm{e}-2$
Llama-3.1 CoT lstm	$0.795\,\pm\,3.91\mathrm{e}-2$	$1.544\,\pm\,5.866\mathrm{e}-2$	$0.558\,\pm\,2.002\mathrm{e}-2$	$1.267\,\pm\,5.593\mathrm{e}-2$

Appendix

Table 40: Cross-site transfer results - Los. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	mae	mse	mae	mse
No_sum_Trend lstm	$0.704\,\pm\,8.339\mathrm{e}-2$	$1.389\,\pm\,1.546\mathrm{e}-1$	$0.742\,\pm\,4.071\mathrm{e}-2$	$1.461\,\pm\,9.049\mathrm{e}-2$
No_sum_Trend mlp	$0.754\,\pm\,3.342\mathrm{e}-2$	$1.496\,\pm\,4.605\mathrm{e}-2$	$0.794\,\pm\,3.188\mathrm{e}-2$	$1.568\,\pm\,6.687\mathrm{e}-2$
No_sum_Trend patchtsmixer	$0.853\,\pm\,3.164\mathrm{e}-2$	$1.606\,\pm\,6.786\mathrm{e}-2$	$0.776\,\pm\,6.463\mathrm{e}-2$	$1.468\,\pm\,1.277\mathrm{e}-1$
No_sum_Trend timemixer	$0.802\,\pm\,5.126\mathrm{e}-2$	$1.562\,\pm\,1.056\mathrm{e}-1$	$0.753\,\pm\,3.791\mathrm{e}-2$	$1.451\,\pm\,8.137\mathrm{e}-2$
medgemma CoT +TFM lstm	$0.369\,\pm\,5.866\mathrm{e}-3$	$1.007\,\pm\,5.832\mathrm{e}-3$	$0.352\,\pm\,6.795\mathrm{e}-3$	$0.963\,\pm\,3.323\mathrm{e}-3$

Appendix

Table 41: Cross-site transfer results - Los. (part 2/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.434\,\pm\,4.047\mathrm{e}-2$	$0.888\,\pm\,1.253\mathrm{e}-2$	$0.539\,\pm\,1.934\mathrm{e}-2$	$0.851\,\pm\,8.073\mathrm{e}-3$	$0.573\,\pm\,2.554\mathrm{e}-2$	$0.796\,\pm\,1.254\mathrm{e}-2$
Llama-3.1 CoT +TFM mlp	$0.443\,\pm\,2.504\mathrm{e}-2$	$0.89\,\pm\,1.262\mathrm{e}-2$	$0.514\,\pm\,2.709\mathrm{e}-2$	$0.847\,\pm\,1.626\mathrm{e}-2$	$0.546\,\pm\,4.932\mathrm{e}-2$	$0.789\,\pm\,3.076\mathrm{e}-2$
Llama-3.1 CoT +TFM patchtsmixer	$0.351\,\pm\,4.992\mathrm{e}-3$	$0.847\,\pm\,2.807\mathrm{e}-3$	$0.412\,\pm\,4.796\mathrm{e}-3$	$0.776\,\pm\,4.092\mathrm{e}-3$	$0.247\,\pm\,1.763\mathrm{e}-3$	$0.506\,\pm\,1.556\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.343\,\pm\,1.168\mathrm{e}-2$	$0.847\,\pm\,4.997\mathrm{e}-3$	$0.488\,\pm\,4.005\mathrm{e}-3$	$0.834\,\pm\,2.875\mathrm{e}-3$	$0.537\,\pm\,3.13\mathrm{e}-2$	$0.788\,\pm\,1.751\mathrm{e}-2$
Llama-3.1 CoT lstm	$0.341\,\pm\,9.298\mathrm{e}-3$	$0.843\,\pm\,5.597\mathrm{e}-3$	$0.451\,\pm\,5.876\mathrm{e}-3$	$0.815\,\pm\,3.777\mathrm{e}-3$	$0.457\,\pm\,2.569\mathrm{e}-2$	$0.731\,\pm\,1.79\mathrm{e}-2$

Appendix

Table 42: In-distribution results - Mort. (part 1/2)

Method	hirid $\rightarrow$ hirid		mimic $\rightarrow$ mimic		ppicu $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc	auprc	auroc
medgemma ICD +TFM lstm	$0.582\,\pm\,6.357\mathrm{e}-3$	$0.927\,\pm\,2.878\mathrm{e}-3$	$0.572\,\pm\,1.236\mathrm{e}-2$	$0.87\,\pm\,5.52\mathrm{e}-3$	$0.66\,\pm\,2.533\mathrm{e}-2$	$0.844\,\pm\,1.297\mathrm{e}-2$
medgemma ICD +TFM mlp	$0.582\,\pm\,2.505\mathrm{e}-2$	$0.929\,\pm\,8.287\mathrm{e}-3$	$0.571\,\pm\,4.68\mathrm{e}-3$	$0.874\,\pm\,4.162\mathrm{e}-4$	$0.634\,\pm\,1.234\mathrm{e}-2$	$0.837\,\pm\,7.092\mathrm{e}-3$
medgemma ICD +TFM patchtsmixer	$0.467\,\pm\,5.098\mathrm{e}-3$	$0.899\,\pm\,2.98\mathrm{e}-3$	$0.489\,\pm\,2.237\mathrm{e}-2$	$0.821\,\pm\,1.109\mathrm{e}-2$	$0.582\,\pm\,1.334\mathrm{e}-2$	$0.786\,\pm\,8.672\mathrm{e}-3$
medgemma ICD +TFM timemixer	$0.453\,\pm\,6.038\mathrm{e}-2$	$0.894\,\pm\,1.792\mathrm{e}-2$	$0.526\,\pm\,1.474\mathrm{e}-2$	$0.853\,\pm\,7.344\mathrm{e}-3$	$0.609\,\pm\,1.786\mathrm{e}-2$	$0.827\,\pm\,1.059\mathrm{e}-2$
medgemma ICD lstm	$0.506\,\pm\,2.012\mathrm{e}-2$	$0.906\,\pm\,7.309\mathrm{e}-3$	$0.547\,\pm\,1.449\mathrm{e}-2$	$0.858\,\pm\,7.275\mathrm{e}-3$	$0.594\,\pm\,8.807\mathrm{e}-3$	$0.81\,\pm\,5.606\mathrm{e}-3$

Appendix

Table 43: In-distribution results - Mort. (part 2/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
Llama-3.1 CoT +TFM lstm	$0.365\,\pm\,5.073\mathrm{e}-3$	$0.656\,\pm\,2.368\mathrm{e}-3$	$0.392\,\pm\,2.422\mathrm{e}-3$	$0.681\,\pm\,9.8\mathrm{e}-4$
Llama-3.1 CoT +TFM mlp	$0.373\,\pm\,8.213\mathrm{e}-4$	$0.664\,\pm\,1.107\mathrm{e}-3$	$0.383\,\pm\,7.763\mathrm{e}-3$	$0.67\,\pm\,6.792\mathrm{e}-3$
Llama-3.1 CoT +TFM patchtsmixer	$0.342\,\pm\,2.198\mathrm{e}-3$	$0.625\,\pm\,1.408\mathrm{e}-3$	$0.361\,\pm\,4.654\mathrm{e}-3$	$0.633\,\pm\,5.435\mathrm{e}-3$
Llama-3.1 CoT +TFM timemixer	$0.369\,\pm\,4.226\mathrm{e}-3$	$0.662\,\pm\,3.855\mathrm{e}-3$	$0.376\,\pm\,2.861\mathrm{e}-3$	$0.665\,\pm\,2.893\mathrm{e}-3$
Llama-3.1 CoT lstm	$0.362\,\pm\,3.588\mathrm{e}-3$	$0.653\,\pm\,1.988\mathrm{e}-3$	$0.365\,\pm\,1.458\mathrm{e}-3$	$0.652\,\pm\,1.682\mathrm{e}-3$

Appendix

Table 44: Cross-site transfer results - Mort. (part 1/2)

Method	hirid $\rightarrow$ ppicu		mimic $\rightarrow$ ppicu
	auprc	auroc	auprc	auroc
medgemma ICD +TFM lstm	$0.459\,\pm\,2.197\mathrm{e}-3$	$0.715\,\pm\,1.475\mathrm{e}-3$	$0.489\,\pm\,4.21\mathrm{e}-3$	$0.741\,\pm\,2.302\mathrm{e}-3$
medgemma ICD +TFM mlp	$0.46\,\pm\,7.566\mathrm{e}-3$	$0.717\,\pm\,3.012\mathrm{e}-3$	$0.484\,\pm\,9.579\mathrm{e}-3$	$0.736\,\pm\,6.33\mathrm{e}-3$
medgemma ICD +TFM patchtsmixer	$0.408\,\pm\,2.124\mathrm{e}-3$	$0.675\,\pm\,2.087\mathrm{e}-3$	$0.449\,\pm\,2.893\mathrm{e}-3$	$0.7\,\pm\,5.408\mathrm{e}-4$
medgemma ICD +TFM timemixer	$0.438\,\pm\,1.577\mathrm{e}-2$	$0.712\,\pm\,4.393\mathrm{e}-3$	$0.482\,\pm\,3.808\mathrm{e}-3$	$0.735\,\pm\,6.012\mathrm{e}-4$
medgemma ICD lstm	$0.455\,\pm\,7.277\mathrm{e}-3$	$0.715\,\pm\,1.228\mathrm{e}-3$	$0.49\,\pm\,4.828\mathrm{e}-3$	$0.741\,\pm\,2.204\mathrm{e}-3$

Appendix

Table 45: Cross-site transfer results - Mort. (part 2/2)

Explore similar work

Future Querying: Can LLMs Serve as Implicit Medical World Models?
Aug 24, 2026Siri Willems, James Butterworth, Lore Goetschalckx +4Medical LLMs Clinical Prediction
imec, AI-labs⋆⋆, Paris, France
Training Large Language Models to Predict Clinical Events
May 12, 2026Benjamin Turtel, Paul Wilczewski, Kris SkotheimHealthcare Language Model Calibration
Lightning Rod Labs
ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning
Jun 1, 2026Bo-Hong Wang, Baicheng Peng, Ruilin Wang +3LLM Grounding LLM Alignment
School of Computer Science, McGill University, Montreal, QC, Canada · Mila - Quebec AI Institute, Montreal, QC, Canada

Aug 24, 2026J/K move · Enter open · S save

Future Querying: Can LLMs Serve as Implicit Medical World Models?

Siri Willems, James Butterworth, Lore Goetschalckx, Peter Vrancx+3

imec, AI-labs⋆⋆, Paris, France

Medical LLMs Clinical Prediction Longitudinal EHR Modeling Clinical Outcome Prediction

Open paper AskPDF arXiv

Traditional clinical prediction models rely on task-specific pipelines and curated, structured data, which scale poorly and underutilize unstructured text. To address this, we introduce future querying, a paradigm that probes whether large language models (LLMs) can function as implicit medical world models by evaluating their ability to answer time-indexed clinical queries about a patient's future. Our framework operates on unstructured clinical documentation using endpoint-agnostic training, enabling a single model to answer diverse clinical queries over patient trajectories without manual feature engineering or task-specific retraining. We show that small, locally fine-tuned open-weight models can match or approach larger proprietary systems, making the framework suitable for privacy-preserving, on-premise deployment. Evaluated on a new synthetic medical reports dataset and real ICU notes from the MIMIC-IV dataset, our results provide encouraging evidence that LLMs can capture aspects of clinical dynamics.

Abstract

Explore similar work

Future Querying: Can LLMs Serve as Implicit Medical World Models?

Training Large Language Models to Predict Clinical Events

ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning