Calibration-First Cross-Cohort Multimodal Temporal Learning for Transferable Asthma-Risk Forecasting

Taimoor Ahmad

cs.LGSep 17, 2026

Calibration-First Cross-Cohort Multimodal Temporal Learning for Transferable Asthma-Risk Forecasting

Authors: Taimoor Ahmad

Organizations: Superior University Lahore, Pakistan

Abstract

Asthma deterioration forecasting must remain reli- able when patient populations, sensor ecosystems, and available modalities change across cohorts. Existing models commonly optimize within-cohort discrimination and may produce poorly calibrated probabilities after transfer. We present CALIBRA, a calibration-first multimodal temporal framework for short- horizon risk prediction with incomplete data. Dedicated recurrent encoders process environmental, pulmonary, symptom, medication, wearable, and context streams; a reliability-conditioned gate suppresses stale or absent modalities, while gradient-reversal training discourages avoidable cohort signatures. A shrinkage- based hierarchical logistic layer calibrates probabilities using a patient-disjoint target subset, and split conformal prediction provides abstention-capable prediction sets. To avoid fabricating clinical evidence, we evaluate the complete implementation on a documented three-cohort semi-synthetic benchmark with controlled distribution shift, informative missingness, and sealed target patients. Across five configured seeds, CALIBRA achieved mean target-test AUPRC 0.224 versus 0.240 for the strongest non-ablation comparator, TemporalTransformer; mean AUROC was 0.717, and Brier score was 0.098. Experiments additionally assess complete-modality failures, calibration, conformal coverage, decision curves, subgroup behavior, ablations, runtime, and parameter count. The results verify the method and reproducible pipeline under controlled shift, but do not establish clinical effectiveness. External validation on harmonized real asthma. Overall this artifact provides evidence for carefully governed real-cohort validation.

Figures & tables

Study	Primary focus	MM	Shift	Cal.	Set UQ	Principal limitation for this problem
[ 5 ] (2018)	Mobile cohort	✓	–	–	–	No external sensor-set transfer
[ 6 ] (2017)	App review	–	–	–	–	Evidence and implementation heterogeneity
[ 7 ] (2017)	Sensor platform	✓	–	–	–	Single deployment setting
[ 8 ] (2023)	Attack prediction	✓	–	–	–	Small cohort; limited transportability
[ 9 ] (2023)	Prediction review	✓	–	✓	–	Heterogeneous outcomes and validation
[ 10 ] (2026)	AI review	✓	–	✓	–	No integrated transferable model

TABLE I: Comparison with representative literature. MM denotes explicit multimodal or missing-modality handling; shift denotes an explicit cross-domain or external-transport component; set UQ denotes set-valued uncertainty quantification. A dash means the capability was not the cited study’s primary contribution, not that it is impossible to add.

Symbol	Meaning
$C,\mathcal{C}$	Number and set of cohorts.
$M,\mathcal{M}$	Number and set of modality groups.
$i,c,t$	Patient, cohort, and window-end indices.
$L,H$	Look-back length and forecast horizon in days.
$d_{m}$	Number of variables in modality $m$ .
$\mathbf{x}^{(m)}$	Observed or imputed modality values.

TABLE II: Principal notation.

Panel of Figure 3 — Fig. 1: CALIBRA-ASTHMA pipeline. Modality-specific encoders permit nonidentical sensor sets across cohorts. Reliability and availability determine fusion weights. Unlabeled target-calibration windows support adversarial alignment during training; their labels are used only for post-hoc calibration and conformal fitting. Target-test patients remain untouched until final evaluation.

Partition	Cohort	Patients	Windows	Event rate (%)
Source training	A	49	2254	7.5
Source training	B	49	2254	11.8
Source validation	A	10	460	4.3
Source validation	B	10	460	4.6
Source test	A	11	506	4.7
Source test	B	11	506	8.3

TABLE III: Semi-synthetic benchmark partitions. Patients are disjoint across partitions.

Item	Setting
Observation window	7 patient-days
Prediction horizon	3 days
Source cohorts	A and B
Target cohort	C
Target calibration	25% of target patients
Primary repetitions	5 random seeds

TABLE IV: Primary experimental configuration.

Method	AUROC $\uparrow$	AUPRC $\uparrow$	Brier $\downarrow$	ECE $\downarrow$
Prevalence	0.500 $\pm$ 0.000	0.117 $\pm$ 0.000	0.104 $\pm$ 0.000	0.006 $\pm$ 0.000
ElasticNet	0.685 $\pm$ 0.000	0.198 $\pm$ 0.000	0.102 $\pm$ 0.000	0.026 $\pm$ 0.000
HistGB	0.676 $\pm$ 0.000	0.200 $\pm$ 0.000	0.100 $\pm$ 0.000	0.020 $\pm$ 0.000
GRU-D	0.701 $\pm$ 0.012	0.217 $\pm$ 0.014	0.099 $\pm$ 0.001	0.015 $\pm$ 0.002
Transformer	0.715 $\pm$ 0.006	0.240 $\pm$ 0.007	0.098 $\pm$ 0.000	0.015 $\pm$ 0.001
DANN–GRU	0.704 $\pm$ 0.012	0.218 $\pm$ 0.014	0.098 $\pm$ 0.001	0.014 $\pm$ 0.003

TABLE V: Held-out cohort performance over five seeded runs. Values are mean

\pm

standard deviation. Higher is better for AUROC and AUPRC; lower is better for Brier score and ECE. Bold and underlined entries denote the best and second-best directly comparable results.

Panel of Figure 7 — Fig. 2: Primary held-out cohort results. Bars show means over five seeded runs; error bars show one standard deviation. The prevalence model provides a non-discriminative reference.

Metric	Baseline	Difference	$p_{H}$
AUPRC	DANN–GRU	0.007	1.000
AUPRC	GRU-D	0.007	1.000
AUPRC	HistGB	0.025	0.750
AUPRC	Transformer	-0.016	1.000
Brier	DANN–GRU	0.001	0.750
Brier	GRU-D	0.001	0.750

TABLE VI: One-sided paired Wilcoxon comparisons of CALIBRA with representative baselines. Positive differences favor CALIBRA;

p_{H}

is Holm-adjusted.

Panel of Figure 9 — Fig. 4: Robustness and uncertainty analyses. Left: performance after prespecified modality removal. Center and right: empirical split-conformal coverage and efficiency. Coverage is marginal for the defined target-calibration/test exchangeability design; the plots do not imply unrestricted guarantees under arbitrary drift.

Panel of Figure 10 — Fig. 5: Interpretability and optimization diagnostics. Gate weights summarize the held-out target cohort and should be interpreted jointly with observation fractions, not as causal feature importance. The training panel shows the first configured seed; early stopping uses source-validation AUPRC.

Panel of Figure 11 — Fig. 6: Operational and subgroup diagnostics. Decision curves show that model utility depends on the intervention threshold and false-alert trade-off. Severity-stratified results are simulator diagnostics only and cannot support clinical fairness conclusions.

Variant (runs)	AUPRC	Brier	ECE
Full (5)	0.224 $\pm$ 0.007	0.098 $\pm$ 0.001	0.013 $\pm$ 0.002
No alignment (3)	0.227 $\pm$ 0.006	0.098 $\pm$ 0.001	0.012 $\pm$ 0.003
Uniform gate (3)	0.225 $\pm$ 0.008	0.098 $\pm$ 0.001	0.013 $\pm$ 0.004
No reliability (3)	0.231 $\pm$ 0.022	0.097 $\pm$ 0.001	0.016 $\pm$ 0.005

TABLE VII: Ablation results on the held-out cohort. The number of seeded runs is shown in parentheses.

Panel of Figure 13 — Fig. 7: Component and efficiency analysis. Ablations use three seeds to control computation; primary model comparisons use five. Inference timings reflect the recorded software/hardware environment and exclude disk I/O.

Method	Train (s)	Infer (ms/sample)	Size (MiB)
Prevalence	0.0	0.0000	0.001
ElasticNet	8.3	0.0011	0.006
HistGB	0.9	0.0047	0.632
GRU-D	1.3	0.0235	0.012
Transformer	3.8	0.0257	0.081
DANN–GRU	1.7	0.0164	0.013

TABLE VIII: Computational profile on the recorded execution device. Inference excludes data loading.

Clinical Risk Prediction

Clinical risk prediction estimates patients’ future health outcomes from medical records, physiological signals, imaging, and other health data.

See all 217 papers

Aug 7, 2026

Conformal Calibration for Multi-Modal Regression with Missing Modalities

Prediction intervals for multi-modal regression with tabular variables, text, images, or other input sources are difficult to calibrate when those sources disagree or one is missing. A single global quantile averages these regimes together instead of calibrating to the modality pattern observed at test time. We address this through a modality-aware conformal calibration layer. The layer trains or reuses one predictor per modality, computes a disagreement score from their predictions, and uses that score in split conformal calibration under a strict split protocol. We use the score in two complementary ways. First, a continuous disagreement-scaled method reallocates interval width across examples while preserving the usual marginal split-conformal guarantee. Second, a Mondrian (stratified) method calibrates within groups defined by disagreement or modality availability fixed before calibration, giving group guarantees under joint exchangeability of the calibration and test examples. Across four multi-modal datasets, the disagreement-scaled layer matches or improves the marginal conformal baseline in 59 of 60 paired runs for interval continuous ranked probability score (CRPS) and in 52 of 60 for interval width, while keeping empirical coverage near the 95% target. In stress tests with missing modalities, mask-matched recalibration recovers up to 19.5 percentage points of coverage in the hardest fixed-mask regime. The result is a simple, model-agnostic reliability layer for multi-modal regression systems. A project page is available at https://unco3892.github.io/modality-aware-conformal.

Ilia Azizi

Department of Operations, HEC Lausanne, University of Lausanne, Switzerland · BegooAI, Switzerland

Group-Conditional Conformal Prediction Multimodal Regression Model Calibration Multimodal Learning Regression Prediction Intervals

Ask PDF

May 22, 2026

Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Prediction

Deploying clinical prediction models across healthcare systems often fails when key training covariates are unavailable at deployment and labeled outcomes are limited in the target domain. For example, high-performing models for out-of-hospital cardiac arrest (OHCA) rely on detailed prehospital measurements routinely collected in high-resource settings but unavailable in many international registries. Existing methods either discard missing covariates, sacrificing predictive information, or rely on untestable assumptions about their target distribution. We propose DRUM (Distributionally Robust Unsupervised transfer learning with structurally Missing covariates), a framework that transfers prediction models to target populations where certain covariates are structurally absent and outcome labels are unavailable. DRUM partitions covariates into shared components (

X

), observed across all settings, and missing components (

A

), observed only in the source. Rather than imputing missing covariates, DRUM optimizes worst-case predictive performance over the unknown target distribution of

A \mid X

using a neural network generator, with a robustness parameter controlling allowable deviation from the source conditional. We further develop a bias correction procedure that reduces sensitivity to nuisance estimation error. Simulations show substantial improvements in both mean and worst-case prediction error under distribution shift. Applied to cross-national OHCA prediction, transferring models from a US registry to multiple Asian registries where prehospital variables are unrecorded, DRUM yields better-calibrated predictions and improved clinical classification performance across sites.

Siqi Li, Chuan Hong, Ziye Tian +9

Centre for Biomedical Data Science, Duke-NUS Medical School, Singapore · 2Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School, Singapore · Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA +13

Healthcare Clinical Outcome Prediction Cross-Domain Transfer Learning Transfer Learning Clinical Risk Prediction

Ask PDF

Mar 9, 2026

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data

Safe predictions are a crucial requirement for integrating predictive models into clinical decision support systems. One approach to improving trustworthiness is to enable models to express uncertainty about individual predictions. However, current machine learning models frequently lack reliable uncertainty estimation, hindering real-world deployment. This limitation is particularly evident in multimodal settings, where models must effectively integrate heterogeneous information. In this work, we propose MedCertAIn, a predictive uncertainty framework that leverages multimodal clinical data to improve model performance and reliability for in-hospital mortality risk prediction as an indicator of patient deterioration. We design data-driven priors over neural network parameters using a hybrid strategy that considers cross-modal similarity in self-supervised latent representations and modality-specific data corruptions. We train and evaluate the models with such priors using clinical time-series and chest X-ray images from the publicly available datasets MIMIC-IV and MIMIC-CXR. Our results show that MedCertAIn achieves competitive predictive performance and substantial gains in selective prediction compared with the evaluated deterministic and stochastic baselines. These findings highlight the promise of data-driven priors in advancing robust, uncertainty-aware AI tools for high-stakes clinical applications. Our implementation is publicly available at: https://github.com/jlaitue/medcertain.

L. Julián Lechuga López, Tim G. J. Rudner, Farah E. Shamout

Tandon School of Engineering, New York University · New York University Abu Dhabi · University of Toronto

Bayesian Neural Networks Uncertainty Quantification Selective Prediction Mortality Prediction Clinical Risk Prediction Multimodal Learning

Ask PDF

Abstract

Clinical Risk Prediction

Explore similar work

Conformal Calibration for Multi-Modal Regression with Missing Modalities

Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Prediction

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data