Calibration-First Cross-Cohort Multimodal Temporal Learning for Transferable Asthma-Risk Forecasting

Taimoor Ahmad

cs.LGSep 17, 2026

Calibration-First Cross-Cohort Multimodal Temporal Learning for Transferable Asthma-Risk Forecasting

Authors: Taimoor Ahmad

Organizations: Superior University Lahore, Pakistan

Abstract

Asthma deterioration forecasting must remain reli- able when patient populations, sensor ecosystems, and available modalities change across cohorts. Existing models commonly optimize within-cohort discrimination and may produce poorly calibrated probabilities after transfer. We present CALIBRA, a calibration-first multimodal temporal framework for short- horizon risk prediction with incomplete data. Dedicated recurrent encoders process environmental, pulmonary, symptom, medication, wearable, and context streams; a reliability-conditioned gate suppresses stale or absent modalities, while gradient-reversal training discourages avoidable cohort signatures. A shrinkage- based hierarchical logistic layer calibrates probabilities using a patient-disjoint target subset, and split conformal prediction provides abstention-capable prediction sets. To avoid fabricating clinical evidence, we evaluate the complete implementation on a documented three-cohort semi-synthetic benchmark with controlled distribution shift, informative missingness, and sealed target patients. Across five configured seeds, CALIBRA achieved mean target-test AUPRC 0.224 versus 0.240 for the strongest non-ablation comparator, TemporalTransformer; mean AUROC was 0.717, and Brier score was 0.098. Experiments additionally assess complete-modality failures, calibration, conformal coverage, decision curves, subgroup behavior, ablations, runtime, and parameter count. The results verify the method and reproducible pipeline under controlled shift, but do not establish clinical effectiveness. External validation on harmonized real asthma. Overall this artifact provides evidence for carefully governed real-cohort validation.

Figures & tables

Study	Primary focus	MM	Shift	Cal.	Set UQ	Principal limitation for this problem
[ 5 ] (2018)	Mobile cohort	✓	–	–	–	No external sensor-set transfer
[ 6 ] (2017)	App review	–	–	–	–	Evidence and implementation heterogeneity
[ 7 ] (2017)	Sensor platform	✓	–	–	–	Single deployment setting
[ 8 ] (2023)	Attack prediction	✓	–	–	–	Small cohort; limited transportability
[ 9 ] (2023)	Prediction review	✓	–	✓	–	Heterogeneous outcomes and validation
[ 10 ] (2026)	AI review	✓	–	✓	–	No integrated transferable model

TABLE I: Comparison with representative literature. MM denotes explicit multimodal or missing-modality handling; shift denotes an explicit cross-domain or external-transport component; set UQ denotes set-valued uncertainty quantification. A dash means the capability was not the cited study’s primary contribution, not that it is impossible to add.

Symbol	Meaning
$C,\mathcal{C}$	Number and set of cohorts.
$M,\mathcal{M}$	Number and set of modality groups.
$i,c,t$	Patient, cohort, and window-end indices.
$L,H$	Look-back length and forecast horizon in days.
$d_{m}$	Number of variables in modality $m$ .
$\mathbf{x}^{(m)}$	Observed or imputed modality values.

TABLE II: Principal notation.

Panel of Figure 3 — Fig. 1: CALIBRA-ASTHMA pipeline. Modality-specific encoders permit nonidentical sensor sets across cohorts. Reliability and availability determine fusion weights. Unlabeled target-calibration windows support adversarial alignment during training; their labels are used only for post-hoc calibration and conformal fitting. Target-test patients remain untouched until final evaluation.

Partition	Cohort	Patients	Windows	Event rate (%)
Source training	A	49	2254	7.5
Source training	B	49	2254	11.8
Source validation	A	10	460	4.3
Source validation	B	10	460	4.6
Source test	A	11	506	4.7
Source test	B	11	506	8.3

TABLE III: Semi-synthetic benchmark partitions. Patients are disjoint across partitions.

Item	Setting
Observation window	7 patient-days
Prediction horizon	3 days
Source cohorts	A and B
Target cohort	C
Target calibration	25% of target patients
Primary repetitions	5 random seeds

TABLE IV: Primary experimental configuration.

Method	AUROC $\uparrow$	AUPRC $\uparrow$	Brier $\downarrow$	ECE $\downarrow$
Prevalence	0.500 $\pm$ 0.000	0.117 $\pm$ 0.000	0.104 $\pm$ 0.000	0.006 $\pm$ 0.000
ElasticNet	0.685 $\pm$ 0.000	0.198 $\pm$ 0.000	0.102 $\pm$ 0.000	0.026 $\pm$ 0.000
HistGB	0.676 $\pm$ 0.000	0.200 $\pm$ 0.000	0.100 $\pm$ 0.000	0.020 $\pm$ 0.000
GRU-D	0.701 $\pm$ 0.012	0.217 $\pm$ 0.014	0.099 $\pm$ 0.001	0.015 $\pm$ 0.002
Transformer	0.715 $\pm$ 0.006	0.240 $\pm$ 0.007	0.098 $\pm$ 0.000	0.015 $\pm$ 0.001
DANN–GRU	0.704 $\pm$ 0.012	0.218 $\pm$ 0.014	0.098 $\pm$ 0.001	0.014 $\pm$ 0.003

TABLE V: Held-out cohort performance over five seeded runs. Values are mean

\pm

standard deviation. Higher is better for AUROC and AUPRC; lower is better for Brier score and ECE. Bold and underlined entries denote the best and second-best directly comparable results.

Panel of Figure 7 — Fig. 2: Primary held-out cohort results. Bars show means over five seeded runs; error bars show one standard deviation. The prevalence model provides a non-discriminative reference.

Metric	Baseline	Difference	$p_{H}$
AUPRC	DANN–GRU	0.007	1.000
AUPRC	GRU-D	0.007	1.000
AUPRC	HistGB	0.025	0.750
AUPRC	Transformer	-0.016	1.000
Brier	DANN–GRU	0.001	0.750
Brier	GRU-D	0.001	0.750

TABLE VI: One-sided paired Wilcoxon comparisons of CALIBRA with representative baselines. Positive differences favor CALIBRA;

p_{H}

is Holm-adjusted.

Panel of Figure 9 — Fig. 4: Robustness and uncertainty analyses. Left: performance after prespecified modality removal. Center and right: empirical split-conformal coverage and efficiency. Coverage is marginal for the defined target-calibration/test exchangeability design; the plots do not imply unrestricted guarantees under arbitrary drift.

Panel of Figure 10 — Fig. 5: Interpretability and optimization diagnostics. Gate weights summarize the held-out target cohort and should be interpreted jointly with observation fractions, not as causal feature importance. The training panel shows the first configured seed; early stopping uses source-validation AUPRC.

Panel of Figure 11 — Fig. 6: Operational and subgroup diagnostics. Decision curves show that model utility depends on the intervention threshold and false-alert trade-off. Severity-stratified results are simulator diagnostics only and cannot support clinical fairness conclusions.

Variant (runs)	AUPRC	Brier	ECE
Full (5)	0.224 $\pm$ 0.007	0.098 $\pm$ 0.001	0.013 $\pm$ 0.002
No alignment (3)	0.227 $\pm$ 0.006	0.098 $\pm$ 0.001	0.012 $\pm$ 0.003
Uniform gate (3)	0.225 $\pm$ 0.008	0.098 $\pm$ 0.001	0.013 $\pm$ 0.004
No reliability (3)	0.231 $\pm$ 0.022	0.097 $\pm$ 0.001	0.016 $\pm$ 0.005

TABLE VII: Ablation results on the held-out cohort. The number of seeded runs is shown in parentheses.

Panel of Figure 13 — Fig. 7: Component and efficiency analysis. Ablations use three seeds to control computation; primary model comparisons use five. Inference timings reflect the recorded software/hardware environment and exclude disk I/O.

Method	Train (s)	Infer (ms/sample)	Size (MiB)
Prevalence	0.0	0.0000	0.001
ElasticNet	8.3	0.0011	0.006
HistGB	0.9	0.0047	0.632
GRU-D	1.3	0.0235	0.012
Transformer	3.8	0.0257	0.081
DANN–GRU	1.7	0.0164	0.013

TABLE VIII: Computational profile on the recorded execution device. Inference excludes data loading.

Clinical Risk Prediction

Clinical risk prediction estimates patients’ future health outcomes from medical records, physiological signals, imaging, and other health data.

See all 217 papers

Conformal Calibration for Multi-Modal Regression with Missing Modalities
Aug 7, 2026Ilia AziziGroup-Conditional Conformal Prediction Multimodal Regression
Department of Operations, HEC Lausanne, University of Lausanne, Switzerland · BegooAI, Switzerland
Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Prediction
May 22, 2026Siqi Li, Chuan Hong, Ziye Tian +9Healthcare Clinical Outcome Prediction
Centre for Biomedical Data Science, Duke-NUS Medical School, Singapore · 2Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School, Singapore · Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA +13
Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data
Mar 9, 2026L. Julián Lechuga López, Tim G. J. Rudner, Farah E. ShamoutBayesian Neural Networks Uncertainty Quantification
Tandon School of Engineering, New York University · New York University Abu Dhabi · University of Toronto

Aug 7, 2026J/K move · Enter open · S save

Conformal Calibration for Multi-Modal Regression with Missing Modalities

Ilia Azizi

Department of Operations, HEC Lausanne, University of Lausanne, Switzerland · BegooAI, Switzerland

Group-Conditional Conformal Prediction Multimodal Regression Model Calibration Multimodal Learning Regression Prediction Intervals

Open paper AskPDF arXiv

Prediction intervals for multi-modal regression with tabular variables, text, images, or other input sources are difficult to calibrate when those sources disagree or one is missing. A single global quantile averages these regimes together instead of calibrating to the modality pattern observed at test time. We address this through a modality-aware conformal calibration layer. The layer trains or reuses one predictor per modality, computes a disagreement score from their predictions, and uses that score in split conformal calibration under a strict split protocol. We use the score in two complementary ways. First, a continuous disagreement-scaled method reallocates interval width across examples while preserving the usual marginal split-conformal guarantee. Second, a Mondrian (stratified) method calibrates within groups defined by disagreement or modality availability fixed before calibration, giving group guarantees under joint exchangeability of the calibration and test examples. Across four multi-modal datasets, the disagreement-scaled layer matches or improves the marginal conformal baseline in 59 of 60 paired runs for interval continuous ranked probability score (CRPS) and in 52 of 60 for interval width, while keeping empirical coverage near the 95% target. In stress tests with missing modalities, mask-matched recalibration recovers up to 19.5 percentage points of coverage in the hardest fixed-mask regime. The result is a simple, model-agnostic reliability layer for multi-modal regression systems. A project page is available at https://unco3892.github.io/modality-aware-conformal.

Abstract

Clinical Risk Prediction

Explore similar work

Conformal Calibration for Multi-Modal Regression with Missing Modalities

Distributionally Robust Transfer Learning with Structurally Missing Covariates, with Application to Cross-National Cardiac Arrest Prediction

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data