cs.LGOct 9, 2026
SaveHidden Data Challenges in Clinical Machine Learning
Organizations: Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Hannover Medical School, Germany · DATEXIS, Berliner Hochschule für Technik, Germany · Lower Saxony Center for AI and Causal Methods in Medicine (CAIMed), Hannover, Germany · L3S Research Center, Leibniz University Hannover, Germany
Abstract
Widely reused preprocessing pipelines for electronic health records can encode assumptions about when information became available, what clinical episode a label represents, and what missingness means. These assumptions are rarely audited systematically and may propagate across subsequent studies when preprocessing pipelines are reused or adapted. Using MIMIC-IV as a case study, we identify and quantify violations of three complementary forms of preprocessing alignment, i.e., temporal, semantic, and population alignment, which manifest as temporal leakage, feature-task misalignment, and biased imputation. We show how ambiguities in timestamp semantics, episode definitions, endpoint construction, modality selection, and missingness handling can alter the information available to a model and affect the interpretation of its predictions. Through analyses of MIMIC-IV data and empirical case studies, we demonstrate that such choices may remain largely invisible in aggregate predictive performance while changing the signals on which models rely. We derive practical recommendations for temporally valid, clinically aligned, and reproducible preprocessing of MIMIC-IV and related electronic health record datasets. The code for our experiments, including our proposed preprocessing, is available at: https://github.com/m2l-lab/mimic-iv-preprocess.
Figures & tables
Figure 1: A broad overview of the different generations of the MIMIC dataset and the information categories they provide. The release dates, version numbers, and patient and visit counts correspond to the details documented in the official release publications accompanying each iteration. Information types shared across generations should not necessarily be interpreted as equivalent in scope, as the granularity, completeness, and source quality of the documented data have improved substantially across iterations. This paper’s analyses use the latest version of MIMIC data available at the time of writing, i.e., MIMIC-IV 3.1.
Figure 2: Overview of the table structure of the MIMIC-IV modules. The timestamps shown for the highlighted tables serve as representative examples of the different types of temporal information provided in MIMIC-IV. For the full overview of linkage identifiers and temporal fields relevant for episode alignment, refer to Figure 9 in Appendix A
Figure 3: A conceptual illustration showing how information from different modules reconstructs a patient’s clinical timeline. The overlap between ED and ICU stays, as well as the misaligned end of stay, shows discrepancies between documented admission and discharge times across stations. Because the discharge report is written retrospectively, its final timestamp can fall outside the documented hospital stay. ICD codes, for which no precise timestamp is documented, can only be matched to the respective stay. The figure highlights how different timestamps represent different temporal perspectives: for example, charttime can support patient-state modeling, whereas storetime provides a later proxy for information availability in clinical decision-making. Due to the temporal difference, the choice of timestamps can affect the information available at prediction time.
| ICD Code | Description of Code | Count |
| 998.1 | Other complications of procedures not elsewhere classified: | 4808 |
| Hemorrhage or hematoma complicating a procedure not elsewhere classified | ||
| T82.8 | Complications of cardiac and vascular prosthetic devices, implants and grafts: | 4712 |
| Other specified complications of cardiac and vascular prosthetic devices, implants and grafts | ||
| 996.7 | Complications peculiar to certain specified procedures: | 4494 |
| Other complications due to internal prosthetic device, implant, and graft |
Table 1: Examples of ICD codes that are likely to occur during a hospital stay and their frequency in the MIMIC-IV 3.1 ( Johnson et al., 2024a ) dataset.
Figure 4: Analysis of the time difference in hours on a log scale between charttime and storetime in the labevents and microbiologyevents tables. Median is denoted by a dot, 25th to 75th percentile by a thick line, and 5th to 95th percentile by a thin line. Exact median and 5th to 95th percentile are 1.2 hours (0.1–3.6) for labevents, 53.2 hours (3.2–149.1) for the first finalized test of microbiologyevents, and 120.7 hours (10.7–311.5) for the last finalized test of microbiologyevents.
| Section | Temporal Connection | |||
| Historical | At Admission | During the Stay | At Discharge | |
| Sex | ||||
| Service | ||||
| Allergies | ||||
| Chief Complaint | ||||
| Major Surgical or Invasive Procedure | ||||
Table 2: Temporal organization of sections in MIMIC-IV discharge notes. An x indicates that the section typically contains information connected to the corresponding temporal interval.
Figure 5: Differences in ICD code prevalence between mortality subgroups of ICU stays. Percentages for ICD code occurrences are reported relative to the full subgroup; in-hospital includes 9,475 stays (11.1%), out-of-hospital 14,383 stays (16.9%), and survived yr 61,384 stays (72.0%).
Figure 6: Visualization of descriptive statistics for six laboratory concepts from MIMIC-IV 3.1 ( Johnson et al., 2024a ) . Their names, IDs, units of measurement, and reference ranges are directly taken from MIMIC. We show the reference ranges for each laboratory concept as shaded areas. The 25th to 75th percentile is indicated by a thick line, and the 5th to 95th percentile by a thin line. The mean is denoted as a dot, and the median as a diamond. Since no outliers are excluded and the mean is sensitive to them, we also report the trimmed mean calculated only from values between the 5th and 95th percentiles, visualized as a square.
| Name | Modality | Type | Unit | Static value |
| Demographics | ||||
| Age | tabular | discrete | years | - |
| Weight | tabular | continuous | kg | median |
| Sex | tabular | nominal | - | - |
| Ethnicity | tabular | nominal | - | - |
| Height | tabular | continuous | cm | median |
Table 3: Time series and tabular variables included in the dataset.
| Preprocessing Approach | In-hospital Mortality Prediction | |
| AUROC in % | AUPRC in % | |
| Static value storetime | ||
| Statistical storetime | ||
| Static value charttime | ||
| Statistical charttime | ||
| Static value storetime + missing | ||
Table 4: Averaged results over 5 runs of predicting in-hospital mortality with models trained on dataset variations of timestamp choice (chart- or storetime), imputation strategy (statistical global mean/mode or static value imputation), and optional inclusion of missingness information.
Figure 7: Differences between mean SHAP values across charttime and storetime timestamps for predicting in-hospital mortality in models trained with data preprocessed using: (a) statistical mean/mode imputation, (b) static value imputation, (c) statistical mean/mode imputation with missingness information, and (d) static value imputation with missingness information.
Figure 8: The percentage of total mean SHAP values assigned to features derived from the image modality for each phenotype prediction task (lower x-axis) and the performance delta between only structured data and structured data with CXR as input features (upper x-axis). Model performance delta for each phenotype is reported for AUROC and AUPRC. Mean values and confidence intervals are reported over five runs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Overview of the MIMIC-IV modules and tables included in the schema. The figure emphasizes linkage identifiers and temporal fields that are relevant for episode alignment and the resolution of temporal ambiguity rather than listing every available variable. Colors indicate recurring identifier scopes across modules, including the global patient identifier ( subject_id ), encounter- and stay-level identifiers ( hadm_id , stay_id ), and subsystem-specific local identifiers. Tables containing the complete list of variables for all included MIMIC-IV tables are provided in the Appendix. For the complete official schema, including table and column descriptions, see the MIMIC-IV documentation at https://mimic.mit.edu .
Figure 10: Schema overview of the MIMIC-IV hosp module, highlighting the main tables, linkage identifiers, and temporal fields.
Figure 11: Schema overview of the MIMIC-IV ED, ICU, note, ECG, and CXR modules, highlighting the main tables, linkage identifiers, and temporal fields.
| Train | Validation | Test | |
| Unique patients, n | 25,276 | 1,758 | 4,411 |
| Unique stays, n | 29,171 | 2,161 | 5,302 |
| Age, mean | |||
| Sex, % | |||
| Female | 44.08 | 43.54 | 43.29 |
| Male | 55.92 | 56.46 | 56.71 |
Table 5: Cohort description for the in-hospital mortality experiment by data split. For each time series variable, partial measurements (at least one bin measured, but not all) and completely missing measurements (no bin measured) within the 48h window are given. For tabular data, only complete missingness is reported. Chart- and storetime variants are listed separately.
| Train | Validation | Test | |
| Unique patients, n | 4,449 | 419 | 1,118 |
| Unique stays, n | 4,956 | 490 | 1,231 |
| Age, mean | |||
| Sex, % | |||
| Female | 45.16 | 42.86 | 43.38 |
| Male | 54.84 | 57.14 | 56.62 |
Table 6: Cohort description for the phenotype prediction experiment separated by train/validation/test split.
More in this topic
Training Data Curation
Training data curation improves the quality, diversity, organization, and reliability of datasets used to train machine-learning models.
See all 153 papers