Temporal Island Sparse Autoencoders for Interpreting Clinical Time-Series Models
Organizations: Department of Electrical and Computer Engineering and Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Republic of Korea · McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA · UNIST InnoCORE AI-Space Solar Initiative, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Republic of Korea
Abstract
Deep learning models for electronic health records achieve strong predictive performance but provide little insight into what clinical concepts they represent internally. Post-hoc attribution methods produce per-cell importance scores that lack temporal coherence, cross-variable structure, and reusability across patients. We introduce the Temporal Island Sparse Autoencoder (TI-SAE), which learns a backbone-specific, patient-shared clinical concept dictionary from a frozen predictive model's representations. Each dictionary entry (latent) is defined by learned temporal Gaussian islands (specifying when the concept is active) and a sparse variable-composition vector (specifying which variables are involved). A decoupled selector then identifies which dictionary entries drive each patient's prediction, separating representation learning from attribution. We evaluate TI-SAE across five backbone architectures spanning different representation strategies on MIMIC-III and MIMIC-IV (three prediction tasks each). TI-SAE produces structured clinical concepts that go beyond transient attribution heatmaps while achieving strong performance on standard faithfulness benchmarks.
Figures & tables
| Method family | Temporal struct. | Decomposable | Post-hoc | No concept sup. |
|---|---|---|---|---|
| Attention | ✓ | ✓ | ||
| Gradient-based | (✓) | ✓ | ✓ | |
| Perturbation-based | (✓) | ✓ | ✓ | |
| Learned selection | (✓) | ✓ | ✓ | |
| Concept bottleneck | ✓ | |||
| TI-SAE (ours) | ✓ | ✓ | ✓ | ✓ |
| Faithfulness (AOPC ) | Clinical Plausibility (AUROC/AUPRC ) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Disc. | D+PT | HyM. | BAT | STraTS | Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
| TI-SAE (soft) | .086 | .076 | .097 | .043 | .016 | .593 / .286 | .589 / .255 | .614 / .276 | .588 / .251 | .536/.227 |
| TI-SAE (hard) | .079 | .068 | .090 | .041 | .016 | .569/ .267 | .568/.243 | .581/.257 | .567/.243 | .535/.228 |
| Attention | .053 | .019 | .051 | – | .016 | .570/.242 | .539/.219 | .554/.218 | – | .551/.217 |
| IG | .060 | .051 | .049 | .031 | .067 | .545/.225 | .549/.229 | .596/.244 | .584/.239 | .577/.223 |
| TSR | .063 | .054 | .062 | .037 | .071 | .566/.243 | .573/.241 | .625 /.272 | .608 / .261 | .595 / .234 |
| AOPC | AUROC/AUPRC | |||
|---|---|---|---|---|
| ID (MIMIC-III) | OOD (MIMIC-IV) | ID (MIMIC-III) | OOD (MIMIC-IV) | |
| TI-SAE (soft) | .149 | .130 | .756/.386 | .756/.370 |
| Attention | .100 | .097 | .616/.259 | .631/.261 |
| IG | .086 | .045 | .567/.255 | .552/.224 |
| TSR | .088 | .046 | .593/.280 | .587/.245 |
| LLM judge | Clinician | |||
|---|---|---|---|---|
| vs. Att | vs. IG | vs. TSR | vs. IG | |
| Concept | 30 –17 (3) | 33 –15 (2) | 33 –16 (1) | 39 –4 (7) |
| Actionability | 28 –22 (0) | 32 –18 (0) | 27 –23 (0) | 23 –9 (18) |
| Faithfulness | 19 –15 (16) | 15 –11 (24) | 28 –9 (13) | 4–4 (42) |
Appendix figures & tables53 assets
Supplementary material from the paper’s appendix.
Appendix
| Backbone | Layers | Heads | Dropout | LR | Tokenization | |
|---|---|---|---|---|---|---|
| Discrete | 128 | 4 | 8 | 0.3 | Binned (single-token) | |
| Discrete+PT | 128 | 4 | 8 | 0.3 | Binned (single-token) | |
| STraTS | 64 | 4 | 16 | 0.2 | CVE (single-token) | |
| HyMaTE | 64 | 2 3 | 4 | 0.2 | CVE (single-token) | |
| BAT | 88 | 1 | 1 | 0.2 | Grid-native |
| Category | Parameter | Value |
| Architecture | Latents | 1280 |
| Islands per latent | 4 | |
| Selection budget | 128 | |
| Encoder / Decoder | Encoder Variable Routing sparsity | 5 |
| Decoder Variable Routing sparsity | 3 | |
| Atom normalization | L2-normalize |
| Component | Dim | Range | Description |
|---|---|---|---|
| 1 | Normalized island center position | ||
| 1 | Normalized island width | ||
| 1 | Gaussian-weighted activation energy (batch-normalized) | ||
| 1 | Peak activation at (batch-normalized) | ||
| 1 | Budget gate value from IslandParamNet | ||
| 1 | Gradient activation prior score (batch-normalized) |
| Ch | Name | Variables |
|---|---|---|
| 0 | Respiratory (12) | O 2 Saturation, RR, FiO 2 , Intubated, Chest Tube, PO , PCO , pH Blood ∗ , Total CO , Base Excess ∗ , Propofol ∗ , Fentanyl ∗ |
| 1 | Cardiovascular (27) | MBP, HR, SBP, DBP, Vasopressin, Neosynephrine, Nitroglycerine, Amiodarone, Norepinephrine, Epinephrine, Milrinone, Nitroprusside, Troponin T † , Troponin I, NTproBNP, Hydralazine, Metoprolol, CRR, Lactate ∗ , Dopamine ∗ , Furosemide ∗ , Heparin ∗ , Normal Saline ∗ , Lactated Ringers ∗ , Colloid ∗ , OR/PACU Crystalloid ∗ , Weight ∗ |
| 2 | Coagulation (7) | PTT, Fibrinogen, Heparin ∗ , Platelet Count ∗ , Fresh Frozen Plasma ∗ , INR ∗ , PT ∗ |
| 3 | Liver (13) | Bilirubin (Total), Bilirubin (Direct), Bilirubin (Indirect), ALP, ALT, AST, Albumin, Ammonia, LDH ∗ , Albumin 25% ∗ , Albumin 5% ∗ , INR ∗ , PT ∗ |
| 4 | Renal (16) | Creatinine Blood, Creatinine Urine, BUN, Urine, SG Urine, pH Urine, Ultrafiltrate † , Furosemide ∗ , Dopamine ∗ , Normal Saline ∗ , Lactated Ringers ∗ , Colloid ∗ , OR/PACU Crystalloid ∗ , Weight ∗ , Albumin 25% ∗ , Albumin 5% ∗ |
| 5 | Neurological (9) | GCS_eye, GCS_verbal, GCS_motor, Midazolam, Lorazepam, Hydromorphone, Morphine Sulfate, Propofol ∗ , Fentanyl ∗ |
| MIMIC-III ( ) | MIMIC-IV ( ) | |||||
| Backbone | Dead | Dead% | Dead | Dead% | ||
| Discrete | 26 | 2.0% | 471 | 40 | 3.1% | 554 |
| Discrete+PT | 54 | 4.2% | 524 | 21 | 1.6% | 537 |
| HyMaTE | 104 | 8.1% | 413 | 36 | 2.8% | 458 |
| BAT | 154 | 12.0% | 381 | 251 | 19.6% | 365 |
| STraTS | 295 | 23.0% | 467 | 311 | 24.3% | 404 |
| Rater | 1 | 2 | 3 | 4 | 5 | Mean |
|---|---|---|---|---|---|---|
| Rater 1 | 0 | 5 | 2 | 6 | 17 | 4.17 |
| Rater 2 | 0 | 1 | 8 | 9 | 12 | 4.07 |
| Rater 3 | 0 | 2 | 5 | 8 | 15 | 4.20 |
| Rater 4 | 0 | 0 | 3 | 12 | 15 | 4.40 |
| Metric | Actual | Random |
|---|---|---|
| Decoder cosine | ||
| Variable overlap ( ) | ||
| Latent energy correlation | ||
| Latent temporal cosine | ||
| Prediction selector NDCG |
| Stage | Metric | Actual | Random null |
|---|---|---|---|
| A — dictionary | Decoder cosine | ||
| Variable overlap ( ) | |||
| Latent energy correlation | |||
| Latent temporal cosine | |||
| B — selector (dict. fixed) | Top-64 concept Jaccard | ||
| PS-score rank corr. (Spearman) |
| In-distribution | Out-of-distribution | |
| (MIMIC-III) | (MIMIC-IV) | |
| Backbone | ||
| Prediction AUROC | .861 | .864 |
| Faithfulness (AOPC , neighborhood-based) | ||
| TI-SAE (soft) | .149 | .130 |
| TI-SAE (hard) | .134 | .121 |
| Backbone | Dec. cos | Var 2/3 | Energy corr | Temp. cos | PS NDCG |
|---|---|---|---|---|---|
| Disc.+PT | .801 | 88.2% | .230 | .674 | .163 |
| HyMaTE | .731 | 77.5% | .246 | .592 | .155 |
| BAT | .716 | 82.0% | .228 | .583 | .089 |
| STraTS | .712 | 82.0% | .108 | .596 | .099 |
| Random | .029 | 0.7% | .051 | .390 | .055 |
| Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
|---|---|---|---|---|---|
| TI-SAE (soft) | .086 .003 | .076 .003 | .097 .004 | .043 .002 | .016 .001 |
| TI-SAE (hard) | .079 .003 | .068 .002 | .090 .003 | .041 .002 | .016 .001 |
| Attention | .053 .002 | .019 .002 | .051 .003 | – | .016 .001 |
| IG | .060 .003 | .051 .002 | .049 .003 | .031 .003 | .067 .003 |
| TSR | .063 .003 | .054 .003 | .062 .003 | .037 .003 | .071 .003 |
| DynaMask | .019 .002 | .032 .002 | .012 .001 | .028 .002 | .034 .002 |
| Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
| AUROC | |||||
| TI-SAE (soft) | .593 .002 | .589 .002 | .613 .002 | .588 .002 | .536 .002 |
| TI-SAE (hard) | .569 .002 | .568 .002 | .580 .002 | .567 .002 | .535 .002 |
| Attention | .570 .002 | .539 .002 | .554 .002 | – | .552 .002 |
| IG | .545 .002 | .548 .002 | .596 .002 | .583 .002 | .576 .002 |
| TSR | .566 .002 | .573 .002 | .625 .002 | .608 .002 | .594 .002 |
| Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
|---|---|---|---|---|---|
| TI-SAE (soft) | .197 .005 | .071 .003 | .118 .003 | .074 .004 | .067 .003 |
| TI-SAE (hard) | .172 .004 | .066 .003 | .102 .003 | .081 .003 | .053 .002 |
| Attention | .191 .006 | .068 .004 | .143 .004 | – | .038 .002 |
| IG | .112 .005 | .061 .003 | .066 .004 | .091 .005 | .095 .003 |
| TSR | .112 .005 | .060 .004 | .068 .003 | .100 .005 | .100 .003 |
| DynaMask | .070 .003 | .068 .003 | .098 .003 | .043 .003 | .252 .004 |
| Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
|---|---|---|---|---|---|
| TI-SAE (soft) | .092 .003 | .071 .003 | .099 .004 | .038 .003 | .015 .003 |
| TI-SAE (hard) | .084 .003 | .066 .003 | .090 .004 | .047 .003 | .013 .001 |
| Attention | .085 .004 | .033 .003 | .082 .004 | – | .019 .001 |
| IG | .100 .004 | .086 .003 | .098 .005 | .069 .004 | .106 .004 |
| TSR | .098 .004 | .085 .003 | .098 .005 | .071 .004 | .105 .004 |
| DynaMask | .080 .003 | .043 .002 | .041 .003 | .049 .003 | .076 .003 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .082 | .166 | .200 | .176 | .146 | .149 | .004 | .009 | .019 | .017 | .011 |
| TI-SAE (hard) | .074 | .149 | .168 | .160 | .150 | .134 | .007 | .021 | .028 | .014 | .009 |
| Attention | .045 | .091 | .131 | .132 | .128 | .100 | .091 | .059 | .045 | .033 | .021 |
| IG | .057 | .082 | .101 | .108 | .110 | .086 | .164 | .123 | .072 | .055 | .050 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .033 | .080 | .109 | .129 | .137 | .091 | .022 | .002 | .003 | .001 | .003 |
| TI-SAE (hard) | .032 | .071 | .093 | .110 | .123 | .081 | .022 | .008 | .004 | .003 | .015 |
| Attention | .012 | .030 | .042 | .052 | .060 | .036 | .250 | .244 | .220 | .171 | .117 |
| IG | .035 | .062 | .086 | .103 | .113 | .076 | .065 | .052 | .041 | .036 | .037 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .082 | .141 | .172 | .180 | .182 | .143 | .071 | .053 | .041 | .026 | .009 |
| TI-SAE (hard) | .065 | .124 | .156 | .170 | .165 | .127 | .048 | .040 | .057 | .051 | .036 |
| Attention | .041 | .086 | .115 | .136 | .140 | .096 | .035 | .042 | .027 | .017 | .015 |
| IG | .049 | .079 | .094 | .108 | .116 | .085 | .118 | .088 | .050 | .030 | .018 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .022 | .052 | .076 | .091 | .109 | .065 | .065 | .067 | .064 | .067 | .052 |
| TI-SAE (hard) | .021 | .055 | .080 | .099 | .121 | .070 | .055 | .039 | .050 | .048 | .055 |
| Attention | – | – | – | – | – | – | – | – | – | – | – |
| IG | .023 | .055 | .064 | .075 | .089 | .057 | .095 | .079 | .056 | .031 | .024 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .004 | .007 | .023 | .024 | .025 | .015 | .089 | .080 | .070 | .060 | .044 |
| TI-SAE (hard) | .003 | .008 | .017 | .021 | .032 | .013 | .080 | .072 | .059 | .051 | .045 |
| Attention | .004 | .008 | .008 | .010 | .020 | .010 | .089 | .098 | .090 | .083 | .073 |
| IG | .037 | .058 | .073 | .084 | .084 | .064 | .051 | .037 | .021 | .014 | .008 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .034 | .040 | .036 | .046 | .062 | .041 | .064 | .046 | .037 | .030 | .020 |
| TI-SAE (hard) | .032 | .043 | .037 | .042 | .051 | .039 | .060 | .050 | .043 | .034 | .015 |
| Attention | .011 | .036 | .050 | .051 | .055 | .037 | .047 | .025 | .016 | .015 | .013 |
| IG | .021 | .038 | .057 | .071 | .086 | .050 | .016 | .012 | .024 | .023 | .017 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .010 | .025 | .043 | .057 | .068 | .037 | .031 | .019 | .013 | .006 | .001 |
| TI-SAE (hard) | .007 | .022 | .038 | .055 | .061 | .034 | .033 | .023 | .016 | .007 | .003 |
| Attention | .012 | .006 | .000 | .010 | .025 | .002 | .076 | .050 | .047 | .041 | .029 |
| IG | .010 | .022 | .034 | .049 | .058 | .032 | .035 | .017 | .007 | .003 | .003 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .112 | .129 | .142 | .138 | .130 | .124 | .039 | .029 | .023 | .018 | .008 |
| TI-SAE (hard) | .117 | .131 | .147 | .149 | .144 | .130 | .028 | .030 | .017 | .004 | .005 |
| Attention | .053 | .064 | .074 | .074 | .085 | .067 | .016 | .024 | .017 | .011 | .008 |
| IG | .033 | .045 | .058 | .069 | .082 | .054 | .055 | .045 | .027 | .019 | .012 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .011 | .017 | .024 | .028 | .038 | .022 | .040 | .026 | .019 | .010 | .002 |
| TI-SAE (hard) | .010 | .017 | .025 | .025 | .034 | .020 | .036 | .028 | .018 | .011 | .003 |
| Attention | – | – | – | – | – | – | – | – | – | – | – |
| IG | .016 | .023 | .029 | .032 | .036 | .025 | .037 | .023 | .016 | .011 | .008 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .014 | .025 | .026 | .021 | .032 | .023 | .135 | .143 | .131 | .127 | .108 |
| TI-SAE (hard) | .010 | .021 | .032 | .028 | .027 | .023 | .131 | .134 | .121 | .122 | .110 |
| Attention | .012 | .012 | .015 | .029 | .040 | .020 | .120 | .106 | .094 | .080 | .061 |
| IG | .044 | .052 | .058 | .073 | .085 | .060 | .022 | .018 | .011 | .006 | .002 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .066 | .112 | .126 | .127 | .128 | .106 | .063 | .049 | .041 | .029 | .022 |
| TI-SAE (hard) | .054 | .090 | .102 | .110 | .110 | .088 | .050 | .037 | .019 | .006 | .006 |
| Attention | .025 | .051 | .064 | .078 | .088 | .056 | .003 | .010 | .010 | .008 | .004 |
| IG | .037 | .054 | .058 | .059 | .060 | .051 | .007 | .017 | .018 | .015 | .010 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .050 | .095 | .117 | .123 | .129 | .097 | .008 | .011 | .015 | .009 | .006 |
| TI-SAE (hard) | .044 | .083 | .099 | .111 | .116 | .086 | .008 | .001 | .001 | .004 | .020 |
| Attention | .003 | .005 | .003 | .007 | .010 | .005 | .119 | .103 | .096 | .090 | .083 |
| IG | .049 | .062 | .071 | .081 | .090 | .068 | .039 | .034 | .031 | .026 | .023 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .025 | .057 | .077 | .069 | .058 | .055 | .024 | .010 | .007 | .014 | .010 |
| TI-SAE (hard) | .025 | .051 | .066 | .069 | .072 | .054 | .028 | .025 | .017 | .003 | .003 |
| Attention | .033 | .035 | .039 | .043 | .048 | .038 | .048 | .044 | .038 | .026 | .016 |
| IG | .030 | .033 | .036 | .039 | .041 | .035 | .055 | .049 | .035 | .024 | .018 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .005 | .019 | .028 | .038 | .051 | .025 | .084 | .067 | .059 | .047 | .040 |
| TI-SAE (hard) | .006 | .018 | .030 | .041 | .057 | .028 | .084 | .074 | .061 | .046 | .035 |
| Attention | – | – | – | – | – | – | – | – | – | – | – |
| IG | .002 | .011 | .020 | .033 | .042 | .018 | .143 | .111 | .088 | .063 | .045 |
| Comprehensiveness | Sufficiency | ||||||||||
| Method | @.10 | @.20 | @.30 | @.40 | @.50 | AOPC | @.10 | @.20 | @.30 | @.40 | @.50 |
| MIMIC-III | |||||||||||
| TI-SAE (soft) | .003 | .008 | .013 | .014 | .017 | .010 | .075 | .074 | .075 | .071 | .061 |
| TI-SAE (hard) | .003 | .008 | .010 | .014 | .016 | .010 | .076 | .074 | .074 | .074 | .066 |
| Attention | .002 | .004 | .009 | .013 | .020 | .008 | .073 | .064 | .057 | .047 | .035 |
| IG | .055 | .059 | .060 | .058 | .059 | .057 | .013 | .013 | .010 | .007 | .005 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .756 /.384 | .631 /.303 | .707 /.356 | .598/.249 | .516/.216 |
| TI-SAE (hard) | .639 /.313 | .625 /.289 | .621/.301 | .575/.240 | .506/.218 |
| Attention | .615/.258 | .468/.189 | .568/.231 | – | .610/.249 |
| IG | .567/.254 | .555/.245 | .641/.278 | .620 /.265 | .621/.251 |
| TSR | .594/.279 | .586/.262 | .673 /.320 | .649 /.296 | .636 /.264 |
| DynaMask | .507/.212 | .507/.205 | .492/.189 | .491/.189 | .492/.189 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .705 /.362 | .569/.256 | .611 /.255 | .638 /.296 | .545/.248 |
| TI-SAE (hard) | .632/.314 | .533/.245 | .556/.232 | .621 /.276 | .561/.252 |
| Attention | .668 /.312 | .624 /.268 | .523/.203 | – | .497/.190 |
| IG | .562/.231 | .530/.218 | .595/.246 | .572/.230 | .541/.208 |
| TSR | .578/.244 | .552/.232 | .619 /.269 | .592/.248 | .564 /.218 |
| DynaMask | .506/.199 | .491/.193 | .495/.188 | .515/.202 | .484/.203 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .699 /.360 | .514/.197 | .623 /.295 | .633 /.279 | .559/.242 |
| TI-SAE (hard) | .659 /.335 | .508/.193 | .614/.275 | .612 /.266 | .555/.241 |
| Attention | .617/.282 | .453/.184 | .560/.243 | – | .637 /.260 |
| IG | .543/.222 | .543/.234 | .587/.257 | .591/.243 | .593/.239 |
| TSR | .569/.242 | .581 /.248 | .626 /.290 | .611/.264 | .629/.261 |
| DynaMask | .510/.203 | .505/.204 | .492/.189 | .493/.189 | .492/.189 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .312/.181 | .629 /.263 | .588/.255 | .525/.219 | .601/.244 |
| TI-SAE (hard) | .417/.217 | .572/.237 | .541/.244 | .499/.217 | .606 /.249 |
| Attention | .523/.220 | .619 /.260 | .577/.236 | – | .477/.181 |
| IG | .516/.214 | .535/.232 | .606 /.263 | .564 /.239 | .552/.220 |
| TSR | .544 /.230 | .565/.249 | .644 /.298 | .587 /.259 | .580/.237 |
| DynaMask | .500/.198 | .500/.199 | .495/.188 | .474/.186 | .494/.188 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .533/.213 | .608 /.261 | .593/.261 | .577/.241 | .522/.195 |
| TI-SAE (hard) | .534/.211 | .596/.251 | .616/.262 | .558/.232 | .515/.194 |
| Attention | .464/.175 | .463/.175 | .573/.203 | – | .627 /.249 |
| IG | .553/.220 | .574/.223 | .593/.227 | .590 /.240 | .605 /.219 |
| TSR | .573 /.245 | .584/.228 | .618 /.249 | .615 /.258 | .603/.218 |
| DynaMask | .506/.200 | .515/.208 | .494/.183 | .495/.184 | .497/.192 |
| Method | Discrete | Disc.+PT | HyMaTE | BAT | STraTS |
|---|---|---|---|---|---|
| TI-SAE (soft) | .556 /.215 | .581/.247 | .558/.234 | .559/.220 | .471/.215 |
| TI-SAE (hard) | .534/.210 | .572/.240 | .534/.228 | .537/.224 | .464/.214 |
| Attention | .532/.200 | .606 /.239 | .524/.189 | – | .462/.171 |
| IG | .527/.210 | .553/.219 | .552/.191 | .564 /.215 | .544/.193 |
| TSR | .542/.219 | .571/.229 | .569 /.199 | .592 /.235 | .554 /.197 |
| DynaMask | .530/.195 | .503/.191 | .495/.182 | .488/.182 | .521/.201 |
| Backbone | Layer | Layer | Layer | Layer |
|---|---|---|---|---|
| Discrete | .149 | .121 | .114 | .105 |
| Discrete+PT | .029 | .037 | .091 | .216 |
| HyMaTE | .109 | .143 | .121 | – |
| STraTS | .002 | .003 | .003 | .015 |
| BAT | .061 | .065 | – | – |
| Discrete | Disc.+PT | HyMaTE | BAT | STraTS | |
|---|---|---|---|---|---|
| Activation (energy) | .037 | .008 | .028 | .006 | .008 |
| Gradient Activation | .035 | .027 | .032 | .022 | .014 |
| Prediction selector (hard) | .079 | .068 | .090 | .035 | .020 |
| Prediction selector (soft) | .086 | .076 | .097 | .043 | .016 |
| Faithfulness | Plausibility | ||||
|---|---|---|---|---|---|
| Condition | AOPC | [email protected] | AUROC | AUPRC | |
| Full TI-SAE | .149 | .004 | .756 | .386 | |
| Temporal | .145 | .006 | .777 | .418 | |
| Channel org | .184 | .003 | .734 | .358 | |
| Var routing | .061 | .052 | .629 | .280 | |
| Standard SAE | .023 (.155) | .126 (.041) | .487 (.568) | .189 (.224) | |
| Default | ||
|---|---|---|
| k=113 | Temperature, CRP, Eosinophils [all Infection] | Hypothermia w/ Inflammatory Markers |
| k=1246 | TPN [Other], RDW [Heme], Gastric [Other] | TPN-Dependent Feeding Intolerance |
| k=1119 | GCS_verbal, GCS_eye [Neuro], PO2 [Resp] | Coma with Mechanical Ventilation |
| k=931 | BUN, Creatinine, Creatinine Urine [all Renal] | Chronic Kidney Disease with Azotemia |
| k=886 | BUN, Creatinine, Urine [all Renal] | Acute Kidney Injury with Oliguria |
| No channel organization | ||
| Island shape | AOPC | Plausibility |
|---|---|---|
| Symmetric Gaussian | ||
| Learned asymmetric | ||
| Fixed asymmetric ( ) |
| Utilization | Faithfulness | Plausibility | |||||
|---|---|---|---|---|---|---|---|
| Dead% | 1% | AOPC | [email protected] | AUROC | AUPRC | ||
| 320 | 0.3% | 281 | .137 | .030 | .711 | .353 | |
| 640 | 0.0% | 417 | .165 | .005 | .734 | .375 | |
| 1280 | 2.0% | 469 | .149 | .004 | .756 | .386 | |
| 2560 | 23.5% | 524 | .143 | .005 | .700 | .333 | |
| Utilization | Faithfulness | Plausibility | ||||||
|---|---|---|---|---|---|---|---|---|
| Dead% | 1% | AOPC | [email protected] | AUROC | AUPRC | |||
| 32 | 0.6% | 26.2% | 136 | .134 | .015 | .713 | .333 | |
| 64 | 1.2% | 8.2% | 259 | .151 | .007 | .716 | .353 | |
| 128 | 2.5% | 2.0% | 469 | .149 | .004 | .756 | .386 | |
| 256 | 5.0% | 0.1% | 767 | .141 | .000 | .714 | .364 | |
| 512 | 10.0% | 0.2% | 1022 | .128 | .019 | .763 | .395 | |
| Faithfulness | Plausibility | |||
|---|---|---|---|---|
| AOPC | [email protected] | AUROC | AUPRC | |
| 1 | .131 | .021 | .723 | .346 |
| 2 | .152 | .004 | .731 | .341 |
| 4 | .149 | .004 | .756 | .386 |
| 8 | .144 | .005 | .703 | .357 |
| AOPC | AUROC | AUPRC | ||||
|---|---|---|---|---|---|---|
| Method | Orig | LN | Orig | LN | Orig | LN |
| TI-SAE (soft) | .015 | .044 | .516 | .652 | .216 | .296 |
| TI-SAE (hard) | .013 | .042 | .506 | .637 | .218 | .285 |
| Attention | .010 | .046 | .611 | .542 | .250 | .220 |
| IG | .064 | .048 | .622 | .608 | .252 | .244 |
| TSR | .061 | .050 | .637 | .625 | .265 | .260 |
| Mortality | LOS 7d | Readmission | ||||
|---|---|---|---|---|---|---|
| Backbone | III | IV | III | IV | III | IV |
| Discrete | .861 | .867 | .832 | .824 | .632 | .652 |
| Discrete+PT | .861 | .865 | .834 | .827 | .618 | .651 |
| HyMaTE | .861 | .872 | .830 | .829 | .638 | .649 |
| STraTS | .873 | .867 | .843 | .823 | .644 | .649 |
| BAT | .858 | .868 | .826 | .822 | .612 | .648 |
| Concept | Actionability | Faithfulness | |
|---|---|---|---|
| Clinician majority | 39 / 4 / 7 | 23 / 9 / 18 | 4 / 4 / 42 |
| LLM judge | 33 / 15 / 2 | 32 / 18 / 0 | 15 / 11 / 24 |
| Rater | Concept | Actionability | Faithfulness |
|---|---|---|---|
| Rater 1 | 38 / 8 / 4 | 24 / 5 / 21 | 3 / 2 / 45 |
| Rater 2 | 30 / 20 / 0 | 22 / 22 / 6 | 21 / 29 / 0 |
| Rater 3 | 30 / 3 / 17 | 23 / 15 / 12 | 14 / 13 / 23 |
| Rater 4 | 36 / 9 / 5 | 25 / 10 / 15 | 6 / 3 / 41 |
| TI-SAE condition | vs. IG | vs. Attention | vs. TSR |
|---|---|---|---|
| Label removed | 24–23 (3) | 16–29 (5) | 25–23 (2) |
| + cross-episode summary, no label | 31–19 (0) | 26–20 (4) | 27–21 (2) |
| Baseline condition | vs. IG | vs. Attention | vs. TSR |
|---|---|---|---|
| LLM-summarized (per-episode) | 11–39 (0) | 11–37 (2) | 18–30 (2) |
| Channel-grouped, no label | 27–21 (2) | 29–20 (1) | 28–20 (2) |
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 329 papers