Organizations: Signal and System Theory Group, Paderborn University, Paderborn, Germany · Department of Medicine I, Ludwig Maximilian University, Munich, Germany · Konrad Zuse School of Excellence in Reliable AI, Munich, Germany · Institute of Sports Medicine, Paderborn University, Paderborn, Germany · Division of Sports Neurology and Neurosciences, Mass General Brigham, Boston, USA · Division of Epilepsy and Clinical Neurophysiology, Boston Children’s Hospital, Harvard Medical School, Boston, USA
The use of wearables in medicine and wellness, enabled by AI-based models, offers tremendous potential for real-time monitoring and interpretable event detection. Explainable AI (XAI) is required to assess what models have learned and build trust in model outputs, for patients, healthcare professionals, model developers, and domain experts alike. Explaining AI decisions made on time-series data recorded by wearables is especially challenging due to the data's complex nature and temporal dependencies. Too often, explainability using interpretable features leads to performance loss. We propose a novel XAI method that combines explanation spaces and concept-based explanations to explain AI predictions on time-series data. By using Inherently Interpretable Components (IICs), which encapsulate domain-specific, interpretable concepts within a custom explanation space, we preserve the performance of models trained on time series while achieving the interpretability of concept-based explanations based on extracted features. Furthermore, we define a domain-specific set of IICs for wearable-based health monitoring and demonstrate their usability in real applications, including state assessment and epileptic seizure detection.
Figures & tables
Figure 1: Visualization of decomposition and the extended network pipeline. The original time series x gets decomposed into components Cx via F . The inverse transformation F−1(⋅) , combined with the original time series network M , results in a new, extended model M^ that uses interpretable components as inputs, without retraining.
Figure 2: Exemplary decomposition of an accelerometer signal into IICs.
Figure 3: Exemplary decomposition of a heart rate signal into IICs.
Figure 4: Exemplary decomposition of an EDA signal into IICs.
Figure 5: Exemplary decomposition of a temperature signal into IICS.
Accuarcy
F1
IIC
99.0%±0.6%
99.0%±0.6%
FCSHAP
95.4%±0.8%
96.1%±0.7%
LCBM
81.1%±2.5%
77.7%±0.7 %
Table 1: Evaluation of classification performance for the state assessment task based on wearable autonomic data (no accelerometry data included).
Figure 6: Exemplary stepping segment that was correctly classified by all approaches, with corresponding explanations (oB denotes over baseline).
IIC
LCBM
FCSHAP
k
Component
Import.
Feature
Import.
Feature
Import.
1
HR Mean oB
0.516
HR Mean oB
0.377
HR Min
0.302
2
EDA Tonic Mean oB
0.137
TEMP Rising
0.126
HR Mean
0.263
3
TEMP Mean oB
0.074
EDA Tonic Mean oB
0.118
HR Max
0.215
4
HR Variability
0.055
TEMP Falling
0.108
TEMP Mean
0.036
5
TEMP Rising
0.004
EDA Tonic Changes
0.074
EDA Max
0.029
Table 2: Top 5 important components and features from the IIC, LCBM, and FCSHAP approaches (oB denotes over baseline).
Figure 7: Analysis of the component-wise distribution of scalar summaries, considered from the IIC approach for the state assessment task. We show distributions for true positives (stepping) and true negatives (supine), separately.
Fidelity
k
IIC
LCBM
FCSHAP
1
51.2%
49.5%
27.9%
2
52.0%
49.5%
29.5%
3
50.2%
49.5%
55.5%
Sufficiency
0.2%
0.7%
3.8%
Table 3: Evaluation of fidelity and sufficiency by measuring the flip rate for the state assessment task.
Accuracy
F1
IIC
87.8%±1.4%
86.7%±2.1%
FCSHAP
82.6%±2.7%
81.6%±1.9%
LCBM
63.6%±2.5%
63.7%±2.7%
Table 4: Comparison of classification performance for the seizure detection task.
Figure 8: Exemplary seizure segment that was correctly classified as a seizure by both approaches, with corresponding explanations (oB denotes over baseline).
Figure 9: Exemplary interictal segment that was classified as a seizure segment by both approaches, with corresponding explanations (oB denotes over baseline).
IIC
FCSHAP
k
Component
Import.
Feature
Import.
1
ACC Outlier
0.421
ACC Std
0.233
2
HR Mean oB
0.342
HR Max
0.176
3
EDA Tonic Mean oB
0.262
HR Min
0.101
4
HR Variability
0.220
HR Mean
0.098
5
ACC Activity
0.214
HR Std
0.065
Table 5: Top 10 components and features from the IIC and FCSHAP approaches in decreasing order (oB denotes over baseline).
Figure 10: Analysis of the component-wise distribution of scalar summaries, considered from the IIC approach for the seizure detection task. We show distributions for true positives (seizure) and true negatives (no seizure), separately.
Fidelity
k
IIC
FCSHAP
1
9.7%
36.1%
2
16.4%
41.0%
3
15.5%
42.0%
4
19.9%
40.0%
5
33.9%
42.0%
Table 6: Evaluation of fidelity and sufficiency by measuring the flip rate for the seizure detection task.
Properties
Proposed IIC
LCBM
FCSHAP
Time-Series Native
✓
×
×
Interpretable model
×
✓
×
Local Explanations
✓
×
✓
Global Explanations
✓
✓
✓
Label-independent
✓
×
×
Model-independent
✓
×
×
Table 7: Comparison of the approaches and their properties.
Explainable time series forecasting makes predictions over time interpretable through attributions, structured models, causal reasoning, and counterfactuals.
Department of Electrical Engineering, University of South Florida, Tampa, FL 33620 USA · Department of Computer Engineering, California Polytechnic State University, San Luis Obispo, CA 93407 USA
Department of Chemistry, Chemical Physics Theory Group, University of Toronto, Toronto, Canada · Vector Institute for Artificial Intelligence, Toronto, Canada · Institute for AI in Medicine (IKIM), Charité – Universitätsmedizin Berlin, Germany +5