Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms
Organizations: Department of Electrical Engineering, University of South Florida, Tampa, FL 33620 USA · Department of Computer Engineering, California Polytechnic State University, San Luis Obispo, CA 93407 USA
Abstract
Human activity recognition (HAR) has become a key component of intelligent systems for healthcare monitoring, assistive living, smart environments, and human-computer interaction. Although deep learning has substantially improved HAR performance on multivariate sensor data, the resulting models often remain opaque, limiting trust, reliability, and real-world deployment. Explainable artificial intelligence (XAI) has therefore emerged as a critical direction for making HAR systems more transparent and human-centered. This paper presents a comprehensive review of explainable HAR methods across wearable, ambient, physiological, and multimodal sensing settings. We introduce a unified perspective that separates conceptual dimensions of explainability from algorithmic explanation mechanisms, reducing ambiguities in prior surveys. Building on this distinction, we present a mechanism-centric taxonomy of XAI-HAR methods covering major explanation paradigms. The review examines how these methods address the temporal, multimodal, and semantic complexities of HAR, and summarize their interpretability objectives, explanation targets, and limitations. In addition, we discuss current evaluation practices, highlight key challenges in achieving reliable and deployable XAI-HAR, and outline directions toward trustworthy activity recognition systems that better support human understanding and decision-making.
Figures & tables
| Section | XAI Method Category | Primary HAR Question Answered | Explanation Target | Key Limitation in HAR Context |
|---|---|---|---|---|
| IV-A | Perturbation & Sampling (LIME, SHAP, Anchors, PFI) | Which input features or temporal segments influenced this prediction? | Local attribution; some global importance | Weak temporal consistency; sensitive to correlated features |
| IV-B | Gradient & Path-Based (Saliency, IG, Grad-CAM) | Which timesteps or channels were most influential? | Local, model-internal (input/activation space) | Correlation-based, not causal; sensitive to gradients and baselines |
| IV-C | Surrogate & Symbolic Methods | How can this decision be approximated in an interpretable form? | Rules, trees, symbolic/temporal logic | Lose fine-grained temporal dynamics and sequential dependencies |
| IV-D | Graph-Structured Methods | How do sensors or entities interact structurally? | Nodes, edges, subgraphs (spatio-temporal) | Limited empirical validation and scalability in HAR |
| IV-E | Generative Methods | How can sensor data generate semantic explanations of activities? | Generated semantic or visual artifacts | High complexity; difficult to ver- ify faithfulness to underlying model reasoning |
| IV-F | Concept / Counterfactual / Prototype | What concepts, counterfactual alternatives, or prototypes define this activity? | Concepts, counterfactuals, prototype similarity | Limited coverage; weak temporal consistency |