Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms

Mainak Kundu; Catherine Chen; Rifatul Islam; Ismail Uysal; Ria Kanjilal

cs.LGApr 10, 2026

Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms

Authors: Mainak Kundu, Catherine Chen, Rifatul Islam, Ismail Uysal, Ria Kanjilal

Organizations: Department of Electrical Engineering, University of South Florida, Tampa, FL 33620 USA · Department of Computer Engineering, California Polytechnic State University, San Luis Obispo, CA 93407 USA

Abstract

Human activity recognition (HAR) has become a key component of intelligent systems for healthcare monitoring, assistive living, smart environments, and human-computer interaction. Although deep learning has substantially improved HAR performance on multivariate sensor data, the resulting models often remain opaque, limiting trust, reliability, and real-world deployment. Explainable artificial intelligence (XAI) has therefore emerged as a critical direction for making HAR systems more transparent and human-centered. This paper presents a comprehensive review of explainable HAR methods across wearable, ambient, physiological, and multimodal sensing settings. We introduce a unified perspective that separates conceptual dimensions of explainability from algorithmic explanation mechanisms, reducing ambiguities in prior surveys. Building on this distinction, we present a mechanism-centric taxonomy of XAI-HAR methods covering major explanation paradigms. The review examines how these methods address the temporal, multimodal, and semantic complexities of HAR, and summarize their interpretability objectives, explanation targets, and limitations. In addition, we discuss current evaluation practices, highlight key challenges in achieving reliable and deployable XAI-HAR, and outline directions toward trustworthy activity recognition systems that better support human understanding and decision-making.

Figures & tables

Fig. 1: Canonical explainable HAR pipeline illustrating the separation between training and inference and two complementary explainability paradigms. During inference, explainability is realized either intrinsically within the HAR model or post-hoc via an external XAI module, with both paths converging at a unified activity detection and explanation stage.

Fig. 2: Structural overview of XAI-HAR illustrating two complementary layers: (a) the conceptual layer, which defines what aspects of model behavior should be explained; and (b) the algorithmic layer, which categorizes how explanations are generated through different families of methods. The conceptual layer guides the selection and design of algorithmic approaches.

Fig. 3: Visualization of the local contribution of EEG features using the LIME model for classifying a single test instance (predicted class = Working activity) with the Random Forest model. The red-marked cells represent the features that contributed most to the classification of the Working activity [ 39 ] .

Fig. 4: SHAP interaction plot showing the most influential features and their pairwise interactions affecting CNN–LSTM–based activity predictions [ 33 ] .

Fig. 5: (a) Overview of the EfficientGCN pipeline illustrating the variables used to compute faithfulness and stability, with perturbations applied during the data preprocessing stage. (b) (Left) CAM, Grad-CAM, and random baseline attributions for a representative ‘standing up’ activity instance (class 8), averaged across all frames and normalized. The color gradient represents attribution intensity, ranging from blue (0) to red (1). (Right) Corresponding numerical attribution scores, where k denotes the body point index [ 29 ] .

Fig. 6: (a) Output of the original GNNExplainer, limited to arc importance, where the thickness of each arrow represents the importance value associated with the arc. (b) Output of the adapted version of GNNExplainer on the same input example shown in (a) [ 3 ] .

Section	XAI Method Category	Primary HAR Question Answered	Explanation Target	Key Limitation in HAR Context
IV-A	Perturbation & Sampling (LIME, SHAP, Anchors, PFI)	Which input features or temporal segments influenced this prediction?	Local attribution; some global importance	Weak temporal consistency; sensitive to correlated features
IV-B	Gradient & Path-Based (Saliency, IG, Grad-CAM)	Which timesteps or channels were most influential?	Local, model-internal (input/activation space)	Correlation-based, not causal; sensitive to gradients and baselines
IV-C	Surrogate & Symbolic Methods	How can this decision be approximated in an interpretable form?	Rules, trees, symbolic/temporal logic	Lose fine-grained temporal dynamics and sequential dependencies
IV-D	Graph-Structured Methods	How do sensors or entities interact structurally?	Nodes, edges, subgraphs (spatio-temporal)	Limited empirical validation and scalability in HAR
IV-E	Generative Methods	How can sensor data generate semantic explanations of activities?	Generated semantic or visual artifacts	High complexity; difficult to ver- ify faithfulness to underlying model reasoning
IV-F	Concept / Counterfactual / Prototype	What concepts, counterfactual alternatives, or prototypes define this activity?	Concepts, counterfactuals, prototype similarity	Limited coverage; weak temporal consistency

TABLE I: Summary of XAI methods in HAR (Section IV ), highlighting the key question, explanation target, and limitations.

Explore similar work

ActivityNarrated: An Open-Ended Narrative Paradigm for Wearable Human Activity Understanding
Apr 1, 2026Lala Shakti Swarup Ray, Mengxi Liu, Alcina Pinto +4Human Activity Recognition Cross-Modal Alignment
DFKI, Germany · RPTU
EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition
Sep 13, 2026He Zhang, Siyu Yuan, Siyu Liu +2Human Activity Recognition Wearable Sensing
Northwestern Polytechnical University Xi’an, Shaanxi, China · RPTU University Kaiserslautern-Landau Kaiserslautern, Rheinland Pfalz, Germany · Xi’an University of Technology Xi’an, Shaanxi, China
You Don't Need To Train: Agentic Heuristic Learning Studio for Executable Human Activity Recognition
Sep 13, 2026Siyu Yuan, He Zhang, Sizhen Bian +1Human Activity Recognition Program Synthesis
RPTU University Kaiserslautern-Landau Kaiserslautern, Rheinland Pfalz, Germany · Northwestern Polytechnical University Xi’an, Shaanxi, China

Apr 1, 2026J/K move · Enter open · S save

ActivityNarrated: An Open-Ended Narrative Paradigm for Wearable Human Activity Understanding

Lala Shakti Swarup Ray, Mengxi Liu, Alcina Pinto, Deepika Gurung+3

DFKI, Germany · RPTU

Human Activity Recognition Cross-Modal Alignment Wearable Sensing

Open paper AskPDF arXiv

Wearable human activity recognition (HAR) has made steady progress, yet much of this progress remains grounded in fixed-window, closed-set classification benchmarks. This formulation is poorly matched to everyday behavior, where activities are open-ended, unscripted, personalized, variable in duration, and often compositional. To address this mismatch, we introduce ActivityNarrated, an open-ended narrative paradigm for language-grounded wearable activity understanding. We formulate this setting as dense sensor signal captioning with a comprehensive benchmark protocol that measures temporal localization, caption quality, sensor-language alignment, conventional closed-set classification as a downstream diagnostic, and additional robustness measures. We further present ActNarrator, a 3-stage architecture that discretizes continuous IMU signals into reusable motion tokens and uses an external frozen small language model to generate open-vocabulary activity captions. Experiments show that our method provides high quality dense sensor captioning with superior adaptivity and robustness, enabling various downstream tasks by turning sensor-based human activity understanding into sensor-grounded text-level reasoning. This includes downstream classification where ActNarrator outperforms state-of-the-art HAR models by 3.8 - 31.6 % in Macro-F1. This paradigm also enables novel activity understanding capabilities such as complex question-answering over long time horizons.

Abstract

Explore similar work

ActivityNarrated: An Open-Ended Narrative Paradigm for Wearable Human Activity Understanding

EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition

You Don't Need To Train: Agentic Heuristic Learning Studio for Executable Human Activity Recognition