cs.CVNov 4, 2026

TraCE: Trajectory-Aware Concepts for Explainable Video Understanding

Authors: Wooil Lee, Junpyo Hong, Jongseo Lee, Tuomas Oikarinen, Tsui-Wei Weng, Jinwoo Choi

Organizations: Kyung Hee University · University of California San Diego

Abstract

Video understanding models often rely on motion, yet their explanations can obscure which motion sources support a prediction. We identify motion-source under-specification as a limitation of existing video XAI: region-centric explanations can be source-ambiguous, while motion vocabularies restricted to single-person pose sequences are source-incomplete. We propose TraCE (Trajectory-Aware Concepts for Explainable Video Understanding), a concept-based framework that discovers source-flexible motion concepts by clustering optical-flow trajectories from training videos. These concepts represent dynamic evidence from people, objects, and interactions, while object and scene concepts account for static context. The resulting typed trajectory/object/scene interface supports class prediction for action recognition and question-conditioned concept matching for video question answering (VQA). We evaluate TraCE across five random seeds on six benchmarks: NTU RGB+D Mutual, Chiral SSv2, UCF-101, KTH, TOMATO, and STAR. Relative to non-interpretable baselines, TraCE maintains competitive task performance, with an average accuracy drop of 1.0 points for action recognition and a nominal gain of 0.4 points for VQA. Motion perturbation and concept-removal tests provide complementary evidence of directional semantic alignment and decision faithfulness, while sample-level interventions illustrate targeted model debugging. Our project page is available at https://leewooil.github.io/TraCE_project_page/.

Figures & tables

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Motion-o: Trajectory-Grounded Video Reasoning

    Mar 19, 2026Bishoy Galoaa, Shayda Moezzi, Xiangyu Bai +1Video ReasoningVLM Reasoning

  2. Evidence-Backed Video Question Answering

    Jul 13, 2026Shijie Wang, Honglu Zhou, Ziyang Wang +5Temporal Video GroundingVideo Question Answering