cs.CVNov 13, 2026

BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations

Authors: Genki Kinoshita, Ko Nishino

Organizations: Graduate School of Informatics, Kyoto University

Abstract

Learning semantically meaningful human movement representations from 3D pose sequences is essential for human behavior modeling. Existing self-supervised methods derive supervision from artificial assumptions, e.g., invariance to augmentation within a certain range or recoverability of masked regions at a certain temporal scale, introducing heuristic hyperparameters that make the learned semantics sensitive to their settings. We propose BAL, a self-supervised framework that instead exploits supervision intrinsic to human movement: the natural ordering structure of everyday behavior. BAL autoregressively predicts in a semantically abstract latent space, analogous to how humans anticipate others' actions rather than their precise joint configurations. Bidirectional autoregressive prediction, both forward and backward in time, fully leverages these ordering constraints while enabling representation extraction using both past and future context at inference. To suppress representation collapse, BAL decomposes each prediction target into a global component capturing sequence-level semantics and a local component capturing context-independent movement. We prove that the local targets retain nontrivial angular diversity unless the representations are completely collapsed, preserving a meaningful training signal. Experiments on large-scale datasets demonstrate that BAL's representations are highly effective for motion captioning, forecasting, and interpolation, outperforming baselines on most evaluation metrics.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Action Motifs: Self-Supervised Hierarchical Representation of Human Body Movements

    Apr 30, 2026Genki Kinoshita, Shu Nakamura, Ryo Kawahara +3Latent Action LearningRepresentation Learning

  2. Fine-grained Human Motion Understanding with Language Models

    Jun 18, 2026Thomas Markhorst, Zhi-Yi Lin, Jouh Yeong Chew +2Human Motion Understanding

  3. Zero-Gated Language-conditioned Human Motion Prediction

    Jun 28, 2026Guanhui Qiao, Lu Zhou, Ding Jiang +1Human Motion Prediction