Motion Representation Learning

Overview

Motion representation learning studies how to encode movement from sources such as body skeletons, wearable sensors, point trajectories, and human–object interactions so models can recognize, predict, generate, or control motion. Approaches range from discrete motion tokens and continuous latent spaces to geometric, graph-based, and frequency-aware representations, trained with masked prediction, contrastive learning, or temporal prediction. Language and video supervision increasingly connect these encodings to semantic descriptions and robot actions, while diffusion and flow-matching models use them for tasks such as text-to-motion generation and motion completion. A central challenge is balancing compact, efficient representations with fine-grained dynamics and meaningful semantics, while ensuring transfer across datasets, body shapes, sensor setups, and real-world conditions.

Latest papers

CardsList

184 papers25 papers in the last four weeks, down 43% on the four weeks before. 0.2% of all new papers.

  1. TOMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting

    Nov 18, 2026Guanzhong Sun, Junyi Ma, Yuxuan Wu +4Human Motion PredictionHand-Object Interaction

  2. ROSE: A Shared Cross-Embodiment Facial-Motion Latent Space for Humanoid Robot Expression

    Nov 16, 2026Yongji Fu, Man Cao, Yi Zhou +3Facial Expression SynthesisHumanoid Robot Control

  3. Does Pose Encoding Matter? Analyzing Geometric Representations for Discrete Motion Vocabulary Learning

    Nov 8, 2026Amir Kimiyaie, Nahal Mafi, Hamed TabkhiTokenizationMotion Representation Learning

  4. A Unified Inertial Encoder for Human and Humanoid Motion

    Nov 6, 2026Yutian Chen, Shibo Zhao, Rocky Duan +5Robot State EstimationInertial Odometry

  5. Compact Motion Embeddings for Feed-forward Unified Human Animation

    Nov 5, 2026Seonwook Park, Hyunsoo Cha, Amrita Mazumdar +5Human Avatar AnimationMotion Representation Learning

  6. FFMT: Frequency-Factorized Multi-Rate Tokenization for Music-to-Dance Generation

    Nov 5, 2026Qirong Guo, Ruijie Li, Zeyan Li +3Autoregressive Motion GenerationHuman Motion Generation

  7. DPAGait: Dynamic Pose-Knowledge-Aware Network for Robust Gait Recognition in the Wild

    Nov 3, 2026Zenan Lin, Junduan Huang, Kun Liu +2Gait RecognitionBiometric Identification

  8. Gnomon4D: Learning Where and When to Focus for Geometric Motion-Grounded World-Action Modeling

    Nov 2, 2026Lingwei Dang, Yan Zhou, Juntong Li +10World Action ModelsRobot Learning from Video

  9. MoPort: A View-Aware 2D Interface for Motion Language Models

    Nov 1, 2026Lixing Tan, Qing Xia, Yuting Guo +2Motion Representation Learning

  10. TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation

    Oct 29, 2026Xinmu Wang, Fumi Honda, Xiang Gao +3Human Motion GenerationCo-Speech Gesture Generation

  11. Learning a Motion Manifold to Constrain Ambiguity in Unsupervised 4D Medical Image Interpolation

    Oct 29, 2026Haojin Li, Huazhu Fu, Heng Li +14D ReconstructionMotion Representation Learning

  12. Geometry-Aware Trajectory State Modeling for Few-Shot Action Recognition

    Oct 28, 2026Zhiqi Miao, Marko Haralović, Stanislav Levendeev +5Few-Shot Action RecognitionVideo Action Recognition

  13. Whole-Body Humanoid Control via Stochastic Motion Tokenization

    Oct 27, 2026Michael Drolet, Oleg Arenz, Jan PetersHumanoid Robot ControlHumanoid Motion Tracking

  14. EXE: Egocentric Cross-Embodiment Pretraining via Factorized Latent Actions for VLA Models

    Oct 24, 2026Qingyuan Cai, Bo Zhang, Yongzhen Huang +1Cross-Embodiment Robot LearningRobot Learning from Video

  15. KiRoSe: Using Kinematic Data for Efficient Segmentation of Robot Demonstrations

    Oct 22, 2026Shuangyu Xie, Kaiyuan Chen, Muhammad Zubair Irshad +3Motion SegmentationTemporal Action Segmentation

  16. PiHand: Latent Hand-Object Interaction Priors for Physics-Based Policy Learning

    Oct 22, 2026Niklas Berndt, Xianghui Xie, Enes Duran +2Hand-Object InteractionRobotic Manipulation

  17. MoRE: Motion-Regularized Encoding for Articulated 3D Generation

    Oct 21, 2026Guantian Zheng, Shan Jin, Tim Tso +1Articulated Object ReconstructionArticulated 3D Asset Generation

  18. SP-HLM: Sparse-Pose-Driven Motion Generation and Whole-Body Control for Humanoid Loco-Manipulation

    Oct 21, 2026Zhaohui Yang, Quanbao Lin, Chengqi Shi +9Whole-Body Humanoid ControlHumanoid Loco-Manipulation

  19. PhyMo: Learning Physical Dynamics with Accurate and Continuous Motion from Multi-View Videos

    Oct 21, 2026Shangjia Liu, Jinxi Li, Siyuan Zhou +2Dynamic Scene ReconstructionMotion Representation Learning

  20. EndoWorld-JEPA: Motion-Guided Global-Local Latent World Modeling for Endoscopic Video Representation Learning

    Oct 20, 2026Jie Wang, Shilong Yang, Tao Zhou +5Video Representation LearningLatent World Models

  21. A latent control model for realistic rodent motion

    Oct 19, 2026Aidan Sirbu, Charles Y. Zhang, Yuanjia Yang +3Latent Action LearningAnimal Behavior Analysis

  22. Representing short-term movement to reveal animal identity and learning progress

    Oct 19, 2026Ping Zhang, Quang-Huy Nguyen, Yingtie Lei +5Animal Behavior AnalysisTrajectory Representation Learning

  23. PRIMA: Pose Refined Inductive Motion Adapter for Zero Shot Social Bodily Behavior Recognition

    Oct 18, 2026Mingcong Ding, Babette Bühler, Tim Fütterer +2VLM AdaptationZero-Shot Action Recognition