LingoMotion: An Interpretable and Unambiguous Symbolic Representation for Human Motion

Yao Zhang; Zhuchenyang Liu; Yu Xiao

cs.CVMar 13, 2026

LingoMotion: An Interpretable and Unambiguous Symbolic Representation for Human Motion

Authors: Yao Zhang, Zhuchenyang Liu, Yu Xiao

Organizations: Aalto University, Espoo, Finland

Abstract

Existing representations for human motion, such as MotionGPT, often operate as black-box latent vectors with limited interpretability and build on joint positions which can cause ambiguity. Inspired by the hierarchical structure of natural languages - from letters to words, phrases, and sentences - we propose LingoMotion, a motion language that facilitates interpretable and unambiguous symbolic representation for both simple and complex human motion. In this paper, we introduce the concept design of LingoMotion, including the definitions of motion alphabet based on joint angles, the morphology for forming words and phrases to describe simple actions like walking and their attributes like speed and scale, as well as the syntax for describing more complex human activities with sequences of words and phrases. The preliminary results, including the implementation and evaluation of motion alphabet using a large-scale motion dataset Motion-X, demonstrate the high fidelity of motion representation.

Figures & tables

Fig. 1: The core analogy of our paper: Just as natural language decomposes continuous sound waves into discrete letters, words, phrases, and sentences, we propose a ”motion language” that decomposes continuous joint-angle signals into discrete motion letters, words, phrases, and sentences to describe complex human activities. This provides a structured, interpretable, and compositional framework for human motion representation.

Fig. 2: (a) Illustration of the 22 joints in the SMPL model, with joint angles calculated based on biological standards [ 10 , 13 ] . (b) Joint angles of the right hip, which has 3 DoFs (flexion, adduction, rotation), reveal periodic hip movement during walking. (c) The 3D positions of the right knee during walking, which lose the periodic information.

Fig. 3: Our 3-stage pipeline for discovering the motion alphabet from right hip flexion measurements. (a) Segment the signal at local extrema. (b) Learn a codebook of canonical shapes via K-Means clustering on normalized segments. (c) Represent the original signal as a sequence of discrete ”motion letters” (shape s ) and their continuous attributes ( $\sigma,\mu,l$ ). (d) Visualization of the 3D motion, rendered from the joint position, illustrating the corresponding physical poses at the segment boundaries.

Experiment	RMSE	NMSE	R²
Joint positions $\rightarrow$ angles $\rightarrow$ positions	0.009	0.005	0.978
Joint angles $\rightarrow$ motion letters $\rightarrow$ joint angles	0.921	0.103	0.716

TABLE I: Accuracy of representation learning. RMSE (Root Mean Square Error) and NMSE (Normalized Mean Square Error) are reported.

Explore similar work

LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference
Mar 12, 2026Junkun Jiang, Ho Yin Au, Jingyu Xiang +1Text-to-Motion Generation Human Motion Generation
Department of Computer Science, Hong Kong Baptist University, HKSAR
Encoder-Free Human Motion Understanding via Structured Motion Descriptions
Apr 23, 2026Yao Zhang, Zhuchenyang Liu, Thomas Ploetz +1
Aalto University, Espoo, Finland · Georgia Institute of Technology, Atlanta, GA, USA
Fine-grained Human Motion Understanding with Language Models
Jun 18, 2026Thomas Markhorst, Zhi-Yi Lin, Jouh Yeong Chew +2
Delft University of Technology · Honda Research Institute Japan

Mar 12, 2026J/K move · Enter open · S save

LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference

Junkun Jiang, Ho Yin Au, Jingyu Xiang, Jie Chen

Department of Computer Science, Hong Kong Baptist University, HKSAR

Text-to-Motion Generation Human Motion Generation Motion Representation Learning

Open paper AskPDF arXiv

Human motion is highly expressive and naturally aligned with language, yet prevailing methods relying heavily on joint text-motion embeddings struggle to synthesize temporally accurate, detailed motions and often lack explainability. To address these limitations, we introduce LabanLite, a motion representation developed by adapting and extending the Labanotation system. Unlike black-box text-motion embeddings, LabanLite encodes each atomic body-part action (e.g., a single left-foot step) as a discrete Laban symbol paired with a textual template. This abstraction decomposes complex motions into interpretable symbol sequences and body-part instructions, establishing a symbolic link between high-level language and low-level motion trajectories. Building on LabanLite, we present LaMoGen, a Text-to-LabanLite-to-Motion Generation framework that enables large language models (LLMs) to compose motion sequences through symbolic reasoning. The LLM interprets motion patterns, relates them to textual descriptions, and recombines symbols into executable plans, producing motions that are both interpretable and linguistically grounded. To support rigorous evaluation, we introduce a Labanotation-based benchmark with structured description-motion pairs and three metrics that jointly measure text-motion alignment across symbolic, temporal, and harmony dimensions. Experiments demonstrate that LaMoGen establishes a new baseline for both interpretability and controllability, outperforming prior methods on our benchmark and two public datasets. These results highlight the advantages of symbolic reasoning and agent-based design for language-driven motion synthesis.

Abstract

Explore similar work

LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference

Encoder-Free Human Motion Understanding via Structured Motion Descriptions

Fine-grained Human Motion Understanding with Language Models