Linear Multi-granular Body Modeling for Text-Driven Motion Generation

Xingzu Zhan; Genpei Zhang; Chen Xie; Honghang Chen; Haoran Sun; Xiaochun Mai

cs.CVMar 10, 2025

Linear Multi-granular Body Modeling for Text-Driven Motion Generation

Authors: Xingzu Zhan, Genpei Zhang, Chen Xie, Honghang Chen, Haoran Sun, Xiaochun Mai

Organizations: Shenzhen University, Shenzhen, China

Abstract

Text-to-motion generation is a rapidly growing field at the nexus of multimodal learning and computer graphics, enabling flexible and cost-effective applications in gaming, animation, robotics, and virtual reality. Existing part-aware pipelines typically split the body into segments and then recover whole-body coordination via attention-based fusion, which incurs quadratic complexity and fails to model part-level and whole-body cues simultaneously. Moreover, we observe that naively stacking spatial and temporal modules accumulates redundant features, thereby requiring deeper networks to achieve competitive fidelity. To address these issues, we propose HiSTF Mamba, a linear-time framework composed of Dual-Spatial Mamba, Bi-Temporal Mamba, and the Dynamic Spatiotemporal Fusion Module (DSFM). Dual-Spatial Mamba performs parallel part-based and whole-based modeling to simultaneously capture local joint dynamics and global coordination. Bi-Temporal Mamba employs bidirectional scanning to encode both short-term motion details and long-term dependencies. DSFM further removes redundancy and extracts complementary temporal information before fusing it with spatial features, yielding an expressive spatiotemporal representation. Extensive experiments on HumanML3D and KIT-ML demonstrate that HiSTF Mamba improves generation fidelity and semantic alignment while requiring substantially fewer parameters than existing segment-based architectures.

Figures & tables

Fig. 1: Qualitative visualization of motions generated by HiSTF Mamba.

Fig. 2: The overview of the proposed HiSTF Mamba. Given a text prompt and diffusion timestep, the model iteratively denoises from Gaussian noise to synthesize a motion sequence, as illustrated by the inference process at the bottom. The backbone consists of N Bi-Temporal Mamba that encodes bidirectional temporal features $\{T_{1},\ldots,T_{N}\}$ , a Dual-Spatial Mamba that captures multi-granular spatial cues via parallel part-based and whole-based scans, and a DSFM that suppresses redundant temporal information and fuses refined temporal cues with spatial features. The right panel visualizes scan order of whole SSMs and the Part SSMs.

Methods	R Precision $\uparrow$			FID $\downarrow$	MM Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$	Params (M) $\downarrow$
Methods	Top 1	Top 2	Top 3	FID $\downarrow$	MM Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$	Params (M) $\downarrow$
On the HumanML3D	0.511 ±.003	0.703 ±.003	0.797 ±.002	0.002 ±.000	2.974 ±.008	9.503 ±.065	–	–
T2M [ 5 ]	0.457 ±.002	0.639 ±.003	0.740 ±.003	1.067 ±.002	3.340 ±.008	9.188 ±.002	2.090 ±.083	31.16
MDM [ 14 ]	0.320 ±.005	0.498 ±.004	0.611 ±.007	0.544 ±.044	5.566 ±.027	9.559 ±.086	2.799 ±.072	17.88
MotionDiffuse [ 18 ]	0.491 ±.001	0.681 ±.001	0.782 ±.001	0.630 ±.001	3.113 ±.001	9.410 ±.049	1.553 ±.042	87.15
MLD [ 2 ]	0.481 ±.003	0.673 ±.003	0.772 ±.002	0.473 ±.013	3.196 ±.010	9.724 ±.082	2.413 ±.079	26.38

TABLE I: Comparison of results on the HumanML3D and KIT-ML test sets. The right arrow (

\rightarrow

) indicates that values closer to real motions correspond to better performance. Each evaluation was run 20 times, and “

\pm

” denotes the 95% confidence interval. Bold text highlights the best scores, while underlined entries mark the second best.

Fig. 3: Qualitative comparison of typical methods . We randomly selected three textual samples to generate motion sequences. Our results exhibit closer alignment with the text and present richer joint-level details.

N	L	R Precision (Top 3) $\uparrow$	FID $\downarrow$	MM Dist $\downarrow$
2	2	$0.813^{\pm.002}$	$0.069^{\pm.002}$	$2.907^{\pm.005}$
3	2	${0.813}^{\pm.002}$	$\textbf{0.039}^{\pm.002}$	$2.884^{\pm.008}$
4	2	$\textbf{0.817}^{\pm.002}$	$0.074^{\pm.002}$	$\textbf{2.872}^{\pm.007}$
2	3	$0.814^{\pm.003}$	$0.062^{\pm.003}$	$2.893^{\pm.007}$
3	3	$0.810^{\pm.002}$	$0.054^{\pm.003}$	$2.914^{\pm.008}$
4	3	$0.816^{\pm.002}$	$0.052^{\pm.002}$	$2.873^{\pm.008}$

TABLE II: Ablation study on varying the number of Bi-Temporal Mamba blocks (

N

) and the number of layers (

L

), evaluated on HumanML3D.

	R-Top3 $\uparrow$	FID $\downarrow$	MM Dist $\downarrow$
w/o DSFM	$\textbf{0.817}^{\pm.002}$	$0.096^{\pm.003}$	$2.889^{\pm.010}$
w/o Part-b	$0.816^{\pm.002}$	$0.066^{\pm.003}$	$\textbf{2.869}^{\pm.008}$
w/o Whole-b	$0.812^{\pm.002}$	$0.065^{\pm.003}$	$2.901^{\pm.009}$
w/o Part-b $\&$ Whole-b	$0.815^{\pm.002}$	$0.070^{\pm.003}$	$2.874^{\pm.008}$
HiSTF Mamba (Ours)	${0.813}^{\pm.002}$	$\textbf{0.039}^{\pm.002}$	$2.884^{\pm.008}$

TABLE III: Ablation study on key components, evaluated on HumanML3D. We use Part-b and Whole-b to denote the part-based learning and whole-based learning in Dual-Spatial Mamba , respectively.

Explore similar work

FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling
Apr 13, 2026Dawei Guan, Di Yang, Chengjie Jin +1Text-to-Motion Generation Human Motion Generation
School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China
T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation
Feb 1, 2026Xingzu Zhan, Chen Xie, Honghang Chen +2Text-to-Motion Generation Mamba
Shenzhen University, Shenzhen, China · Jinan University, Guangzhou, China
MotionHiFlow: Text-to-motion via hierarchical flow matching
Apr 25, 2026Heng Li, Xiaotong Lin, Ling-An Zeng +3Flow Matching Text-to-Motion Generation
Sun Yat-sen University · Shandong University · Guangdong Province Key Laboratory of Information Security Technology, China +1

Apr 13, 2026J/K move · Enter open · S save

FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling

Dawei Guan, Di Yang, Chengjie Jin, Jiangtao Wang

School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China

Text-to-Motion Generation Human Motion Generation Motion Representation Learning Flow-Based Generative Modeling

Open paper AskPDF arXiv

Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.

Abstract

Explore similar work

FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling

T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation

MotionHiFlow: Text-to-motion via hierarchical flow matching