FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling

Dawei Guan; Di Yang; Chengjie Jin; Jiangtao Wang

cs.CVApr 13, 2026

FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modeling

Authors: Dawei Guan, Di Yang, Chengjie Jin, Jiangtao Wang

Organizations: School of Artificial Intelligence and Data Science, University of Science and Technology of China · Suzhou Institute for Advanced Research, University of Science and Technology of China

Abstract

Text-to-motion generation is driven by learning motion representations for semantic alignment with language. Existing methods rely on either continuous or discrete motion representations. However, continuous representations entangle semantics with dynamics, while discrete representations lose fine-grained motion details. In this context, we propose FlowCoMotion, a novel motion generation framework that unifies both treatments from a modeling perspective. Specifically, FlowCoMotion employs token-latent coupling to capture both semantic content and high-fidelity motion details. In the latent branch, we apply multi-view distillation to regularize the continuous latent space, while in the token branch we use discrete temporal resolution quantization to extract high-level semantic cues. The motion latent is then obtained by combining the representations from the two branches through a token-latent coupling network. Subsequently, a velocity field is predicted based on the textual conditions. An ODE solver integrates this velocity field from a simple prior, thereby guiding the sample to the potential state of the target motion. Extensive experiments show that FlowCoMotion achieves competitive performance on text-to-motion benchmarks, including HumanML3D and SnapMoGen.

Figures & tables

Figure 1: We propose a novel framework called FlowCoMotion , which can handle fine-grained semantic content to guide the generation of high-quality motion. This method is particularly suitable for handling various forms of input, including but not limited to numbers, shapes and directions.

Figure 2: Method Overview. FlowCoMotion mainly includes three key components: (1) Motion compression ( Section 3.1 ): The motion sequence is compressed into the latent space using VAE to obtain $z\in\mathbb{R}^{n\times b}$ . (2) Motion representation & Coupling ( Sections 3.2 and 3.3 ): The Token Branch further uses RVQ-VAE to extract discrete Token representations and then dequantizes them to obtain $z_{q}$ . The Latent Branch directly uses the continuous latent space vector $z_{c}$ . Finally, the final motion representation $z_{gt}$ is obtained through the token-latent coupling network. (3) Motion Flow Matching in Condition Generation ( Section 3.4 ): The flow velocity field $\hat{v}_{t}$ is predicted through the text condition $c$ and time $t$ . The predicted latent vector $\hat{z}_{1}$ is solved by the ODE solver and finally handed over to the Decoder $\mathcal{D}$ .

Figure 3: Two Conditional Transformer Blocks for Motion Generation. (a) The Nova Block integrates a unified conditioning signal via LN-modulated cross-attention and uses a learnable gate to control the strength of the conditioned update on the hidden state $x$ . (b) The Orbit Block separates conditioning into time and text inputs, performing cross-attention with $QK$ normalization for improved stability. It also applies the same learnable gate to modulate the conditioned residual.

Figure 4: We propose a three-stage training strategy for the motion-language model: Stage 1: Train the Encoder $\mathcal{E}$ , Decoder $\mathcal{D}$ , and learnable latent space, refining the latent space representation. Stage 2: Freeze the Encoder and learnable latent space, and fine-tune the Decoder by kinematic losses. Stage 3: Train the flow matching head $\mathcal{H}$ to predict the velocity flow from random noise to the motion latent space. Inference process: Conditions are injected at each block, and the predicted latent vector is obtained by solving the ODE through $N$ steps after sampling from random noise. Finally, it is fed into the Decoder to obtain the predicted motion sequence.

Figure 5: Qualitative evaluation on text-to-motion generation. We qualitatively compared the visualizations generated by our method with those produced by MotionGPT Jiang et al. (2023) , MoMask++ Guo et al. (2025) , and MotionGPT3 Zhu et al. (2025) . Misaligned motions of other methods are marked with red text. The results show that FlowCoMotion demonstrates powerful text understanding capabilities, enabling it to generate more accurate and coherent motions.

Methods	R-precision $\uparrow$			FID $\downarrow$	MultiModal Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Methods	Top 1	Top 2	Top 3	FID $\downarrow$	MultiModal Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Real motions	$0.511^{\pm.003}$	$0.703^{\pm.003}$	$0.797^{\pm.002}$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	$9.503^{\pm.065}$	-
TM2D	$0.319^{\pm.000}$	-	-	$1.021^{\pm.000}$	$4.098^{\pm.000}$	$\mathbf{9.513}^{\pm.000}$	$\mathbf{4.139}^{\pm.000}$
MotionCraft	$0.501^{\pm.003}$	$0.697^{\pm.003}$	$0.796^{\pm.002}$	$0.173^{\pm.002}$	$3.025^{\pm.008}$	$9.543^{\pm.098}$	-
ReMoDiffuse	$0.510^{\pm.005}$	$0.698^{\pm.006}$	$0.795^{\pm.004}$	$0.103^{\pm.004}$	$2.974^{\pm.016}$	$9.018^{\pm.075}$	$1.795^{\pm.043}$
MMM	$0.504^{\pm.003}$	$0.696^{\pm.003}$	$0.794^{\pm.002}$	$0.080^{\pm.003}$	$2.998^{\pm.007}$	$9.411^{\pm.058}$	$1.164^{\pm.041}$

Table 1: Quantitative evaluation on HumanML3D test set.

Methods	R-precision $\uparrow$			FID $\downarrow$	CLIP Score $\uparrow$	MModality $\uparrow$
Methods	Top 1	Top 2	Top 3	FID $\downarrow$	CLIP Score $\uparrow$	MModality $\uparrow$
Real motions	$0.940^{\pm.001}$	$0.976^{\pm.001}$	$0.985^{\pm.001}$	$0.001^{\pm.000}$	$0.837^{\pm.000}$	-
MDM	$0.503^{\pm.002}$	$0.653^{\pm.002}$	$0.727^{\pm.002}$	$57.783^{\pm.092}$	$0.481^{\pm.001}$	$\mathbf{13.412}^{\pm.231}$
T2M-GPT	$0.618^{\pm.002}$	$0.773^{\pm.002}$	$0.812^{\pm.002}$	$32.629^{\pm.087}$	$0.573^{\pm.001}$	$9.172^{\pm.181}$
StableMoFusion	$0.679^{\pm.002}$	$0.823^{\pm.002}$	$0.888^{\pm.002}$	$27.801^{\pm.063}$	$0.605^{\pm.001}$	$9.064^{\pm.138}$
MARDM	$0.659^{\pm.002}$	$0.812^{\pm.002}$	$0.860^{\pm.002}$	$26.878^{\pm.131}$	$0.602^{\pm.001}$	$9.812^{\pm.287}$

Table 2: Quantitative evaluation on SnapMoGen test set.

	VQ Config.				VQ Reconstruction		T2M Generation
	#Codes	#Quant.	F/M	w/ Att.	FID $\downarrow$	Joint Pos. Err. $\downarrow$	FID $\downarrow$	R-Precison@3 $\uparrow$
HumanML3D	1024	4	M	✓	0.0622	0.0488	$0.122^{\pm.002}$	$0.809^{\pm.003}$
	512	4	M	✓	0.0859	0.0514	$0.218^{\pm.003}$	$0.771^{\pm.002}$
	Representation		Dimension		VQ Reconstruction		T2M Generation
	Type		Dim.		FID $\downarrow$	Joint Pos. Err. $\downarrow$	FID $\downarrow$	R-Precison@3 $\uparrow$
	Latent		256		0.0054	0.0302	$0.087^{\pm.004}$	$0.796^{\pm.003}$

Table 3: Ablation analysis of motion representation on Text-to-Motion Tasks.

Representation	Dimension	VQ Reconstruction		T2M Generation
Type	Dim.	FID $\downarrow$	Joint Pos. Err. $\downarrow$	FID $\downarrow$	R-Precison@3 $\uparrow$	Inference time $\downarrow$
Latent + Flow Matching	256	0.0054	0.0302	$0.087^{\pm.004}$	$0.796^{\pm.003}$	0.49
Hybrid + Flow Matching	64, 192	0.0036	0.0234	$0.061^{\pm.003}$	$0.832^{\pm.002}$	0.52
Latent + Diffusion	256	0.0054	0.0302	$0.078^{\pm.003}$	$0.781^{\pm.002}$	7.66
Hybrid + Diffusion	64, 192	0.0036	0.0234	$0.075^{\pm.003}$	$0.801^{\pm.002}$	8.02

Table 4: Ablation analysis on HumanML3D test set of motion representation and generation heads.

Methods	R-precision $\uparrow$			FID $\downarrow$
Methods	Top 1	Top 2	Top 3	FID $\downarrow$
MoMask	$0.467^{\pm.002}$	$0.655^{\pm.003}$	$0.748^{\pm.003}$	$\mathbf{0.231}^{\pm.012}$
MoMask++	$0.473^{\pm.003}$	$0.669^{\pm.003}$	$0.769^{\pm.002}$	$0.354^{\pm.015}$
MotionGPT3	$0.496^{\pm.002}$	$0.685^{\pm.002}$	$0.787^{\pm.003}$	$0.658^{\pm.032}$
Ours ${}^{\text{nova}}$	$0.479^{\pm.002}$	$0.678^{\pm.003}$	$0.776^{\pm.002}$	$0.349^{\pm.014}$
Ours ${}^{\text{orbit}}$	$\mathbf{0.502}^{\pm.003}$	$\mathbf{0.693}^{\pm.004}$	$\mathbf{0.796}^{\pm.002}$	$0.332^{\pm.011}$

Table 5: Quantitative evaluation on harder test set which selected from HumanML3D.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Qualitative evaluation on text-to-motion generation. The state-of-the-art methods presented in this paper are all implemented using the same inference and rendering settings on HumanML3D. Misaligned motion is indicated by red text and bounding boxes. We selected some results with shorter prompts. The results demonstrate that our motion-language framework exhibits strong text understanding capabilities, enabling the generation of more accurate and coherent motion.

Figure 7: Qualitative evaluation on text-to-motion generation. The provided state-of-the-art methods are under the same training and inference setting on HumanML3D. Misaligned motions are indicated by red texts. We selected some results with longer prompts. The results indicate that our motion-language framework exhibits strong text comprehension capabilities, leading to more accurate and coherent motion generation.

Methods	R-precision $\uparrow$			FID $\downarrow$	MultiModal Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Methods	Top 1	Top 2	Top 3	FID $\downarrow$	MultiModal Dist $\downarrow$	Diversity $\rightarrow$	MModality $\uparrow$
Real motions	$0.511^{\pm.003}$	$0.703^{\pm.003}$	$0.797^{\pm.002}$	$0.002^{\pm.000}$	$2.974^{\pm.008}$	$9.503^{\pm.065}$	-
TEMOS	$0.424^{\pm.002}$	$0.612^{\pm.002}$	$0.722^{\pm.002}$	$3.734^{\pm.028}$	$3.703^{\pm.008}$	$8.973^{\pm.071}$	$0.368^{\pm.018}$
TM2T	$0.424^{\pm.003}$	$0.618^{\pm.003}$	$0.729^{\pm.002}$	$1.501^{\pm.017}$	$3.467^{\pm.011}$	$8.589^{\pm.076}$	$2.424^{\pm.093}$
T2M	$0.457^{\pm.002}$	$0.639^{\pm.003}$	$0.740^{\pm.003}$	$1.067^{\pm.002}$	$3.340^{\pm.008}$	$9.188^{\pm.002}$	$2.090^{\pm.083}$
TM2D	$0.319^{\pm.000}$	-	-	$1.021^{\pm.000}$	$4.098^{\pm.000}$	$\mathbf{9.513}^{\pm.000}$	$\mathbf{4.139}^{\pm.000}$

Appendix

Table 6: Quantitative evaluation on HumanML3D test set.

Figure 8: Evaluation sweep over ODE numbers L (left) and guidance scale s (right) in inference. We find a accuracy-fidelity sweep spot around s = 2, meanwhile 40 iterations (L = 40) for flow matching yield sufficiently good results.

Figure 9: Comparison of inference time costs. All tests were conducted on a single NVIDIA RTX 2080 Ti. Models closer to the origin indicate better performance. For non-autoregressive models, our method significantly improves inference speed, thanks to the flow matching directional velocity field.

Figure 10: Ablation analysis on representation dimension choices. We report reconstruction quality using FID and MPJPE. The larger of the circles represents a better MPJPE.

Figure 11: User study form. The User Interface (UI) was used in our user study. Our user study consists of 50 individual evaluations and 50 comparative evaluations. We incorporate elements from other methods to prevent the review results from becoming invalid due to user inertia.

Figure 12: Human preference on Text-to-Motion Generation by comparing FlowCoMotion with MotionGPT [ 21 ] , Momask++ [ 11 ] and MotionGPT3 [ 44 ] . We report the scores on three generation aspects.

Figure 13: Details of our Flow Matching Transformer model. Given an Input Latent of shape $B\times 256\times T^{\prime}$ , alongside Time Cond and Text Emb, the initial features are processed through Patchify and Embed layers. These tokens are then passed through $N$ Transformer Blocks, which incorporate Orbit Transformer Block and Nova Transformer Block designs utilizing Multi-Head Self-Attention and Multi-Head Cross-Attention. After passing through the $N$ layers, latent generation proceeds via a Linear & Reshape layer, ultimately yielding the Output Latent and Output Velocity in their original $B\times 256\times T^{\prime}$ dimensions.

Dimension	Matching Keywords / Patterns
$D_{1}$ Direction	left, right, forward, backward, clockwise, counterclockwise, sideways, diagonal, north,,south, east, westward, …
$D_{2}$ Body Parts	arm, leg, hand, foot, head, torso, hip, shoulder, elbow, knee, wrist, ankle, spine, neck, finger, chest, back, …
$D_{3}$ Modifiers	quickly, slowly, slightly, sharply, gently, rapidly, suddenly, gradually, briefly, continuously, repeatedly, …
$D_{4}$ Temporal	then, while, before, after, simultaneously, followed by, at the same time, next, finally, meanwhile, …
$D_{5}$ Count	twice, three times, several times, multiple times, once, again, number of, …
$D_{6}$ Verbs	kick, punch, bend, twist, spin, crawl, kneel, wave, crouch, squat, hop, leap, lunge, stomp, clap, point, rotate, …

Appendix

Table 7: Keyword Lexicon for Multi-Dimensional Difficulty Rules. We define eight linguistic dimensions (

D_{1}

–

D_{8}

) to quantify the complexity of motion prompts. Keywords are matched using heuristic rules to compute the initial difficulty score.

Figure 14: Prompt template for Difficulty Oracle. The system prompt (top) defines linguistic evaluation criteria and enforces JSON output, while the user prompt (bottom) supplies the target caption to enable structured semantic scoring.

Explore similar work

Apr 25, 2026

MotionHiFlow: Text-to-motion via hierarchical flow matching

Text-to-motion generation aims to generate 3D human motions that are tightly aligned with the input text while remaining physically plausible and rich in fine-grained detail. Although recent approaches can produce complex and natural movements, they usually operate at only one temporal scale, which limits both semantic alignment and temporal coherence. Inspired by the fact that complex motions are conceptualized hierarchically rather than at a single temporal scale in the human cognitive system, we propose MotionHiFlow, a hierarchical flow matching framework to generate motion progressively by constructing flow path from low to high temporal scales. The flows at lower scales capture high-level semantics and coarse motion structures, while flows at higher scales refine temporal details. To link the flows across scales, we introduce a novel cross-scale transition process, ensuring continuity and preserving noise consistency. Furthermore, by integrating a Text-Motion Diffusion Transformer and a topology-aware Motion VAE, MotionHiFlow explicitly models structural dependencies among joints via joint-aware positional encoding and skeletal topology, enabling precise semantic alignment alongside fine-grained motion details. Extensive experiments on HumanML3D and KIT-ML benchmarks demonstrate state-of-the-art performance, with ablation studies confirming the effectiveness of the hierarchical design and key components. Code is available at https://github.com/ai-lh/MotionHiFlow.

Heng Li, Xiaotong Lin, Ling-An Zeng +3

Sun Yat-sen University · Shandong University · Guangdong Province Key Laboratory of Information Security Technology, China +1

Flow Matching Text-to-Motion Generation Human Motion Generation Flow-Based Generative Modeling

Ask PDF

May 28, 2026

DC-Motion: Decoupling Structure and Details via Discrete-Continuous Tokens for Human Motion Generation

Text-to-motion generation requires modeling both global action structure and fine-grained motion dynamics from natural language. Existing approaches typically rely on either continuous diffusion models or vector-quantized discrete representations. Diffusion models generate smooth motions but lack explicit compositional structure for temporal planning, while discrete token-based methods improve controllability but compress motion into finite codebooks, losing fine-grained dynamics. We argue that this limitation stems from a representation mismatch: action semantics such as intent, phase transitions, and temporal layout are inherently discrete and compositional, whereas joint trajectories and motion dynamics are continuous and locally correlated. To address this, we propose DC-Motion, a discrete-continuous factorized framework for human motion generation. DC-Motion decomposes motion into discrete structural tokens capturing global action layout and continuous residual latents modeling fine-grained dynamics. A text-conditioned structure generator predicts discrete tokens via iterative masked modeling, and a diffusion-based residual generator produces continuous motion conditioned on the structure. Experiments on HumanML3D and KIT-ML demonstrate that DC-Motion achieves strong performance in both FID and R-Precision, outperforming representative diffusion-based and discrete-token baselines.

Hequan Wang, Xuean Chen, Jiaxu Zhang +2

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China · Geophysical Exploration Brigade of Hubei Provincial Geological Bureau, Wuhan 430058, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

Text-to-Motion Generation Human Motion Generation Generative Modeling Diffusion Models for Motion Generation Conditional Diffusion Models

Ask PDF

Apr 3, 2026

Exploring Motion-Language Alignment for Text-driven Motion Generation

Text-driven human motion generation aims to synthesize realistic motion sequences that follow textual descriptions. Despite recent advances, accurately aligning motion dynamics with textual semantics remains a fundamental challenge. In this paper, we revisit text-to-motion generation from the perspective of motion-language alignment and propose MLA-Gen, a framework that integrates global motion priors with fine-grained local conditioning. This design enables the model to capture common motion patterns, while establishing detailed alignment between texts and motions. Furthermore, we identify a previously overlooked attention sink phenomenon in human motion generation, where attention disproportionately concentrates on the start text token, limiting the utilization of informative textual cues and leading to degraded semantic grounding. To analyze this issue, we introduce SinkRatio, a metric for measuring attention concentration, and develop alignment-aware masking and control strategies to regulate attention during generation. Extensive experiments demonstrate that our approach consistently improves both motion quality and motion-language alignment over strong baselines.

Ruxi Gu, Zilei Wang, Wei Wang

Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China

Text-to-Motion Generation Human Motion Generation Cross-Modal Alignment Attention Sinks

Ask PDF