Coordinate-Based Dual-Constrained Autoregressive Motion Generation

Kang Ding; Hongsong Wang; Jie Gui; Liang Wang

cs.CVApr 9, 2026

Coordinate-Based Dual-Constrained Autoregressive Motion Generation

Authors: Kang Ding, Hongsong Wang, Jie Gui, Liang Wang

Organizations: School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China · School of Computer Science and Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, Southeast University, Nanjing 210096, China · School of Cyber Science and Engineering, Southeast University, also with Purple Mountain Laboratories, Nanjing 211111, China, and also with Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, Southeast University, Nanjing 210096, China · New Laboratory of Pattern Recognition (NLPR), also with State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA), and also with School of Artificial Intelligence, University of Chinese Academy of Sciences

Abstract

Text-to-motion generation has attracted increasing attention in the research community recently, with potential applications in animation, virtual reality, robotics, and human-computer interaction. Diffusion and autoregressive models are two popular and parallel research directions for text-to-motion generation. However, diffusion models often suffer from error amplification during noise prediction, while autoregressive models exhibit mode collapse due to motion discretization. To address these limitations, we propose a flexible, high-fidelity, and semantically faithful text-to-motion framework, named Coordinate-based Dual-constrained Autoregressive Motion Generation (CDAMD). With motion coordinates as input, CDAMD follows the autoregressive paradigm and leverages diffusion-inspired multi-layer perceptrons to enhance the fidelity of predicted motions. Furthermore, a Dual-Constrained Causal Mask is introduced to guide autoregressive generation, where motion tokens act as priors and are concatenated with textual encodings. Since there is limited work on coordinate-based motion synthesis, we establish new benchmarks for both text-to-motion generation and motion editing. Experimental results demonstrate that our approach achieves state-of-the-art performance in terms of both fidelity and semantic consistency on these benchmarks.

Figures & tables

Fig. 1: Comparison of our approach with existing methods: (a) The existing autoregressive model (BAMM [ 11 ] ) fails to generate text-aligned motion for this example text prompt; (b) Top-1 R-Precision and FID results on the HumanML3D (bubble areas represent the 95% confidence interval of the FID). Our approach achieves excellent performance in both FID and R-Precision.

Fig. 2: Architecture illustration of CDAMD. (a) Hybrid Motion Encoders encodes the raw motion sequence into a compact fine-grained latent space. (b) CDAMD model learns to autoregressively predict next tokens conditioned on text embedding from CLIP and compressed motion tokens from RVQ-VAE. (c) Dual-Constrained Causal Attention (DCCA) enforces both temporal and conditional causality, ensuring that motion generation proceeds autoregressively and preserving semantic conditioning.

Methods	FID $\bm{\downarrow}$	R-Precision $\bm{\uparrow}$			Matching $\bm{\downarrow}$	MModality $\bm{\uparrow}$	Diversity $\bm{\uparrow}$
Methods	FID $\bm{\downarrow}$	Top 1	Top 2	Top 3	Matching $\bm{\downarrow}$	MModality $\bm{\uparrow}$	Diversity $\bm{\uparrow}$
MDM-50Step ∗ [ 20 ]	$0.395^{\pm.065}$	$0.435^{\pm.013}$	$0.627^{\pm.0014}$	$0.737^{\pm.011}$	$3.443^{\pm.060}$	$2.182^{\pm.055}$	$9.812^{\pm.148}$
MotionDiffuse ∗ [ 48 ]	$1.334^{\pm.035}$	$0.435^{\pm.008}$	$0.618^{\pm.006}$	$0.725^{\pm.005}$	$3.542^{\pm.015}$	$1.833^{\pm.078}$	$9.634^{\pm.048}$
ReMoDiffuse ∗ [ 23 ]	$0.207^{\pm.006}$	$0.467^{\pm.001}$	$0.654^{\pm.004}$	$0.748^{\pm.002}$	$3.289^{\pm.022}$	$\textbf{2.560}^{\pm.289}$	$9.483^{\pm.245}$
MotionLCM V2 ∗ [ 35 ]	$0.152^{\pm.007}$	$0.511^{\pm.007}$	$\underline{0.707}^{\pm.003}$	$\textbf{0.802}^{\pm.002}$	$3.005^{\pm.009}$	$1.993^{\pm.085}$	$9.703^{\pm.097}$
MMM ∗ [ 49 ]	$0.093^{\pm.004}$	$0.495^{\pm.008}$	$0.687^{\pm.000}$	$0.782^{\pm.003}$	$3.165^{\pm.037}$	$1.303^{\pm.085}$	$9.786^{\pm.352}$

TABLE I: Evaluation of text-to-motion generation on the HumanML3D datasets. More than 10 baselines are re-implemented using joint coordinates to reproduce their results on this dataset. The evaluation is repeated 20 times, and the average is reported with a 95% confidence interval. Bold indicates the best result, and underscore denotes the second best. ∗ refers the result of our re-implementation. VQ denotes the motion quantizers in the Residual Vector Quantized Variational Autoencoder (RVQ-VAE).

Methods	FID $\bm{\downarrow}$	R-Precision $\bm{\uparrow}$			Matching $\bm{\downarrow}$	MModality $\bm{\uparrow}$	Diversity $\bm{\uparrow}$
Methods	FID $\bm{\downarrow}$	Top 1	Top 2	Top 3	Matching $\bm{\downarrow}$	MModality $\bm{\uparrow}$	Diversity $\bm{\uparrow}$
MDM [ 20 ]	$0.782^{\pm.066}$	$0.390^{\pm.007}$	$0.609^{\pm.010}$	$0.733^{\pm.012}$	$3.956^{\pm.110}$	$\textbf{2.402}^{\pm.257}$	$11.019^{\pm.139}$
MotionDiffuse [ 48 ]	$2.793^{\pm.192}$	$0.357^{\pm.010}$	$0.553^{\pm.009}$	$0.667^{\pm.012}$	$3.957^{\pm.062}$	$1.538^{\pm.062}$	$11.316^{\pm.153}$
ReMoDiffuse [ 23 ]	$\textbf{0.257}^{\pm.021}$	$\underline{0.412}^{\pm.006}$	$\underline{0.630}^{\pm.012}$	$\underline{0.754}^{\pm.009}$	$\textbf{3.097}^{\pm.043}$	$1.814^{\pm.103}$	$\underline{11.451}^{\pm.183}$
MoMask [ 17 ]	$0.523^{\pm.022}$	$0.392^{\pm.006}$	$0.604^{\pm.008}$	$0.732^{\pm.006}$	$3.383^{\pm.030}$	$\underline{1.892}^{\pm.085}$	$11.143^{\pm.072}$
CDAMD (ours)	$\underline{0.270}^{\pm.020}$	$\textbf{0.416}^{\pm.006}$	$\textbf{0.632}^{\pm.005}$	$\textbf{0.759}^{\pm.004}$	$\underline{3.160}^{\pm.027}$	$1.211^{\pm.075}$	$\textbf{11.902}^{\pm.154}$

TABLE II: Evaluation of coordinate-based text-to-motion generation on the KIT dataset. Baselines are re-implemented using joint coordinates to reproduce their results on this dataset. Bold indicates the best result, and underscore denotes the second best.

Tasks	Methods	R-Precision $\bm{\uparrow}$			FID $\bm{\uparrow}$	Matching $\bm{\downarrow}$	CLIP-score $\bm{\uparrow}$
Tasks	Methods	Top 1	Top 2	Top 3	FID $\bm{\uparrow}$	Matching $\bm{\downarrow}$	CLIP-score $\bm{\uparrow}$
Temporal Inpainting	BAMM [ 11 ]	0.387	0.554	0.649	0.385	4.046	0.574
	MARDM [ 10 ]	0.503	0.702	0.795	0.120	3.051	0.671
	CDAMD (ours)	0.500	0.691	0.790	0.103	3.049	0.665
Temporal Outpainting	BAMM [ 11 ]	0.433	0.605	0.707	0.206	3.615	0.613
	MARDM [ 10 ]	0.512	0.705	0.797	0.114	3.065	0.671

TABLE III: Results of temporal motion editing on the HumanML3D. We use joint coordinates as motion representation, and re-implement results of BAMM [ 11 ] and MARDM [ 10 ] .

Methods	FID $\bm{\downarrow}$	R-Precision $\bm{\uparrow}$			Matching $\bm{\downarrow}$	Diversity $\bm{\uparrow}$	CLIP-score $\bm{\uparrow}$
Methods	FID $\bm{\downarrow}$	Top 1	Top 2	Top 3	Matching $\bm{\downarrow}$	Diversity $\bm{\uparrow}$	CLIP-score $\bm{\uparrow}$
DCCM	0.082	0.522	0.708	0.800	2.966	9.680	0.679
BCM	0.154	0.520	0.710	0.805	2.925	9.770	0.683
CM	0.231	0.515	0.696	0.792	3.055	9.132	0.671

TABLE IV: Comparison between different masking strategies in the transformer backbone. We compare the proposed Dual-Constrained Causal Mask (DCCM) with the Bidirectional Causal Mask (BCM) [ 11 ] and the Causal Mask (CM) on the HumanML3D dataset.

Methods	FID $\bm{\downarrow}$	R-Precision $\bm{\uparrow}$			Matching $\bm{\downarrow}$	Diversity $\bm{\uparrow}$	CLIP-score $\bm{\uparrow}$
Methods	FID $\bm{\downarrow}$	Top 1	Top 2	Top 3	Matching $\bm{\downarrow}$	Diversity $\bm{\uparrow}$	CLIP-score $\bm{\uparrow}$
CDAMD (ours)	0.082	0.522	0.708	0.800	2.966	9.680	0.679
w/o CMP	0.136	0.520	0.704	0.796	3.015	9.495	0.676
w/o DCCA	0.231	0.515	0.696	0.792	3.055	9.132	0.671
CDAMD (ours) †	0.090	0.523	0.712	0.808	2.942	9.728	0.684
w/o CMP †	0.122	0.516	0.699	0.791	3.109	9.686	0.681

TABLE V: Ablation study of the design choices of CDAMD on the HumanML3D dataset. CMP and DCCA denote Conditioned Motion Prior and Dual-Constrained Causal Attention, respectively. † denotes architectural components under the standard 263-dimensional mixed representation.

Fig. 3: Visualization of temporal editing tasks, inpainting, outpainting, prefix, and suffix where orange indicates conditioned motion and blue refers to generated parts.

Fig. 4: Qualitative results of text-to-motion generation on HumanML3D. Our approach is compared with BAMM [ 11 ] and MoMask [ 17 ] , which are representative autoregressive motion models.

Fig. 5: The failure cases of text-to-motion on HumanML3D test set.

Fig. 6: Visualization comparison if textual to motion to state-of-the-art methods. Both the BAMM [ 11 ] and MoMask [ 17 ] generate motion of insufficient quality, while CDAMD generates higher quality and is more correlated with textual descriptions.

Fig. 7: Validation loss curves with and without conditional motion priors. Loss curves show faster convergence and lower error when motion priors are incorporated.

CDAMD (ours)	MoMask	BAMM	Ground-Truth
39.8	21.5	10.4	28.3

TABLE VI: Comparison of motion preferences among CDAMD, MoMask [ 17 ] , BAMM [ 11 ] , and ground truth in the user study.

Explore similar work

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
Jan 21, 2026Yifei Liu, Changxing Ding, Ling Guo +2Diffusion Model Guidance Text-to-Motion Generation
South China University of Technology · Joy Future Academy
Marrying Text-to-Motion Generation with Skeleton-Based Action Recognition
Apr 18, 2026Jidong Kuang, Hongsong Wang, Jie GuiSkeleton-Based Action Recognition Human Activity Recognition
School of Cyber Science and Engineering, Southeast University, Nanjing, China · School of Computer Science and Engineering, Southeast University, Nanjing, China
Causal Motion Diffusion Models for Autoregressive Motion Generation
Feb 26, 2026Qing Yu, Akihisa Watanabe, Kent FujiwaraHuman Motion Generation Diffusion Models for Motion Generation
LY Corporation · Waseda University

Jan 21, 2026J/K move · Enter open · S save

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation

Yifei Liu, Changxing Ding, Ling Guo, Huaiguang Jiang+1

South China University of Technology · Joy Future Academy

Diffusion Model Guidance Text-to-Motion Generation Human Motion Generation Diffusion Models for Motion Generation

Open paper AskPDF arXiv

Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks due to their impressive generative capabilities and flexibility. However, current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process. This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges. First, RAM leverages a motion latent space as intermediate supervision for text-to-motion generation. To this end, RAM co-trains a motion reconstruction branch with two key objective functions: self-regularization to enhance the discrimination of the motion space and motion-centric latent alignment to enable accurate mapping from text to the motion latent space. Second, we propose Reconstructive Error Guidance (REG), a testing-stage guidance mechanism that exploits the motion diffusion model's inherent self-correction ability to mitigate error propagation. At each denoising step, REG uses the motion reconstruction branch to reconstruct the previous estimate, reproducing the prior error patterns. By amplifying the residual between the current prediction and the reconstructed estimate, REG highlights the improvements in the current prediction. Extensive experiments demonstrate that RAM achieves significant improvements and state-of-the-art performance. Our code will be released.

Abstract

Explore similar work

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation

Marrying Text-to-Motion Generation with Skeleton-Based Action Recognition

Causal Motion Diffusion Models for Autoregressive Motion Generation