Organizations: School of Computer Science and Engineering, Southeast University, Nanjing 211189, China, and also with Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China · State Key Laboratory for Novel Software Technology, School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China · New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA), and also with School of Artificial Intelligence, University of Chinese Academy of Sciences
Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pre-training remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multi-dimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.1% R-Precision gain and 21.7% FID improvement on rotation-based HY-Motion. Our project page with code is publicly available.
Figures & tables
Fig. 1: Comparison of the training costs and generation performance on HumanML3D [ 15 ] . (a) Performance comparison of different fine-tuning methods [ 23 , 39 , 40 ] . (b) Generalization performance across six pre-trained diffusion-based models [ 8 , 41 , 42 , 7 , 14 ] .
Fig. 2: Overview of MotionReward, consisting of unified projection, representation, and multiple preference learning.
Fig. 3: The framework of existing differentiable reward-based methods (left) and our proposed EasyTune (right). Existing methods backpropagate the gradients of the reward model through the overall denoising process, resulting in (1) excessive memory, (2) inefficient, and (3) coarse-grained optimization. In contrast, EasyTune optimizes the diffusion model by directly backpropagating the gradients at each denoising step, overcoming these issues.
Figure 4
Fig. 7: Core insight of EasyTune. By replacing the recursive gradient in Eq.( 17 ) with step-level ones in Eq.( 23 ), EasyTune removes recursive dependencies, enabling (1) step-wise graph storage, (2) efficiency, and (3) fine-grained optimization.
Representation
Accuracy ↑
Precision ↑
Recall ↑
F1-Score ↑
Kinematic
91.78
91.43
99.67
95.37
Joint
93.92
93.58
99.69
96.54
TABLE III: Evaluation on motion authenticity for real versus generated motions under kinematic and joint representations.
Fig. 8: Memory trajectory during optimization. The x-axis is the denoising step and the y-axis is GPU memory.
Figure 8
Fig. 11: Visual results on HumanML3D dataset. “w/o EasyTune” refers to motions generated by the original MLD model [ 8 ] , while “w/ EasyTune” indicates motions generated by the MLD model fine-tuned using our proposed EasyTune [ 43 ] .
Figure 10
Fig. 14: Text-motion retrieval accuracy of ReAlign and SPL.
Fig. 15: Correlation between rotation and joint-mapped rewards on HumanML3D. Evaluated on rotation features xr : (a) scalar reward scores srot vs. sjoint ; (b) optimization gradients grot vs. gjoint .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
Noise
Text-Motion Retrieval ↑
Motion-Text Retrieval ↑
Avg. ↑
R@1
R@2
R@3
R@5
R@10
R@1
R@2
R@3
R@5
R@10
TEMOS [ 59 ]
✗
43.88
58.25
67.00
74.00
84.75
41.88
55.88
65.62
75.25
85.75
65.23
T2M [ 15 ]
✗
42.25
62.62
75.12
87.50
96.12
39.75
62.75
73.62
86.88
95.88
72.25
TMR [ 28 ]
✗
49.25
69.75
78.25
87.88
95.00
50.12
67.12
76.88
88.88
94.75
75.79
LaMP [ 66 ]
✗
52.50
74.80
84.70
92.70
97.60
54.00
75.30
84.40
92.20
97.60
80.58
ReAlign [ 29 ]
✓
52.88
71.62
83.00
91.25
97.62
53.00
72.88
84.38
92.62
96.88
79.61
Appendix
TABLE VIII: Text-motion retrieval on KIT-ML. Noise indicates whether the method handles noisy motion from the denoising process.
Method
Locomotion
Sports & Athletics
Fitness & Outdoor Activities
Daily Activities
Social Interactions & Leisure
Game Character Actions
Overall
HY-Motion-lite
55.89
54.90
50.84
47.06
50.82
56.53
52.59%
w/ MotionRFT (Ours)
56.37
55.52
51.02
47.90
51.36
57.43
53.15%
HY-Motion
56.95
56.35
51.46
49.39
52.34
58.07
53.98%
w/ MotionRFT (Ours)
57.43
56.94
51.61
49.77
52.77
58.62
54.42%
Appendix
TABLE X: SSAE [ 37 ] results on the released HY-Motion model weights with and without MotionRFT.
Method
Top 1 ↑
Top 2 ↑
Top 3 ↑
FID ↓
MM Dist ↓
Foot Floating ↓
Jitter Degree ↓
Dynamic Degree ↑
GT
0.511
0.703
0.797
0.002
2.974
0.0781
0.0219
0.0452
Base MLD
0.504
0.698
0.796
0.450
3.052
0.1941
0.0101
0.0341
EasyTune
0.568
0.754
0.846
0.194
2.672
0.1685
0.0127
0.0317
MotionRFT
0.577
0.767
0.854
0.137
2.622
0.1224
0.00853
0.0357
Appendix
TABLE XII: Semantic and physical motion-quality metrics on HumanML3D under MLD. Best results among Base MLD, EasyTune, and MotionRFT are in bold.
Fig. 16: Sample-level correlation on HumanML3D under pretrained MLD. (a) Matching score and reward. (b) R-Precision and reward. (c) Matching score and mid-late reward. Solid and dashed curves in (a) and (b) are Pearson r and Spearman ρ versus the denoising step.
Setting
Rep.
Text-Motion Retrieval ↑
Motion-Text Retrieval ↑
Avg. ↑
R@1
R@2
R@3
R@5
R@10
R@1
R@2
R@3
R@5
R@10
Kinematic representation of 263 dimensions
MotionReward (multi-projection)
K
67.74
83.35
89.79
94.86
98.37
67.92
83.50
90.03
95.16
98.44
86.92
MotionReward (joint-only)
K
67.74
83.35
89.79
94.86
98.37
67.92
83.50
90.03
95.16
98.44
86.92
Rotation representation of 135 dimensions
MotionReward (multi-projection)
R
64.19
82.13
89.27
93.75
97.87
63.42
80.85
88.42
94.12
97.72
85.17
Appendix
TABLE XV: Text-motion retrieval on HumanML3D under multi-projection versus joint-canonical reward models. Multi-projection denotes MotionReward with dedicated projection heads per representation. Joint-only maps all inputs to 3D joint coordinates and scores them with the joint head alone. Note: For the 263 -d kinematic representation, the joint-only setting uses only the essential 67 dimensions following Meng et al. [ 47 ] .
Settings (1)–(3) : Joint-space reward R66 on mapped inputs vs. ground-truth joints x66
Setting (1)
Setting (2)
Setting (3)
135→66 (FK)
263→66 (rot)
67→66 (ric)
r(R66(x135→66),R66(x66))=0.693
r(R66(x263→66),R66(x66))=0.951
r(R66(x67→66),R66(x66))=1.000
r(∇x66R66(x135→66),∇x66R66(x66))=0.667
r(∇x66R66(x263→66),∇x66R66(x66))=0.849
r(∇x66R66(x67→66),∇x66R66(x66))=1.000
Settings (4)–(6) : Joint-mapped reward R66(xA→66) vs. native reward RD(xD)
Setting (4)
Setting (5)
Setting (6)
Appendix
TABLE XVI: Representation mapping configurations and correlation metrics. Features 135 , 66 , 263 , and 67 denote rotation, 3D joint, complete kinematic, and essential kinematic formats, respectively. xA→66 maps representation A to 3D joints via forward kinematics FK, rotation decoding rot , or relative coordinate recovery ric . R66 and RD are joint and native reward evaluators with D∈{135,263} , and tabulated values are Pearson correlation coefficients r for reward scores and gradients.
Fig. 17: Pearson correlation and gradient alignment across representation mappings on HumanML3D. Top row: scalar reward scores for Settings (1)–(3) (left three panels) and Settings (4)–(6) (right three panels). Bottom row: optimization gradient alignment for the same settings. Panel labels match Tab. XVI .
Method
Reward
Generative Model Output
Reward Input
R-Precision ↑
FID ↓
MM Dist ↓
Diversity →
Top1
Top2
Top3
Kinematic representation
Base MLD++
–
–
–
0.548
0.738
0.829
0.073
2.810
9.658
w/ MotionRFT
Kinematic(Native)
263
263
0.603
0.786
0.872
0.052
2.528
9.683
w/ MotionRFT
Joint
263
66
0.580
0.768
0.855
0.062
2.630
9.668
w/ MotionRFT
Joint
retain 67 of 263
66
0.596
0.777
0.868
0.061
2.675
9.668
Appendix
TABLE XVII: Text-to-motion generation on HumanML3D under multi-projection versus joint-canonical rewards. Multi-projection denotes MotionRFT with native representation-specific reward heads. Joint-only fine-tunes generators using a single joint-space reward head. Generative Model Output is the generator feature dimension; Reward Input is the feature consumed by the reward.
Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text. Motion is inherently temporal, especially in compositional and long-duration sequences that require semantic consistency across multiple action segments and smooth kinematic transitions throughout the trajectory. We posit that the initial noise is central to this consistency: within the Gaussian noise space, certain instances, i.e. winning noise tickets, carry latent structure that biases denoising toward particular motion semantics, even under null prompts. We propose WInning Noise Retrieval and Optimization (WINRO), a training-free, model-agnostic framework that improves text-motion alignment by selecting and refining such tickets before diffusion sampling. WINRO maps random noises to motion features generated under null prompts, retrieves the best-aligned noise for a given text, and refines it via a KL-regularized objective that reduces the residual semantic gap while preserving the Gaussian prior. An optional LoRA-based adapter amortizes this refinement into a single forward pass. WINRO consistently improves text-motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining, improves temporal robustness on the MTT benchmark, and generalizes to applications such as motion stylization and spatial constraint satisfaction.
Sakuya Ota, Qing Yu, Kent Fujiwara +2
Institute of Science Tokyo, Tokyo, Japan · LY Corporation, Tokyo, Japan · National Institute of Informatics (NII), Tokyo, Japan
We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diffusion, either on the whole latent at once or auto-regressively over multiple latents. In this paper, we study how to make diffusion on continuous motion latents work best. We focus on two questions: (1) how to build a semantically aligned latent space so diffusion becomes more effective, and (2) how to best inject text conditioning so the motion follows the description closely. We propose a semantic-aligned motion encoder trained with frame-level text labels so that latents with similar text meaning stay close, which makes the latent space more diffusion-friendly. We also compare single-token conditioning with a multi-token cross-attention scheme and find that cross-attention gives better motion realism and text-motion alignment. With semantically aligned latents, auto-regressive generation, and cross-attention text conditioning, our model sets a new state of the art in human motion generation on standard metrics and in a user study. We will release our code and models for further research and downstream usage.
Yannan He, Garvita Tiwari, Xiaohan Zhang +4
University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany +2
Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks due to their impressive generative capabilities and flexibility. However, current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process. This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges. First, RAM leverages a motion latent space as intermediate supervision for text-to-motion generation. To this end, RAM co-trains a motion reconstruction branch with two key objective functions: self-regularization to enhance the discrimination of the motion space and motion-centric latent alignment to enable accurate mapping from text to the motion latent space. Second, we propose Reconstructive Error Guidance (REG), a testing-stage guidance mechanism that exploits the motion diffusion model's inherent self-correction ability to mitigate error propagation. At each denoising step, REG uses the motion reconstruction branch to reconstruct the previous estimate, reproducing the prior error patterns. By amplifying the residual between the current prediction and the reconstructed estimate, REG highlights the improvements in the current prediction. Extensive experiments demonstrate that RAM achieves significant improvements and state-of-the-art performance. Our code will be released.
Yifei Liu, Changxing Ding, Ling Guo +2
South China University of Technology · Joy Future Academy