Organizations: School of Computer Science and Engineering, Southeast University, Nanjing 211189, China, and also with Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China · State Key Laboratory for Novel Software Technology, School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China · New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA), and also with School of Artificial Intelligence, University of Chinese Academy of Sciences
Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pre-training remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multi-dimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.1% R-Precision gain and 21.7% FID improvement on rotation-based HY-Motion. Our project page with code is publicly available.
Figures & tables
Fig. 1: Comparison of the training costs and generation performance on HumanML3D [ 15 ] . (a) Performance comparison of different fine-tuning methods [ 23 , 39 , 40 ] . (b) Generalization performance across six pre-trained diffusion-based models [ 8 , 41 , 42 , 7 , 14 ] .
Fig. 2: Overview of MotionReward, consisting of unified projection, representation, and multiple preference learning.
Fig. 3: The framework of existing differentiable reward-based methods (left) and our proposed EasyTune (right). Existing methods backpropagate the gradients of the reward model through the overall denoising process, resulting in (1) excessive memory, (2) inefficient, and (3) coarse-grained optimization. In contrast, EasyTune optimizes the diffusion model by directly backpropagating the gradients at each denoising step, overcoming these issues.
Figure 4
Fig. 7: Core insight of EasyTune. By replacing the recursive gradient in Eq.( 17 ) with step-level ones in Eq.( 23 ), EasyTune removes recursive dependencies, enabling (1) step-wise graph storage, (2) efficiency, and (3) fine-grained optimization.
Representation
Accuracy ↑
Precision ↑
Recall ↑
F1-Score ↑
Kinematic
91.78
91.43
99.67
95.37
Joint
93.92
93.58
99.69
96.54
TABLE III: Evaluation on motion authenticity for real versus generated motions under kinematic and joint representations.
Fig. 8: Memory trajectory during optimization. The x-axis is the denoising step and the y-axis is GPU memory.
Figure 8
Fig. 11: Visual results on HumanML3D dataset. “w/o EasyTune” refers to motions generated by the original MLD model [ 8 ] , while “w/ EasyTune” indicates motions generated by the MLD model fine-tuned using our proposed EasyTune [ 43 ] .
Figure 10
Fig. 14: Text-motion retrieval accuracy of ReAlign and SPL.
Fig. 15: Correlation between rotation and joint-mapped rewards on HumanML3D. Evaluated on rotation features xr : (a) scalar reward scores srot vs. sjoint ; (b) optimization gradients grot vs. gjoint .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
Noise
Text-Motion Retrieval ↑
Motion-Text Retrieval ↑
Avg. ↑
R@1
R@2
R@3
R@5
R@10
R@1
R@2
R@3
R@5
R@10
TEMOS [ 59 ]
✗
43.88
58.25
67.00
74.00
84.75
41.88
55.88
65.62
75.25
85.75
65.23
T2M [ 15 ]
✗
42.25
62.62
75.12
87.50
96.12
39.75
62.75
73.62
86.88
95.88
72.25
TMR [ 28 ]
✗
49.25
69.75
78.25
87.88
95.00
50.12
67.12
76.88
88.88
94.75
75.79
LaMP [ 66 ]
✗
52.50
74.80
84.70
92.70
97.60
54.00
75.30
84.40
92.20
97.60
80.58
ReAlign [ 29 ]
✓
52.88
71.62
83.00
91.25
97.62
53.00
72.88
84.38
92.62
96.88
79.61
Appendix
TABLE VIII: Text-motion retrieval on KIT-ML. Noise indicates whether the method handles noisy motion from the denoising process.
Method
Locomotion
Sports & Athletics
Fitness & Outdoor Activities
Daily Activities
Social Interactions & Leisure
Game Character Actions
Overall
HY-Motion-lite
55.89
54.90
50.84
47.06
50.82
56.53
52.59%
w/ MotionRFT (Ours)
56.37
55.52
51.02
47.90
51.36
57.43
53.15%
HY-Motion
56.95
56.35
51.46
49.39
52.34
58.07
53.98%
w/ MotionRFT (Ours)
57.43
56.94
51.61
49.77
52.77
58.62
54.42%
Appendix
TABLE X: SSAE [ 37 ] results on the released HY-Motion model weights with and without MotionRFT.
Method
Top 1 ↑
Top 2 ↑
Top 3 ↑
FID ↓
MM Dist ↓
Foot Floating ↓
Jitter Degree ↓
Dynamic Degree ↑
GT
0.511
0.703
0.797
0.002
2.974
0.0781
0.0219
0.0452
Base MLD
0.504
0.698
0.796
0.450
3.052
0.1941
0.0101
0.0341
EasyTune
0.568
0.754
0.846
0.194
2.672
0.1685
0.0127
0.0317
MotionRFT
0.577
0.767
0.854
0.137
2.622
0.1224
0.00853
0.0357
Appendix
TABLE XII: Semantic and physical motion-quality metrics on HumanML3D under MLD. Best results among Base MLD, EasyTune, and MotionRFT are in bold.
Fig. 16: Sample-level correlation on HumanML3D under pretrained MLD. (a) Matching score and reward. (b) R-Precision and reward. (c) Matching score and mid-late reward. Solid and dashed curves in (a) and (b) are Pearson r and Spearman ρ versus the denoising step.
Setting
Rep.
Text-Motion Retrieval ↑
Motion-Text Retrieval ↑
Avg. ↑
R@1
R@2
R@3
R@5
R@10
R@1
R@2
R@3
R@5
R@10
Kinematic representation of 263 dimensions
MotionReward (multi-projection)
K
67.74
83.35
89.79
94.86
98.37
67.92
83.50
90.03
95.16
98.44
86.92
MotionReward (joint-only)
K
67.74
83.35
89.79
94.86
98.37
67.92
83.50
90.03
95.16
98.44
86.92
Rotation representation of 135 dimensions
MotionReward (multi-projection)
R
64.19
82.13
89.27
93.75
97.87
63.42
80.85
88.42
94.12
97.72
85.17
Appendix
TABLE XV: Text-motion retrieval on HumanML3D under multi-projection versus joint-canonical reward models. Multi-projection denotes MotionReward with dedicated projection heads per representation. Joint-only maps all inputs to 3D joint coordinates and scores them with the joint head alone. Note: For the 263 -d kinematic representation, the joint-only setting uses only the essential 67 dimensions following Meng et al. [ 47 ] .
Settings (1)–(3) : Joint-space reward R66 on mapped inputs vs. ground-truth joints x66
Setting (1)
Setting (2)
Setting (3)
135→66 (FK)
263→66 (rot)
67→66 (ric)
r(R66(x135→66),R66(x66))=0.693
r(R66(x263→66),R66(x66))=0.951
r(R66(x67→66),R66(x66))=1.000
r(∇x66R66(x135→66),∇x66R66(x66))=0.667
r(∇x66R66(x263→66),∇x66R66(x66))=0.849
r(∇x66R66(x67→66),∇x66R66(x66))=1.000
Settings (4)–(6) : Joint-mapped reward R66(xA→66) vs. native reward RD(xD)
Setting (4)
Setting (5)
Setting (6)
Appendix
TABLE XVI: Representation mapping configurations and correlation metrics. Features 135 , 66 , 263 , and 67 denote rotation, 3D joint, complete kinematic, and essential kinematic formats, respectively. xA→66 maps representation A to 3D joints via forward kinematics FK, rotation decoding rot , or relative coordinate recovery ric . R66 and RD are joint and native reward evaluators with D∈{135,263} , and tabulated values are Pearson correlation coefficients r for reward scores and gradients.
Fig. 17: Pearson correlation and gradient alignment across representation mappings on HumanML3D. Top row: scalar reward scores for Settings (1)–(3) (left three panels) and Settings (4)–(6) (right three panels). Bottom row: optimization gradient alignment for the same settings. Panel labels match Tab. XVI .
Method
Reward
Generative Model Output
Reward Input
R-Precision ↑
FID ↓
MM Dist ↓
Diversity →
Top1
Top2
Top3
Kinematic representation
Base MLD++
–
–
–
0.548
0.738
0.829
0.073
2.810
9.658
w/ MotionRFT
Kinematic(Native)
263
263
0.603
0.786
0.872
0.052
2.528
9.683
w/ MotionRFT
Joint
263
66
0.580
0.768
0.855
0.062
2.630
9.668
w/ MotionRFT
Joint
retain 67 of 263
66
0.596
0.777
0.868
0.061
2.675
9.668
Appendix
TABLE XVII: Text-to-motion generation on HumanML3D under multi-projection versus joint-canonical rewards. Multi-projection denotes MotionRFT with native representation-specific reward heads. Joint-only fine-tunes generators using a single joint-space reward head. Generative Model Output is the generator feature dimension; Reward Input is the feature consumed by the reward.