QuadFM: Foundational Text-Driven Quadruped Motion Dataset for Generation and Control
Organizations: AMAP, Alibaba Group
Abstract
Despite significant advances in quadrupedal robotics, a critical gap persists in foundational motion resources that holistically integrate diverse locomotion, emotionally expressive behaviors, and rich language semantics-essential for agile, intuitive human-robot interaction. Current quadruped motion datasets are limited to a few mocap primitives (e.g., walk, trot, sit) and lack diverse behaviors with rich language grounding. To bridge this gap, we introduce Quadruped Foundational Motion (QuadFM) , the first large-scale, ultra-high-fidelity dataset designed for text-to-motion generation and general motion control. QuadFM contains 11,784 curated motion clips spanning locomotion, interactive, and emotion-expressive behaviors (e.g., dancing, stretching, peeing), each with three-layer annotation-fine-grained action labels, interaction scenarios, and natural language commands-totaling 35,352 descriptions to support language-conditioned understanding and command execution. We further propose Gen2Control RL, a unified framework that jointly trains a general motion controller and a text-to-motion generator, enabling efficient end-to-end inference on edge hardware. On a real quadruped robot with an NVIDIA Orin, our system achieves real-time motion synthesis (<500 ms latency). Simulation and real-world results show realistic, diverse motions while maintaining robust physical interaction. The dataset will be released at https://github.com/GaoLii/QuadFM.
Figures & tables
| Motion Category | Data Source | Semantic Subset / Behavior Types | QuadFM (Ours) | DogML | ||
| Clips | Duration | Clips | Duration | |||
| Locomotion | Motion Capture | Basic Gaits (Walking, Trotting, etc.) | 7998 | 10.02 h | 4,024 ∗ | 11.03 h |
| Video Generation | Diverse In-the-wild Behaviors | 1,392 | 1.62 h | 0 | 0.00 h | |
| Interaction & Emotion | Teleoperation | Natural (Greeting), Sad (Cautious) | 696 | 4.74 h | 0 | 0.00 h |
| Artist Design | Happy (Dancing, Excited), etc. | 1,698 | 3.89 h | 0 | 0.00 h | |
| Total | – | All Sources / Comprehensive | 11,784 | 20.27 h | 4,024 ∗ | 11.03 h |
| Stability | Text-Motion Alignment | Smoothness | Naturalness | |
| MotionGPT3 [ 52 ] | 6.27 | 5.77 | 5.62 | 5.54 |
| Gen2Control RL(ours) | 7.58 | 7.98 | 7.46 | 7.40 |
| Related GT | 8.01 | 8.21 | 7.82 | 7.69 |
| Training Data | Ours Val | DogML [ 38 ] Val | ||
| MJPE ( ) | MBPE ( ) | MJPE ( ) | MBPE ( ) | |
| DogML [ 38 ] (100%) | 0.0964 | 0.2832 | 0.1331 | 0.1009 |
| Ours (only teleoperation) | 0.0884 | 0.0806 | 0.2571 | 0.1304 |
| Ours (only artist design) | 0.0871 | 0.0770 | 0.1640 | 0.1072 |
| Ours (only motion capture) | 0.0968 | 0.2338 | 0.2331 | 0.1201 |
| Ours (only video generation) | 0.1507 | 0.1306 | 0.2368 | 0.1149 |
Robot Motion Generation
Robot motion generation creates feasible, task-directed trajectories for robots to move, manipulate objects, and interact with people and environments.
See all 198 papers