Human athletes demonstrate versatile and highly-dynamic tennis skills to successfully conduct competitive rallies with a high-speed tennis ball. However, reproducing such behaviors on humanoid robots is difficult, partially due to the lack of perfect humanoid action data or human kinematic motion data in tennis scenarios as reference. In this work, we propose LATENT, a system that Learns Athletic humanoid TEnnis skills from imperfect human motioN daTa. The imperfect human motion data consist only of motion fragments that capture the primitive skills used when playing tennis rather than precise and complete human-tennis motion sequences from real-world tennis matches, thereby significantly reducing the difficulty of data collection. Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios. With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles. We also propose a series of designs for robust sim-to-real transfer and deploy our policy on the Unitree G1 humanoid robot. Our method achieves surprising results in the real world and can stably sustain multi-shot rallies with human players. Project page: https://zzk273.github.io/LATENT/
Figures & tables
Figure 1: Overview of LATENT . (a) We pre-train a motion tracker on collected imperfect human motion data. (b) We construct a correctable latent action space via online distillation. (c) We train a high-level policy to correct and compose latent actions for tennis task. (d) We transfer the policy to the real world via dynamics randomization and observation noise.
Figure 2: Visualization of generated ball trajectories.
Term
Weight
Term
Weight
Task
Approach to ball
10.0
Ball landing
30.0
Hit success
200.0
Regularization
High-level action
−5⋅10−3
Torque penalty
−2⋅10−5
Lower body action rate
−1.0
Whole body action rate
−0.5
Table 1: Reward terms used in LATENT. The reward terms are divided into three types: Task, Regularization, and Termination.
Figure 3: The motivation of Latent Action Barrier (LAB).
Term
Value
Robot
Feet friction
U(1.0,2.5)
Joint friction
×U(0.75,1.25)
Armature
×U(1.0,1.05)
Body mass
×U(0.75,1.25)
Body CoM
U(−0.05,0.05)
Table 2: Dynamics randomization used in LATENT. We randomize robot dynamics and tennis ball physics.
Method
Forehand
Backhand
Forecourt
Backcourt
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
PPO [ 26 ]
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
MotionVAE [ 44 ]
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
AMP [ 22 ]
41.32
4.71
46.52
13.79
36.81
5.22
37.68
15.74
49.48
4.20
36.16
18.26
41.73
4.85
41.77
19.20
ASE [ 21 ]
63.47
3.62
38.91
11.84
55.22
4.01
31.75
12.92
71.36
3.28
30.84
14.05
68.59
3.90
35.12
15.27
PULSE [ 19 ]
71.85
3.01
32.44
9.76
63.38
3.15
29.60
10.83
74.52
2.97
28.11
12.01
72.94
3.02
31.27
13.44
Table 3: Comparison with baseline methods. Bold numbers indicate the best performance.
Method
Forehand
Backhand
Forecourt
Backcourt
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
SR↑
DE↓
Smth↓
Torque↓
w/o Corr.
82.36
3.41
26.72
8.85
68.94
3.03
29.55
8.92
74.21
3.18
26.86
9.64
79.05
3.52
29.94
9.38
w/o LAB
93.12
2.17
37.64
12.53
76.96
2.21
31.77
12.11
84.73
1.96
30.59
10.94
87.44
2.05
32.08
13.62
Ours
96.52
1.32
25.61
7.40
82.10
1.89
27.76
7.71
86.35
1.73
25.51
7.47
89.80
1.55
27.20
7.60
Table 4: Ablation study. Bold numbers indicate the best performance.
Method
Forehand
Backhand
Forecourt
Backcourt
SR↑
DE↓
SR↑
DE↓
SR↑
DE↓
SR↑
DE↓
w/o DR
16.67%
5.60
25.00%
4.57
28.57%
3.59
14.29%
6.23
w/o ON
50.00%
3.28
0.00%
N/A
28.57%
3.69
37.50%
3.97
Ours
90.90%
3.15
77.78%
3.89
88.89%
3.57
81.82%
3.80
Table 5: Real-world performance. Bold numbers indicate the best, and underlined numbers indicate the second.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Robot movement coverage during consecutive ball returns. Heatmaps of the robot’s global positions accumulated over different numbers of consecutive ball returns (8, 16, 80, and 400). The learned policy enables effective court coverage and adaptive repositioning during consecutive rallies.
Figure 5: Robot-robot self-play evaluation in simulation. (a) Visualization of two robots performing self-play on opposite sides of the court. (b) Distribution of number of consecutive rallies over 50 random games.
Figure 6: Different hitting events in simulation. The red curves denote incoming ball trajectories, the robot posture corresponds to the moment of ball contact, and the green curves indicate the trajectories of the returned balls.
Figure 7: Close-up real-world examples. Representative frames from real-world rallies showing diverse tennis return behaviors, including forehand and backhand strokes, dynamic footwork, and coordinated whole-body motion.
Humanoid tennis requires reactive whole-body coordination under rapidly changing ball dynamics, yet collecting task-specific motion-capture demonstrations that cover diverse approach states, contact locations, and timing conditions is costly. Inspired by how humans acquire new skills by combining reusable motor experience with task-specific demonstrations, we present ATHLETE, a framework for learning reactive humanoid tennis from generic locomotion data and ordinary tennis videos. ATHLETE combines LAFAN1 locomotion with video-derived striking motions and uses Trajectory-Guided Motion Matching (TGMM) to construct diverse task-conditioned whole-body references. These references are transformed into a reactive policy through physics-based teacher--student learning with online DAgger distillation, followed by PPO fine-tuning for the tennis task. The resulting policy responds directly to robot and ball observations without requiring reference-motion inputs at deployment. On a real humanoid robot, ATHLETE achieves forehand and backhand hit rates of 90.00% and 86.67%, respectively, and enables multi-shot rallies with human players. These results demonstrate that ATHLETE can learn dynamic humanoid tennis skills from reusable locomotion data and video demonstrations. Result videos and code can be found at https://anonymous.4open.science/w/ATHLETE2027/.
Zhihao Tian, Zifan Wang, Enzheng Zhu +4
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we propose AdaPT, an Adaptive Motion Planning and Tracking framework that learns professional tennis serving and rally styles directly from broadcast videos. This hierarchical design is motivated by the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal interference with planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades on real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perception. To address these issues, our adaptation mechanism improves tracking robustness by learning to track randomized execution speeds, while conditioning the planner on a learned motion-speed adapter to mitigate compounding errors. Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism in bridging the sim-to-real gap. We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot (1.7m) and demonstrate in-the-wild serving without motion capture. Beyond these results, our real-world experiments reveal both algorithmic and engineering insights for future humanoid ball-sports systems. Videos and code are available on our project website.
Tao Huang, Ruofei Liu, Xuchen Tang +15
Noitom Robotics · Shanghai AI Laboratory · Shanghai Jiao Tong University +1
High-speed racket sports provide a demanding testbed for humanoid robots, requiring time-critical decisions, precise striking, and dynamic whole-body coordination. In badminton, fast-changing shuttle trajectories require timely contact decisions, while successful returns demand precise racket pose and velocity within a brief contact window and across a broad three-dimensional striking workspace. Human motion data provide valuable priors for such athletic skills, but usable badminton references are limited and imperfect. Direct tracking provides insufficient executable variation for diverse shuttle conditions, while purely task-driven optimization may produce unnatural motion. To address these challenges, we present a three-stage hierarchical reinforcement learning framework for dynamic humanoid badminton. First, task-randomized motion augmentation expands sparse annotated hitting events into executable target-conditioned stroke variations, forming a continuous latent skill space. Second, a high-level planner outputs continuous latent skill codes to compose these skills online according to the observed shuttle state. Third, a context-conditioned adversarial regularizer encourages more natural planner-level skill usage while preserving return performance. When deployed on a real humanoid robot, our system achieves sustained multi-skill rallies with human players, including forehand, backhand, and highly dynamic jump returns. This is the first real-world humanoid racket-sport system to demonstrate multi-skill human--robot rallies including highly dynamic jump returns.
Jingzhi Cui, Zhexiong Wang, Bangjie Xu +9
Tsinghua University · Hong Kong Embodied AI Lab · Zhejiang University +3