Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking

Tao Huang; Ruofei Liu; Xuchen Tang; Xinyin Zhang; Junli Ren; Huayi Wang; Feiyu Jia; Yukai Qi

cs.ROAug 20, 2026

Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking

Authors: Tao Huang, Ruofei Liu, Xuchen Tang, Xinyin Zhang, Junli Ren, Huayi Wang, Feiyu Jia, Yukai Qi, +10 more

Organizations: Noitom Robotics · Shanghai AI Laboratory · Shanghai Jiao Tong University · Dobot Robotics

Abstract

Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we propose AdaPT, an Adaptive Motion Planning and Tracking framework that learns professional tennis serving and rally styles directly from broadcast videos. This hierarchical design is motivated by the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal interference with planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades on real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perception. To address these issues, our adaptation mechanism improves tracking robustness by learning to track randomized execution speeds, while conditioning the planner on a learned motion-speed adapter to mitigate compounding errors. Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism in bridging the sim-to-real gap. We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot (1.7m) and demonstrate in-the-wild serving without motion capture. Beyond these results, our real-world experiments reveal both algorithmic and engineering insights for future humanoid ball-sports systems. Videos and code are available on our project website.

Figures & tables

Figure 1: t-SNE visualization of motions and tracker rollouts.

Player	Total Time	# of Clips	Stroke Ratio		Spin Ratio
Player	Total Time	# of Clips	Forehand	Backhand	Flat & Top	Slice
Rafael Nadal	267min	5664	54%	46%	89%	11%
Roger Federer	179min	5264	48%	52%	84%	16%
Novak Djokovic	189min	5288	49%	51%	93%	7%
Mr. Black (MoCap)	219min	2987	57%	43%	76%	24%

Table 1: Statistics of motion data from broadcast videos of three players.

Figure 2: AdaPT learns stylized humanoid tennis skills through the pipeline: (a) broadcast videos are reconstructed, retargeted, and physically corrected to obtain tennis motions; (b) speed-adaptive policies learn to track serve and rally motions; (c) high-level planners select or generate style-consistent motions for serve and rally; (d) our robot localization or ball detection modules enable sim-to-real deployment, especially for serving without MoCap.

Stroke	Rafael Nadal					Roger Federer					Novak Djokovic
Stroke	Succ. $\uparrow$	$E_{\mathrm{bo}}\downarrow$	$E_{\mathrm{FID}}\downarrow$	$E_{\mathrm{pow}}\downarrow$	$E_{\mathrm{acc}}\downarrow$	Succ. $\uparrow$	$E_{\mathrm{bo}}\downarrow$	$E_{\mathrm{FID}}\downarrow$	$E_{\mathrm{pow}}\downarrow$	$E_{\mathrm{acc}}\downarrow$	Succ. $\uparrow$	$E_{\mathrm{bo}}\downarrow$	$E_{\mathrm{FID}}\downarrow$	$E_{\mathrm{pow}}\downarrow$	$E_{\mathrm{acc}}\downarrow$
RL-Scratch [ 34 ]	97.4	1.6 $\pm 0.6$	297.3 $\pm 16.3$	92.7 $\pm 20.0$	123.1 $\pm 8.9$	98.8	2.5 $\pm 1.8$	299.0 $\pm 29.8$	130.8 $\pm 27.9$	145.0 $\pm 8.3$	98.8	2.5 $\pm 1.8$	307.7 $\pm 18.4$	130.8 $\pm 27.9$	145.0 $\pm 8.3$
AMP [ 27 ]	98.1	2.7 $\pm 0.9$	172.1 $\pm 8.0$	118.1 $\pm 39.3$	126.9 $\pm 12.5$	65.1	2.1 $\pm 1.2$	116.0 $\pm 32.3$	112.7 $\pm 62.4$	122.5 $\pm 16.7$	21.6	2.2 $\pm 1.4$	66.9 $\pm 39.7$	185.6 $\pm 92.2$	155.2 $\pm 57.7$
PULSE [ 21 ]	86.2	2.5 $\pm 1.4$	29.7 $\pm 3.5$	92.2 $\pm 31.1$	107.1 $\pm 11.4$	92.2	2.5 $\pm 1.1$	17.9 $\pm 1.5$	94.1 $\pm 26.7$	101.0 $\pm 9.5$	86.1	3.4 $\pm 1.7$	32.9 $\pm 6.3$	78.6 $\pm 29.6$	92.8 $\pm 7.5$
NCP [ 50 ]	85.6	4.8 $\pm 2.3$	9.2 $\pm 3.5$	33.0 $\pm 13.2$	55.3 $\pm 5.6$	89.1	3.6 $\pm 1.5$	6.8 $\pm 4.7$	24.2 $\pm 15.3$	48.6 $\pm 5.2$	69.0	4.9 $\pm 2.3$	24.0 $\pm 2.8$	22.8 $\pm 6.2$	56.6 $\pm 4.6$
Vid2Player3D [ 47 ]	82.6	4.8 $\pm 2.0$	5.1 $\pm 5.2$	15.1 $\pm 5.6$	41.1 $\pm 5.6$	81.5	2.9 $\pm 1.2$	13.4 $\pm 3.2$	18.5 $\pm 8.3$	41.9 $\pm 7.9$	83.2	3.3 $\pm 1.9$	6.3 $\pm 3.2$	24.0 $\pm 9.7$	40.6 $\pm 7.3$

Table 2: Simulation results. Evaluation of all candidate methods across different metrics for rally and serve tasks by learning the motion of three master-level players.

\uparrow

denotes higher is better, while

\downarrow

denotes the opposite. Overall, AdaPT consistently achieves strong motion fidelity while maintaining high task performance, demonstrating an effective balance between stylistic imitation and task performance across both rallying and serving.

Figure 4: Experimental setup for rally and serve task with motion capture.

Player	Method	Adaptive Tracker	Adaptive Planner	Hit Rate (%) $\uparrow$			Net Clearance Rate (%) $\uparrow$			$E_{\mathrm{FID}}$ $\downarrow$			$E_{\mathrm{acc}}$ $\downarrow$
Player	Method	Adaptive Tracker	Adaptive Planner	Sim.	Real-FH	Real-BH	Sim.	Real-FH	Real-BH	Sim.	Real-FH	Real-BH	Sim.	Real-FH	Real-BH
Rafael Nadal	Vid2Player3D	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	82.6	28.0	28.0	68.5	28.5	14.8	7.9	15.4	10.4	45.2	18.8	14.7
	AdaPT (ours)	✓	✓	91.5	44.0	56.0	72.8	27.2	35.7	5.3	19.4	12.9	43.0	19.9	16.3
	w/o-AdaPlanner	✓	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	71.7	44.0	28.0	42.1	27.2	28.5	5.5	22.2	14.3	40.8	16.2	14.5
	w/o-AdaTracker	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	✓	80.6	40.0	36.0	62.9	50.0	44.4	8.0	18.3	12.9	45.7	19.2	19.1
Roger Federer	Vid2Player3D	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	$\mathbin{\raisebox{0.6458pt}{\scalebox{0.8}{$ \bullet $}}}$	81.5	24.0	8.0	88.3	16.6	0.0	5.1	9.1	9.2	41.1	16.5	15.3

Table 3: Real-robot evaluation of rally skills. The results demonstrate that both the Adaptive Tracker and Adaptive Planner contribute to robust real-world rally performance over all three players. The performance gap between simulation and real-world deployment is further analyzed in Sec. 4.5 .

Figure 5: Analysis of failure of RL-Scratch serving.

Player	Method	Succ. $\uparrow$	$E_{\mathrm{Bo}}^{\mathrm{3m}}\downarrow$	$E_{\mathrm{Bo}}^{\mathrm{6m}}\downarrow$	$E_{\mathrm{Bo}}^{\mathrm{9m}}\downarrow$	$E_{\mathrm{dof}}^{\mathrm{track}}\downarrow$	$E_{\mathrm{acc}}\downarrow$
Rafael Nadal	DeepMimic	80.0	0.7 $\pm 0.3$	3.5 $\pm 0.1$	6.4 $\pm 0.6$	0.18 $\pm 0.01$	9.9 $\pm 2.9$
	AdaPT (ours)	66.7	1.5 $\pm 0.5$	1.5 $\pm 0.4$	5.3 $\pm 1.7$	0.17 $\pm 0.01$	12.3 $\pm 0.5$
	w/o-Planner	53.3	1.8 $\pm 1.1$	2.8 $\pm 0.9$	4.6 $\pm 1.7$	0.18 $\pm 0.01$	13.5 $\pm 0.2$
	w/o-TossRew	60.0	1.3 $\pm 0.4$	2.5 $\pm 1.4$	–	0.18 $\pm 0.01$	11.7 $\pm 0.7$
Roger Federer	DeepMimic	53.3	1.6 $\pm 1.0$	4.2 $\pm 1.5$	7.8 $\pm 0.0$	0.36 $\pm 0.03$	28.4 $\pm 0.9$
	AdaPT (ours)	73.3	1.4 $\pm 1.7$	2.8 $\pm 1.2$	6.4 $\pm 1.4$	0.17 $\pm 0.01$	22.0 $\pm 1.0$

Table 4: Real-robot evaluation of serve skills. The results demonstrate that both the Adaptive Tracker and Adaptive Planner contribute to robust real-world rally performance over all three players. Performance drops between simulation and real-world deployment are discussed in Sec. 4.5 .

Figure 6: Planner analysis. The planners successfully learn to adjust motion speed to (a, b) minimize tracking error and (c) adapt to varied ball-tossing height, (d) exhibiting better style than DeepMimic.

Figure 7: Sensitivity of planner-tracker methods to the horizon of the future ball trajectory.

Figure 8: Tossing guidance as a reward is crucial to imitate tossing style and control landing.

Figure 9: Impact of trajectory prediction errors on rally success. (a) Future ball trajectory prediction errors. (b,c) Correlation between pre- and post-bounce prediction errors and rally success, respectively.

Figure 12: Large trajectory estimation noise with camera influence on real performance. But YOLO-based perception is sufficient for serve.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Term	Weight	Description
Racket–ball	$5.0$	Racket–ball contact and MVAE phase alignment; latched after a valid hit.
Opponent landing	$100.0$	Predicted opponent-court bounce; must clear the net ( $\geq 1.1\,\mathrm{m}$ ).
Forward velocity	$50.0$	Deep forward ball speed after hit.
Spin selection	$-1.0$	MVAE spin-skill prediction should be consistent with the desired one.
Self-collision	$-10.0$	Robot self-collisions.
Facing backward	$-20.0$	Base facing away from opponent court.

Appendix

Table 7: Reward functions for rally training.

Term	Weight	Description
Wrist tracking	$1.0$	Tracks striking wrist DoF positions.
Tossing	$5.0$	Enforces accurate ball release position and velocity.
Palm-upward	$1.5$	Encourages an upward palm orientation before the toss.
Hitting	$3.0$	Binary reward for racket-ball strike.
Bounce	$6.0$	Penalizes error between predicted and target landing positions.
Net-crossing	$1.0$	Encourages the ball to pass over the net with height exceeding $1.1\,\mathrm{m}$ .

Appendix

Table 8: Reward functions for serving training.

Physical parameters
Domain Randomization	Sampling Distribution
Foot tangential friction $\mu$	$\mathcal{U}[0.3,\,1.2]$
Encoder bias $\Delta q$	$\mathcal{U}[-0.01,\,0.01]$
Base COM offset $(\Delta x,\,\Delta y,\,\Delta z)$	$\mathcal{U}[-0.08,\,0.08]$
PD gain scale $(k_{p},\,k_{d})$	$\mathcal{U}[0.95,\,1.05]$
Torque limit scale	$\mathcal{U}[0.9,\,1.0]$

Appendix

Table 9: Domain randomization sampling distributions for rally training.

Physical parameters
Domain Randomization	Sampling Distribution
Foot tangential friction $\mu$	$\mathcal{U}[0.3,\,1.2]$
Encoder bias $\Delta q$	$\mathcal{U}[-0.01,\,0.01]$
Base COM offset $(\Delta x,\,\Delta y,\,\Delta z)$	$\mathcal{U}[-0.08,\,0.08]$
PD gain scale $(k_{p},\,k_{d})$	$\mathcal{U}[0.95,\,1.05]$
Torque limit scale	$\mathcal{U}[0.9,\,1.0]$

Appendix

Table 10: Domain randomization sampling distributions for serve training.

Figure 13: Amateur serving motion using the Unitree Dex-3 end-effector.

Figure 14: The setup of in-the-wild serve.

Explore similar work

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data
Mar 13, 2026Zhikai Zhang, Haofei Lu, Yunrui Lian +12Robotics Humanoid Robot Control
Tsinghua University · Galbot Inc. · Shanghai AI Laboratory +2
ATHLETE: Learning Reactive Humanoid Tennis via Trajectory-Guided Motion Matching
Oct 9, 2026Zhihao Tian, Zifan Wang, Enzheng Zhu +4Robot Policy Learning Humanoid Robot Control
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data
Sep 25, 2026Jingzhi Cui, Zhexiong Wang, Bangjie Xu +9Robot Skill Learning Robotics
Tsinghua University · Hong Kong Embodied AI Lab · Zhejiang University +3

Mar 13, 2026J/K move · Enter open · S save

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Zhikai Zhang, Haofei Lu, Yunrui Lian, Ziqing Chen+11

Tsinghua University · Galbot Inc. · Shanghai AI Laboratory+2

Robotics Humanoid Robot Control Robot Imitation Learning

Open paper AskPDF arXiv

Human athletes demonstrate versatile and highly-dynamic tennis skills to successfully conduct competitive rallies with a high-speed tennis ball. However, reproducing such behaviors on humanoid robots is difficult, partially due to the lack of perfect humanoid action data or human kinematic motion data in tennis scenarios as reference. In this work, we propose LATENT, a system that Learns Athletic humanoid TEnnis skills from imperfect human motioN daTa. The imperfect human motion data consist only of motion fragments that capture the primitive skills used when playing tennis rather than precise and complete human-tennis motion sequences from real-world tennis matches, thereby significantly reducing the difficulty of data collection. Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios. With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles. We also propose a series of designs for robust sim-to-real transfer and deploy our policy on the Unitree G1 humanoid robot. Our method achieves surprising results in the real world and can stably sustain multi-shot rallies with human players. Project page: https://zzk273.github.io/LATENT/

Abstract

Explore similar work

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

ATHLETE: Learning Reactive Humanoid Tennis via Trajectory-Guided Motion Matching

Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data