DAVIS: A Depth-Only End-to-End Active-Vision Framework for Humanoid Soccer Skills

Jiakang Jin; Yixiao Huo; Pengyuan Wang; Yinan Han; Tingxuan Zhang; Zhuobing Zhao; Xuanxin Zhou; Zhangchen Ye

cs.ROSep 23, 2026

DAVIS: A Depth-Only End-to-End Active-Vision Framework for Humanoid Soccer Skills

Authors: Jiakang Jin, Yixiao Huo, Pengyuan Wang, Yinan Han, Tingxuan Zhang, Zhuobing Zhao, Xuanxin Zhou, Zhangchen Ye,

Organizations: Noetix Robotics · Tsinghua University

Abstract

Humanoid soccer contact skills require more than producing high-impact foot-ball contacts: the robot must close the loop over perception, approach, alignment, impact, and recovery while its own motion induces substantial viewpoint changes, frequent loss of the ball from view, and uncertain contact outcomes. In this work, we ask a compact yet stricter question: can a humanoid learn soccer contact skills using only a head-mounted depth image, proprioceptive history, and an optional low-dimensional task command, and directly output 25-DoF joint PD targets without extra runtime perception or planning modules? To this end, we propose DAVIS, a depth-only end-to-end framework for humanoid soccer skills that learns visibility-aware auxiliary geometry during training, and combines GT-to-prediction annealing, task curricula, and AMP-style motion priors to smoothly bridge privileged supervision and real deployment. Built on this framework, we instantiate representative soccer contact skills, including goal-directed shooting and directional dribbling, through task-specific definitions of objects, commands, rewards, and curricula, and validate them through simulation, Noetix E1 real-robot experiments, and ablations.

Figures & tables

Fig. 2: Overview of our DAVIS framework. The observation stream contains the command $c_{t}$ , a single $168\times 80$ depth image , proprioceptive history, and privileged simulation states. The structured-depth module encodes the depth input into a 32-D latent, predicts ball/goal geometry and visibility confidence through auxiliary heads , forms confidence-gated geometry features, and combines them with a 64-D HIM/history feature. The actor receives the command, depth latent, gated geometry/confidence features, proprioceptive history, and HIM feature, and outputs a 25-D position residual for the PD controller. The auxiliary losses, GT-to-prediction annealing, PPO update, task reward, and style reward use simulation-only signals from IsaacLab rollouts, including ball, goal, and visibility ground truth. Dashed boxes in the structured-depth module indicate optional task-specific modules.

Fig. 5: Qualitative examples of learned soccer skills. In simulation: (a) curved and penalty kicks, (b) cone-avoidance dribbling, (d) dribbling across a soccer field, and (f) stopping a rolling ball, where red and blue arrows indicate robot and ball motion, respectively, and the purple box marks the stopped ball. On the real robot: (c) foot–ball contact during dribbling, with arrows indicating ball motion, and (e) a close-up dribbling sequence.

Goal-Directed Shooting						Directional Dribbling
Setting	Metric	Easy	Med.	Hard	Total	Setting	Metric	$N=3$			$N=5$			$N=7$
								$60^{\circ}$	$90^{\circ}$	$120^{\circ}$	$60^{\circ}$	$90^{\circ}$	$120^{\circ}$	$60^{\circ}$	$90^{\circ}$	$120^{\circ}$
Sim.	Penalty SR $\uparrow$	0.95	0.96	0.56	0.85	Sim.	SR $\uparrow$	0.71	0.77	0.65	0.66	0.72	0.55	0.68	0.64	0.43
	Free-kick SR $\uparrow$	1.00	0.88	0.71	0.85		$E_{\mathrm{vel}}\downarrow$	0.69	0.74	0.91	0.74	0.73	0.80	0.71	0.81	0.97
	–	–	–	–	–		$R_{\mathrm{bib}}\uparrow$	0.73	0.74	0.64	0.76	0.75	0.70	0.76	0.72	0.60
Real	Penalty SR $\uparrow$	0.68(26/38)	0.57(28/49)	0.55(33/60)	–	Real	SR $\uparrow$	Easy: 0.79(22/28)			Medium: 0.68(19/28)			Hard: 0.61(17/28)

TABLE I: Quantitative results for goal-directed shooting and directional dribbling in simulation and on the real robot. Shooting entries and real-robot dribbling entries report success rate (SR) on a 0–1 scale; simulation dribbling reports SR, velocity error (

E_{\mathrm{vel}}

), and ball-in-band ratio (

R_{\mathrm{bib}}

) over 900 repeated-S slalom trials. Arrows indicate the preferred direction, and bold marks the best result within each fixed-

N

simulation-dribbling group. For shooting, a success requires the ball to cross the goal line in one kick. For dribbling, a success requires the robot to remain upright throughout execution and both the robot and ball to finish within 1 m of the target.

Fig. 6: (a) Free-kick and Penalty-kick performance in simulation. Each region uses over 50 trials. Darker colors indicate higher success rates, and dashed lines mark difficulty regions. (b) Footstep trajectories during shooting. The policy shows a smooth gait and consistent speed pattern.

Fig. 7: Performance of repeated S-slalom in simulation. (a) Statistical results of 100 S-slalom trajectories. (b) Foot-contact record from a representative successful trajectory.

Method	Penalty kick			Free kick			Repeated-S slalom
	SR $\uparrow$	$E_{\mathrm{ang}}\downarrow$	$r_{\mathrm{contact}}\uparrow$	SR $\uparrow$	$E_{\mathrm{ang}}\downarrow$	$r_{\mathrm{contact}}\uparrow$	SR $\uparrow$	$E_{\mathrm{vel}}\downarrow$	$r_{\mathrm{bib}}\uparrow$
DAVIS	0.84	23.29	99.75	0.80	30.18	99.27	0.70	0.79	0.74
(a) Ablation
DAVIS-w/o-curriculum	0.62	27.40	32.53	0.55	34.42	87.67	0.51	0.77	0.69
DAVIS-auxiliary head-w/o-anneal	0.32	54.79	84.87	0.22	76.20	87.92	0.31	0.83	0.66
DAVIS-auxiliary head-w/o-geometry	0.20	53.53	87.33	0.15	69.47	78.13	0.34	0.91	0.54

TABLE II: Simulation ablations and comparison. (a) Ablations. Penalty-kick and free-kick results use over 900 shooting trajectories, and repeated-S slalom uses over 100 dribbling trajectories. Metrics are success rate (SR), body-to-goal angular error (

E_{\mathrm{ang}}

), successful ball-contact ratio (

r_{\mathrm{contact}}

), velocity error (

E_{\mathrm{vel}}

), and time in the control band (

r_{\mathrm{bib}}

). The full method provides the best overall balance across tasks and metrics. (b) Comparison between our end-to-end learned head control (Active vision) and a heuristic head controller (Heuristic) that gazes at the predicted ball. The heuristic performs comparably to active vision in static-object tasks (e.g., penalty kicks), while active vision shows a clear advantage in highly dynamic dribbling scenarios.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Definition
Success rate (SR)	The main paper defines task-level success: shooting requires the ball to cross the goal line in one kick, while dribbling requires the robot to remain upright and both the robot and ball to finish within $1$ m of the target. This appendix expands that definition for evaluation protocols: simulated dribbling additionally requires visiting all waypoints in order (Sec. IV-A ), whereas the real-world protocol applies the same endpoint and uprightness criteria to each trial.
Valid contact ratio ( $r_{\mathrm{contact}}$ )	Fraction of shooting trials with a valid foot–ball contact and goal-directed ball exit.
Angular error ( $E_{\mathrm{ang}}$ )	Heading error between the robot body and the robot-to-goal direction.
Velocity error ( $E_{\mathrm{vel}}$ )	Mean error between commanded and actual ball velocity during dribbling.
Ball-in-band ratio ( $R_{\mathrm{bib}}$ )	Fraction of engaged dribbling time in which the ball stays inside the controllable band.
Turn bias ( $B_{\mathrm{turn}}$ )	$\|\bar{\theta}_{\mathrm{achieved}}-\theta_{\mathrm{cmd}}\|/\theta_{\mathrm{cmd}}$ over scored rounds under the Dribble Master turning protocol; the statistic behind its reported turning error.

Appendix

TABLE III: Metric definitions used throughout the evaluation.

Fig. 8: Multi-position Shooting Simulation and Real Machine Difficulty Level Classification

Field	Definition	Value
Task grid	Repeated-S slalom with trajectory turns $N$ and turn angle $\alpha$	$N\in{3,5,7}$ , $\alpha\in{60,90,120}^{\circ}$
Waypoints	$N{+}2$ turning waypoints plus $N{+}1$ midline gate points	$2N{+}3$ total
Geometry	Fixed arm length for waypoint generation	$L=4.0$ m
Command	Waypoint attraction and pole repulsion, snapped to 12 direction bins	$k_{\mathrm{att}}=1.0$ , $k_{\mathrm{rep}}=0.6$ , $d_{0}=1.2$ m
Ball spawn	Fixed point directly in front of the robot	0.6 m
Engagement	Metrics start after first ball contact	threshold 0.15 m/s; timeout 5 s

Appendix

TABLE IV: Directional dribbling simulation fixture.

Fig. 9: Trajectory visualization for the repeated-S slalom dribbling benchmark. Each subplot corresponds to one task configuration $(N,\alpha)$ , where $N$ is the number of turns and $\alpha$ is the turning angle. The colored curve shows the mean successful ball trajectory with speed-based coloring, and the shaded region indicates the 15–85% success band. The start point and target region are marked for reference. The title of each subplot reports the success rate and the number of successful rounds.

Field	Dribble Master [ 27 ]	DAVIS reproduction
Robot / policy input	Booster T1; detector ball position + in-view flag	Noetix E1; depth image
Command	global ball-velocity vector, issued by a human operator	direction only, scripted (potential field for reaching, fixed heading for turning)
Reaching / obstacle	unreported distance; success = final error $<1$ m	goal at $d\in\{3,5,8\}$ m; success = minimum ball–goal distance $<\tau$ , $\tau\in\{0.5,1.0,1.5\}$ m; round ends at $0.5$ m or after $\max(20,6d)$ s
Obstacle	physical; not perceived by the policy	virtual pole at the path midpoint; not perceived by the policy
Turning	$\pm 45^{\circ}/90^{\circ}$ on entering a $0.4$ m circle at $(1.5,0)$ m	$\pm 30/45/60/90^{\circ}$ on $1.1$ m of forward progress (where a straight path enters that circle)
Trials	5 rollouts per turn condition (sim); 15 per task (real)	50 rounds per condition (sim), Wilson 95% CI

Appendix

TABLE V: Dribble Master’s published dribbling protocol and our reproduction of it on DAVIS.

Task	$d$ (m)	$\mathrm{RR}_{0.5}$	$\mathrm{RR}_{1.0}$	$\mathrm{RR}_{1.5}$	time (s)
Target reaching	3	0.88 [0.76, 0.94]	0.88 [0.76, 0.94]	0.90 [0.79, 0.96]	$3.0\pm 0.6$ (44)
	5	0.82 [0.69, 0.90]	0.84 [0.71, 0.92]	0.84 [0.71, 0.92]	$5.7\pm 0.5$ (42)
	8	0.78 [0.65, 0.87]	0.80 [0.67, 0.89]	0.80 [0.67, 0.89]	$9.4\pm 1.1$ (40)
Single obstacle	3	0.92 [0.81, 0.97]	0.92 [0.81, 0.97]	0.92 [0.81, 0.97]	$4.8\pm 1.1$ (46)
	5	0.94 [0.84, 0.98]	0.96 [0.87, 0.99]	0.96 [0.87, 0.99]	$6.4\pm 1.1$ (48)
	8	0.94 [0.84, 0.98]	0.94 [0.84, 0.98]	0.94 [0.84, 0.98]	$9.9\pm 1.1$ (47)

Appendix

TABLE VI: Target-reaching and single-obstacle results under the Dribble Master protocol (DAVIS simulation,

n{=}50

rounds per row, Wilson 95% CI). Success is reported at three tolerances, together with the time from first ball contact to the

1

m tolerance (mean

\pm

SD over the rounds that got there, count in parentheses); a round ends when the ball first comes within

0.5

m. Dribble Master’s reaching and obstacle values come from real-robot trials with

n{=}15

at an unreported distance.

Turn command	DAVIS				Dribble Master
	scored	$\bar{\theta}\pm\sigma$ ( ∘ )	bias (%)	per-round (%)	$\bar{\theta}$ ( ∘ )	bias (%)
30 ∘ L	40	$+28.6\pm 20.5$	4.8	52.9	–	–
30 ∘ R	39	$-25.5\pm 15.3$	15.1	41.5	–	–
45 ∘ L	44	$+39.3\pm 14.9$	12.6	27.6	+43.58	3.16
45 ∘ R	44	$-44.8\pm 18.2$	0.5	32.1	-46.44	3.20
60 ∘ L	43	$+60.4\pm 19.5$	0.6	26.3	–	–

Appendix

TABLE VII: Turning results under the Dribble Master protocol (DAVIS simulation,

n{=}50

rounds per condition). Achieved heading change is reported as mean

\pm

SD over scored rounds, together with the bias of the mean. Dribble Master’s turning values come from MuJoCo with 5 rollouts per condition.

Fig. 11: Multi-directional shooting real-world example

Difficulty	Distance (m)	Target Angles
Easy	2.5	$-30^{\circ}$ , $-5^{\circ}$ , $+5^{\circ}$ , $+30^{\circ}$
Medium	3.0	$-30^{\circ}$ , $-5^{\circ}$ , $+5^{\circ}$ , $+30^{\circ}$
Hard	2.5–3.0	$-45^{\circ}$ , $+45^{\circ}$

Appendix

TABLE VIII: Target point configurations and difficulty levels used in the real-world shooting evaluation.

Fig. 12: Foot snapshot in dribbling experiment.

Item	Configuration
Actor sensory input	Single-frame depth image + 5-step proprioceptive history
Optional command	Low-dimensional task command if enabled
Shooting proprioception	81-D
Dribbling proprioception	84-D
Depth encoder	LightDepthEncoder
Depth latent	32-D

Appendix

TABLE IX: Observation and network configuration.

Item	Value
Rollout length	24 steps per environment
PPO epochs	5
Mini-batches	4
Learning rate	$1\times 10^{-3}$ , adaptive schedule
Discount factor	$\gamma=0.99$
GAE parameter	$\lambda=0.95$

Appendix

TABLE X: PPO and AMP-HIM optimization settings.

Fig. 13: Loss curve of GT-annealing and w/o GT-annealing

Reward Terms	Components	Weight
Approach and chase	Approach ball reward	3.0
	Adaptive velocity reward	2.0
Active vision	Head track ball reward	3.0
	Keep in horizon reward	-0.5
	Body follow head reward	2.0
Ball–goal alignment	Robot behind ball reward	1.0

Appendix

TABLE XI: Shooting reward summary.

Group	Terms	Weight / setting
Commanded progress & heading	Ball command reward	5.0
	Body command reward	0.4
Ball control	Dense re-approach reward	2.0
	Out-of-band penalty	-1.0
Active vision	Ball in view reward	1.0
	Body follow head reward	0.6

Appendix

TABLE XII: Dribbling reward and regularization summary.

Perturbation	Range / value
Valid depth range	0.3–5.0 m
Distance-proportional depth noise	0.01
Gaussian depth-noise coefficient	0.005
Salt-and-pepper probability	0.01
Stripe dropout probability	0.005
Edge-drag probability	0.01

Appendix

TABLE XIII: Representative training randomization ranges for sim-to-real transfer.

Fig. 15: Auxiliary loss for the ball and obstacles. The prediction heads for both objects converge during training.

Robot Skill Learning

Robot skill learning studies how robots acquire, adapt, and combine reusable behaviors from demonstrations, interaction, and other data.

See all 323 papers

Vision-Based Dribbling for Humanoid Soccer via Privileged Representation Learning
Jul 14, 2026Flavio Maiorana, Valerio Spagnoli, Eugenio Bugli +5RL for Robotics Reinforcement Learning
Dept. of Computer, Control, and Management Engineering Sapienza University of Rome, Rome (Italy) · Institut de Robòtica i Informàtica Industrial (CSIC-UPC), C/ Llorens i Artigas 4-6, 08028, Barcelona, Spain · École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
SkillX: Unified Multi-Skill Policy Learning for Humanoid Soccer
Sep 6, 2026Zhangchen Ye, Enxuan Ruan, Yifei Bao +10Robot Skill Learning Humanoid Robot Control
Noetix Robotics · Tsinghua University
Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Library
Sep 30, 2026Abu Hanif Muhammad Syarubany, Jaehyun Jang, Hwanhee Kim +3Humanoid Robot Locomotion Robot Skill Learning
School of Electrical Engineering, KAIST, Daejeon, Republic of Korea

Jul 14, 2026J/K move · Enter open · S save

Vision-Based Dribbling for Humanoid Soccer via Privileged Representation Learning

Flavio Maiorana, Valerio Spagnoli, Eugenio Bugli, Flavio Volpi+4

Dept. of Computer, Control, and Management Engineering Sapienza University of Rome, Rome (Italy) · Institut de Robòtica i Informàtica Industrial (CSIC-UPC), C/ Llorens i Artigas 4-6, 08028, Barcelona, Spain · École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

RL for Robotics Reinforcement Learning Humanoid Robot Control Robotics Representation Learning for RL Vision-Based Robot Control

Open paper AskPDF arXiv

Recent advances in humanoid robotics have highlighted the importance of deployable loco-manipulation skills. Dribbling a soccer ball while evading active opponents requires simultaneous balance, precise ball control, and awareness of a dynamic adversary under onboard sensing and real-time constraints. Existing approaches typically separate perception and motion, which can be effective in controlled settings but may fail under occlusions, fast ball movements, and complex opponent interactions, since perception is not directly optimized for control. We propose an integrated approach in which a temporal depth encoder is embedded into a reinforcement learning policy through a task-specific projection layer. We apply this framework to a simulated Booster T1 humanoid robot and show that it is possible to learn vision-based, opponent-aware dribbling directly from depth observations, without explicit state estimation or privileged scene information. The learned policy achieves 100% success in nominal target-driven dribbling and 96% success with a single static obstacle, while reaching 46% success against an actively moving ball-attacker opponent. These results demonstrate that the proposed framework supports robust vision-based dribbling in nominal and moderately dynamic settings, and provides a strong foundation for handling more challenging moving-adversary scenarios.

Abstract

Robot Skill Learning

Explore similar work

Vision-Based Dribbling for Humanoid Soccer via Privileged Representation Learning

SkillX: Unified Multi-Skill Policy Learning for Humanoid Soccer

Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Library