Dynamics-Induced Commitment in Learning-Based Robotic Penalty Kicks

Ruize Geng; Hao E. Zhang; Yisen Li; Yikai Wang; H. Eric Tseng; Ding Zhao

cs.ROSep 17, 2026

Dynamics-Induced Commitment in Learning-Based Robotic Penalty Kicks

Authors: Ruize Geng, Hao E. Zhang, Yisen Li, Yikai Wang, H. Eric Tseng, Ding Zhao

Organizations: Carnegie Mellon University · University of Texas at Arlington

Abstract

Learning in robotic games is constrained not only by strategic information but also by what the body can still execute. We study this coupling in a hierarchical humanoid-quadruped penalty system in which game-level self-play policies command fixed soccer whole-body controllers (S-WBCs). The humanoid shooting skill is initialized from self-collected motion-capture data, whereas the quadruped saving skill is learned by reinforcement learning. We introduce dynamics-induced commitment mapping (DIC-Map), a body-grounded analysis that estimates continuation capability, identifies the first persistent loss of a terminal alternative, and tests whether the remaining interaction admits a reduced zero-sum game. For symmetric terminal alternatives, the reduced game yields a closed-form bound on optimal strategy concentration determined by the responder's value of deferring. We further show that, when the responder acts through an estimator, equal response values eliminate the direct terminal-allocation gradient and leave an estimator-mediated first-order learning channel. Experiments locate commitment about 0.29 s before contact, and changing only ball speed shifts deferral coverage. Across four responder policies, replacing the estimator raises save rate from 0.240 to 0.472, whereas a comparable gain in read accuracy obtained by waiting raises it only to 0.246. Posterior analysis is used for the equilibrium comparison because the available coverage terms are observational proxies. Project website: https://chris-ruizegeng.github.io/penaltykick/

Figures & tables

Fig. 1: Overview of the proposed game-learning framework, where physical dynamics induce commitment, the responder’s deferral option bounds equilibrium concentration, and estimator response values determine the active first-order learning channel during zero-sum self-play.

Fig. 2: Three-layer architecture separating executable whole-body skills, strategic self-play, and DIC-Map. Fixed S-WBCs realize humanoid shooting and quadruped saving, game-level actor–critics revise strategic commands under zero-sum self-play, and DIC-Map maps the resulting physical rollouts to commitment timing, an equilibrium concentration constraint, and the active first-order learning channel.

Fig. 3: Penalty-game instrumentation showing (a) the terminal aim regions, (b) continuation capability with $95\%$ CIs, where $\eta=0.5$ places $t^{c}$ at $0.64$ m or $0.29$ s before contact, and (c) outcome readability under nominal and scripted revision.

Fig. 4: Controlled responder comparison at

\beta=0.8

showing (a) read accuracy versus save rate, where A

\rightarrow

D and A

\rightarrow

F yield similar accuracy gains but save-rate gains of

+0.006

and

+0.232

, respectively, (b) cumulative save rate, (c) policy-F class-conditional

Q_{yy}

\sigma(y,y)

, and

g_{y}

, and (d) terminal responder position versus ball crossing.

Fig. 5: Responder outcomes across evaluation settings showing (a) save, goal, and fall rates for policies A, D, E, and F and (b) a scripted-feint comparison in which deferral attains a save rate of

0.736

versus

0.502

for early-read following, with the reactive-world bars reported using

95\%

CIs.

Policy	Variant	Ep.	Read	Save rate $[95\%\ \mathrm{CI}]$
A	Base	$924$	$0.367$	$0.240\ [0.212,0.268]$
D	Deferral	$1081$	$0.732$	$0.246\ [0.220,0.272]$
E	Deferral+decoy	$1050$	$0.456$	$0.422\ [0.392,0.452]$
F	DAgger estimator	$1041$	$0.722$	$0.472\ [0.442,0.502]$

TABLE I: Responder ablation at

\beta=0.8

against the same reactive initiator, reporting episode count, read accuracy, and save rate with

95\%

CIs.

Fig. 6: Hardware deployment of anticipatory goalkeeping in which Go2 predicts G1’s final shot side during kick preparation, initiates an early interception motion, and blocks the ball.

Fig. 7: Posterior equilibrium analysis showing (a) coverage posteriors, (b) the posterior CDF of

r

, (c) the concentration-ceiling posterior against observed

m

, and (d) the posterior CDF of deviation gain. The broader downstream uncertainty reflects limited support for the mismatched-commitment cell.

	Observed		Posterior prediction
Run	$\tilde{p}_{C}$	$m$	$\bar{\mu}\ [95\%\ \mathrm{CrI}]$	Gain $[95\%\ \mathrm{CrI}]$	$\mathbb{P}(m>\bar{\mu})$
v1	$0.741$	$0.850$	$0.86\ [0.50,1.00]$	$0.000\ [0.000,0.112]$	$0.51$
v2	$0.667$	$0.917$	$0.57\ [0.50,1.00]$	$0.042\ [0.000,0.191]$	$0.78$

TABLE II: Posterior concentration-ceiling comparison across league runs obtained by propagating Jeffreys coverage posteriors through Eqs. 10 – 11 .

Fig. 8: Continuous strategic interaction in which G1 changes its shot direction immediately before ball contact, causing Go2 to move toward the wrong interception location and concede, with the upper sequence showing simulation and the lower sequence the corresponding hardware trial.

Fig. 9: Policy-F state recording showing (a) ball trajectories grouped by terminal event, (b) responder read-lock time against the initiator commitment time, where every sampled episode locks after the ball side is fixed, (c) goal rate, and (d) lateral prediction error.

Fig. 10: Physical timing across ball speed showing (a) the responder spare-time margin

M_{G}

and no-reaction margin

\hat{\Delta}

and (b) measured deferral coverage versus

M_{G}

with

95\%

Jeffreys intervals and cell counts.

Fig. 11: League training dynamics for the two analyzed runs showing (a) live save rate, (b) initiator switches per episode, (c) responder action standard deviation, and (d) save rate against the frozen archive, which increases from

0.51

0.68

Fig. 12: Successful anticipatory goalkeeping during continuous strategic interaction, where Go2 predicts G1’s final shot direction despite late target revision and saves the ball, with the top row showing simulation, the middle row showing separately selected and vertically unsynchronized hardware key frames for each robot, and the bottom row showing the hardware overview.

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 755 papers

Banana Kick: Response-Informed Skill Evolution for Humanoid Soccer
Sep 23, 2026Hao E. Zhang, Ruize Geng, Raihan Haque +5Robot Skill Learning Humanoid Robotics
Carnegie Mellon University · University of Texas at Arlington · General Motors
Anticipatory Robot Goalkeeping via Monotone Optimal Stopping
Sep 21, 2026Hao E. Zhang, Ruize Geng, Yisen Li +8Optimal Stopping Robotics
Carnegie Mellon University · University of Texas at Arlington · General Motors
RoboStriker: Latent-Space Strategic Games for Autonomous Humanoid Boxing
Aug 17, 2026Kangning Yin, Kaige Liu, Zhe Cao +9Humanoid Robotics Reinforcement Learning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute +2

Sep 23, 2026J/K move · Enter open · S save

Banana Kick: Response-Informed Skill Evolution for Humanoid Soccer

Hao E. Zhang, Ruize Geng, Raihan Haque, Khalil Zbiss+4

Carnegie Mellon University · University of Texas at Arlington · General Motors

Robot Skill Learning Humanoid Robotics

Open paper AskPDF arXiv

Humanoid kicking requires coordinated whole-body motion and precise contact, while a banana kick demands contact mechanics that generate ball spin and aerodynamic curvature. Motion imitation provides a reliable ordinary-kick prior, but reinforcement learning may improve shot speed and placement accuracy without changing the underlying kicking technique. Adapting this prior to a qualitatively different contact-rich skill can fail even when the reward is dense and optimization remains stable. The failure occurs when the task objective is locally flat over the current policy's responses. We term this condition first-order learning starvation. To address it, we propose response-informed skill evolution (RISE), a closed-loop objective-continuation method for policy adaptation. RISE ranks bounded objective changes using response sensitivity estimated from cached rollouts and accepts updates only when they produce verified response progress while preserving kicking reliability. Our analysis shows that rescaling a saturated spin reward cannot recover first-order sensitivity at zero spin, whereas adapting coupled contact responses can provide a learnable path to spin generation. We integrate RISE into a humanoid kicking pipeline under calibrated contact and Magnus-force aerodynamics, and sim-to-real transfer. Experiments show that RISE evolves the ordinary kick into a high-spin curved kick with 11.55 rad/s mean ball spin, improves the mean evaluation score by 19.8% over a learning-progress curriculum, and raises joint target attainment from 15.2% to 50.9%. Ablations and response diagnostics support the mechanism, while 30 motion-capture-recorded physical trials demonstrate consistent hardware transfer of the learned curved kick. Project website: https://haozhang-thu.github.io/bananakick/

Abstract

Robotics

Explore similar work

Banana Kick: Response-Informed Skill Evolution for Humanoid Soccer

Anticipatory Robot Goalkeeping via Monotone Optimal Stopping

RoboStriker: Latent-Space Strategic Games for Autonomous Humanoid Boxing