Rapid On-Robot Learning for Dynamic Manipulation Skills: Robot Juggling

Taeyoon Lee; Chunpeng Wang; Christopher G. Atkeson; Alfred A. Rizzi; Nicolas Rojas

cs.ROAug 27, 2026

Rapid On-Robot Learning for Dynamic Manipulation Skills: Robot Juggling

Authors: Taeyoon Lee, Chunpeng Wang, Christopher G. Atkeson, Alfred A. Rizzi, Nicolas Rojas

Organizations: Robotics and AI Institute (RAI), Cambridge MA, USA · Robotics Institute, Carnegie Mellon University, Pittsburgh PA, USA

Abstract

We present an online learning framework that enables a bimanual robot to acquire diverse juggling patterns directly on physical hardware within minutes, even with a significant sim2real gap. One of the most important lessons from this work is that a model, even when far from reality, can be extremely useful for learning. This motivates a central philosophy of our approach: learning should build upon the robot's current knowledge rather than replace it. Our regularized memory-based learning puts this principle into practice by learning a local model from accumulated experience while retaining the global prior model to extrapolate where experience is sparse. This enables efficient and stable online learning from each new experience without resorting to uninformed exploration over a vast space of possible behaviors. Equally important to continual on-robot learning is safety, allowing the robot to repeatedly practice and improve in the real world. We construct a mutually reachable set that allows safe transitions between successive throws and catches, without driving either arm into a state from which its next action would require violating the robot's joint or actuator limits. Together, these ideas enable a bimanual robot with multi-fingered hands and onboard vision to safely learn and compose five canonical three-ball juggling patterns, including cascade, tennis, half-shower, shower, and box, within less than 5 minutes of real-world interaction. More broadly, this work points toward robots that build upon imperfect prior knowledge and continually refine their behavior through their own real-world experience.

Figures & tables

List of Movies 1 Robot juggling five distinct three-ball patterns. The bimanual robot AthenaZero continuously juggles the cascade, half-shower, tennis, shower, and box patterns, leveraging onboard vision and multi-fingered hands.

Work	Interaction	Execution
feedback	On-robot
learning	Physical
experience	Demonstration
Buehler et al. [ 7 ]	Paddle	Closed-loop	–	–	1-ball bounce juggling
Rizzi and Koditschek [ 32 ]	Paddle	Closed-loop	–	–	2-ball bounce juggling
Aboaf et al. [ 1 ]	Paddle	Closed-loop	Parametric	160 attempts	1-ball bounce juggling

Table 1: Comparison of representative robotic juggling approaches.

Figure 1: Hierarchical, skill-based system architecture and real-time information flow for on-robot learning. The Orchestrator dispatches skill descriptions to the Skill Executor , where the target outcome ( $y_{d}$ ) is decoded. The Regularized Memory-Based Learner uses the experience Memory to adapt the command ( $u$ ), while the model-based Control System generates joint trajectories for the physical robot. A precomputed Mutually Reachable Set constrains the task-level plan so that successive skills can be safely composed without violating the robot’s physical limits. An asynchronous Perception module tracks object states ( $x$ ) to provide feedback for closed-loop execution. The resulting outcome ( $y$ ) is stored in memory and immediately becomes available for refining commands.

Parameter	Type / Set	Description
Type	$\in\{\texttt{throw},\penalty\ \texttt{catch}\}$	High-level robotic action
Object ID	$\in\{0,\penalty\ 1,\penalty\ 2\}$	Targeted juggling ball index
Task Identifier	String	Semantic routing label (e.g., toR3fromL1 )
Duration	Float (seconds)	Target execution timeframe (e.g., 0.35)

Table 2: Juggling skill description parameters

Figure 2: Learning curves across distinct juggling patterns. The x-axis tracks successive juggling runs separated by manual resets, and the y-axis denotes the consecutive throw count attained in each run. Learning curves of individual trials (1–5), where each trial starts learning from scratch and proceeds across a sequence of resets, are shown in lighter colors, with the solid black line indicating the mean $\pm$ standard deviation. The dashed red line marks the consecutive throw count corresponding to five complete cycles of each juggling pattern. (A) The cascade is rapidly acquired, consistently reaching the five-cycle target after several resets. (B) During continuous learning across Tennis $\rightarrow$ Half-Shower $\rightarrow$ Cascade, experience is retained as the robot transitions between patterns, with the cascade acquired rapidly after learning tennis and half-shower. (C,D) Performance on the more difficult shower and box patterns progressively improves with physical experience, although neither pattern consistently reaches the five-cycle target.

Figure 3: Evolution of 3D ball trajectories and landing distributions during on-robot learning. Each row tracks the spatial distribution of ballistic flight arcs, and ball landing locations over sequential resets for different juggling patterns: ( A ) Cascade, ( B ) a multi-pattern transition sequence (Tennis $\rightarrow$ Half-Shower $\rightarrow$ Cascade), ( C ) Shower, and ( D ) Box. Moving from left to right across successive resets, the scatter points become tightly clustered around their designated targets, with reduced variance in the flight trajectories, demonstrating the sample-efficient improvement achieved through real-time adaptation.

Figure 4: Impact of prior regularization on learning curves. Evaluation is conducted on the cascade juggling pattern across varying choices of the prior regularization coefficient

\gamma>0

. A nonzero

\gamma

regularizes the locally fitted model toward the prior, with larger values placing greater weight on the prior relative to the observed experience. For each coefficient value, the robot executed five independent trials, each learning from scratch. The curves show the mean

\pm

standard deviation across trials to highlight performance trends.

Juggling pattern	Cascade	Tennis
$\rightarrow$ Half-Shower
$\rightarrow$ Cascade	Shower	Box	All
Number of experience (throw) data over all runs	1198	2256	1634	2490	7578
Without MRS constraint (retrospective ablation)
Inside MRS	16 (1.3%)	12 (0.5%)	25 (1.5%)	14 (0.6%)	67 (0.9%)
\SetRow bg=gray!15 Outside MRS	1182 (98.7%)	2244 (99.5%)	1609 (98.5%)	2476 (99.4%)	7511 (99.1%)

Table 3: Reachability and viability of task-level planner solutions with and without the Mutually Reachable Set (MRS) constraint across all juggling experiments. The unconstrained solutions are obtained retrospectively from the same planner queries after removing the MRS constraint. Solutions outside the MRS are further classified as safe when they are both reachable from the preceding transition state and viable for subsequent motion, and unsafe otherwise. Because the MRS is a conservative inner approximation, states outside the set are not necessarily unsafe. Percentages are computed with respect to the total number of planner queries for each pattern.

Figure 5: Effect of the Mutually Reachable Set on target transition states during juggling. The translucent blue polytopes show the precomputed MRS for representative left-arm joints in joint position, velocity, and acceleration space. Blue points denote target transition states generated with the MRS constraint. For the corresponding retrospective ablation without the constraint, green points remain inside the MRS, while states outside the set are classified as reachable and viable (orange), reachable but not viable (purple), viable but not reachable (brown), or neither reachable nor viable (red). Insets highlight regions in which the unconstrained task-level planner produces states outside the MRS. The examples show both potential major failure modes (without MRS) during juggling behaviors on AthenaZero: states that are reachable but not viable and states that are viable but not reachable.

Figure 6: Safe dynamic composition of successive skills using the Mutually Reachable Set ( $\mathcal{M}_{S}$ ). (Top) The Task-Level Planner maps a task command $\mathbf{u}$ from the Learner to a target joint transition state $\mathbf{s}_{f}^{*}$ while constraining the solution to $\mathcal{M}_{S}$ . (Bottom left) State-space representation of the Mutually Reachable Set $\mathcal{M}_{S}$ , illustrating successive transition states within the set and the feasible trajectories connecting them. (Bottom right) Illustration of why satisfying instantaneous joint limits alone is insufficient for safe composition. A state may be reachable from the current state but leave the joint unable to avoid a subsequent position limit ( reachable, but not viable ), or may itself allow safe future motion but not be reachable from the current state without violating the limits ( viable, but not reachable ). Restricting transition states to $\mathcal{M}_{S}$ excludes both cases.

Figure S1: Shared target positions for juggling skills. The six designated spatial targets for the right arm ( $R1$ – $R6$ ) and left arm ( $L1$ – $L6$ ) are shared across the target release, target land/catch configurations. Active targets utilized in a specific juggling pattern are highlighted in red (e.g., $R1$ , $R2$ , $L1$ , and $L2$ are the enabled positions for the cascade pattern).

ID	$x$ (m)	$y$ (m)	$z$ (m)	ID	$x$ (m)	$y$ (m)	$z$ (m)
R1	- 0.056	0.415	1.35	L1	-0.056	0.415	1.35
R2	- 0.206	0.415	1.35	L2	-0.206	0.415	1.35
R3	- 0.206	0.415	1.50	L3	-0.206	0.415	1.50
R4	- 0.356	0.415	1.50	L4	-0.356	0.415	1.50
R5	- 0.226	0.415	1.35	L5	-0.226	0.415	1.35
R6	- 0.226	0.415	1.45	L6	-0.226	0.415	1.45

Table S1: Coordinates of the shared target positions for juggling skills. The

x

y

-, and

z

-axes correspond to the rightward, forward, and upward directions relative to the robot base, respectively.

Figure S2: Mutually reachable set construction for the right arm joints ( RJ1 – RJ7 ) of AthenaZero. Each subplot displays the overlay of the forward reachable set $\mathcal{R}_{f}^{(5)}$ (light blue), backward reachable set $\mathcal{R}_{b}^{(5)}$ (light red), and their resulting intersection—the mutually reachable set $\mathcal{M}_{S}=\mathcal{R}_{f}^{(5)}\cap\mathcal{R}_{b}^{(5)}$ (purple). The volumes are mapped within the joint phase-space coordinates of position $q$ , velocity $\dot{q}$ , and acceleration $\ddot{q}$ , illustrating the dynamically feasible target joint state constraint for each independent degree of freedom.

	Throw	Catch
Task-level command ( $\mathbf{u}$ )	$\mathbf{p}^{u}_{\mathrm{land}}$	$\mathbf{p}_{\mathrm{catch}}$
Optimization variable ( $\mathbf{s}_{f}$ )	$(\mathbf{q}_{f},\dot{\mathbf{q}}_{f},\ddot{\mathbf{q}}_{f})$	$(\mathbf{q}_{f},\mathbf{0},\mathbf{0})$
Objective ( $\phi$ )	$\omega_{p}\lVert\mathbf{q}_{f}\rVert^{2}+\omega_{v}\lVert\dot{\mathbf{q}}_{f}\rVert^{2}+\omega_{a}\lVert\ddot{\mathbf{q}}_{f}\rVert^{2}$	$\omega_{p}\lVert\mathbf{q}_{f}\rVert^{2}+\omega_{\theta}(1-\hat{\mathbf{n}}_{\mathrm{catch}}^{\top}\hat{\mathbf{n}}(\mathbf{q}_{f}))$
	$+\omega_{\theta}(1-\hat{\mathbf{n}}_{\mathrm{throw}}^{\top}\hat{\mathbf{n}}(\mathbf{q}_{f}))$
Constraints ( $\mathbf{g}$ )	$\mathbf{p}_{\mathrm{release}}=\mathbf{f}_{\mathrm{FK}}(\mathbf{q}_{f})$	$\mathbf{p}_{\mathrm{catch}}=\mathbf{f}_{\mathrm{FK}}(\mathbf{q}_{f})$
	$\mathbf{v}_{\mathrm{throw}}=\mathbf{J}(\mathbf{q}_{f})\dot{\mathbf{q}}_{f}$

Table S2: Optimization formulations of the task-level planner for throwing and catching tasks. The planner maps high-level task commands

\mathbf{u}

to target joint states

\mathbf{s}_{f}

Phase	Right Arm Skill	Left Arm Skill
Start	[throw, 0, toL2fromR1FIP, 0.7]	[catch, -1, atL2fromR1, 0.7]
	[catch, -1, atR2fromL1, 0.3]	[throw, 1, toR2fromL1Init, 0.35]
	[throw, 2, toL2fromR1Init, 0.4]	[catch, 0, atL2fromR1, 0.35]
Repeat	Repeat the following sequence 5 times:
	[catch, 1, atR2fromL1, 0.35]	[throw, 0, toR2fromL1, 0.35]
	[throw, 1, toL2fromR1, 0.35]	[catch, 2, atL2fromR1, 0.35]

Table S3: Skill sequence configuration for the cascade juggling pattern. The Type parameter explicitly encompasses throw and catch . The Object ID specifies the tracked ball index (

0,1,2

), where

-1

denotes an open-loop execution that bypasses visual tracking during a catch phase. The Task Identifier utilizes spatial targets (

R1,R2,L1,L2

) appended with specialized suffixes: FIP and HIP indicate fully opening or half-opening the index and pinky fingers while the thumb remains fully closed, whereas Init distinguishes introductory phase throws to facilitate memory-based learning. For skills within the stabilized cycle, the step duration is fixed at

0.35\text{ s}

, exactly half of the

0.7\text{ s}

ballistic object flight time.

Phase	Right Arm Skill	Left Arm Skill
Start	Identical to the cascade pattern start sequence
Repeat	Phase A: Tennis Pattern. Repeat the following sequence 5 times:
	[catch, 1, atR2fromL1, 0.35]	[throw, 0, toR3fromL4, 0.35]
	[throw, 1, toL2fromR1, 0.35]	[catch, 2, atL2fromR1, 0.35]
	[catch, 0, atR3fromL4, 0.35]	[throw, 2, toR2fromL1, 0.35]
	[throw, 0, toL3fromR4, 0.35]	[catch, 1, atL2fromR1, 0.35]

Table S4: Skill sequence configuration for tennis, half-shower, and cascade patterns. The skill formatting (Type, Object ID, and Duration) remains consistent with Table S3 . The Task Identifier strings incorporate expanded spatial targets (

R3,R4,L3,L4

) alongside the cascade coordinates to accommodate the altered ballistic trajectories required across the distinct phases.

Phase	Right Arm Skill	Left Arm Skill
Start	[throw, 0, toL6fromR5FIP, 0.7]	[catch, -1, atL6fromR5, 0.35]
	[catch, -1, atR5fromL6, 0.24]	[catch, -1, atL6fromR5, 0.35]
	[throw, 2, toL6fromR5Init, 0.24]	[throw, 1, toR5fromL6Init, 0.48]
Repeat	Repeat the following sequence 5 times:
	[catch, -1, atR5fromL6, 0.24]	[catch, 0, atL6fromR5, 0.24]
	[throw, 1, toL6fromR5, 0.24]	[throw, 0, toR5fromL6, 0.24]

Table S5: Skill sequence configuration for the shower juggling pattern. The skill formatting (Type and Object ID) remains consistent with Table S3 . The Task Identifier utilizes spatial targets (

R5,L6

) appended with specialized behavior suffixes: FIP and HIP indicate fully opening or half-opening the index and pinky fingers while the thumb remains fully closed. The Init string distinguishes initial phase throws to facilitate memory-based learning. Within the stabilized cycle, the step duration is fixed at

0.24\text{ s}

, corresponding to the flight time of the lower horizontal pass and exactly one-third of the

0.72\text{ s}

ballistic flight time of the higher arc throw.

Phase	Right Arm Skill	Left Arm Skill
Start	[throw, 0, toL6fromR5FIP, 0.7]	[catch, -1, atL6fromL5, 0.35]
	[catch, -1, atR6fromR5, 0.24]	[catch, -1, atL6fromL5, 0.35]
	[throw, 2, toR6fromR5Init, 0.24]	[throw, 1, toR5fromL6Init, 0.48]
	[catch, -1, atR5fromL6, 0.24]	[catch, 0, atL6fromR5, 0.24]
Repeat	Repeat the following sequence 5 times:
	[throw, 1, toL5fromR6, 0.24]	[throw, 0, toL6fromL5, 0.24]

Table S6: Skill sequence configuration for the box juggling pattern. The skill formatting (Type, Object ID, and Duration) remains consistent with Table S5 . The Task Identifier extends spatial targets with

R6,L5

and maintains the same special suffixes in Table S5 .

Dynamic Robotic Manipulation

Dynamic robotic manipulation studies how robots act on moving or deformable objects and respond to fast-changing physical interactions.

See all 94 papers

Jun 15, 2026

Task-Error Residual Learning for Real-Robot Five-Ball Juggling

For residual learning that refines existing behavior, sample efficiency depends on two things: how much information each rollout returns, and how efficiently the learner uses that information. Reinforcement learning's standard scalar reward carries far less information than the directional task error that defines the task. Random exploration further discards whatever information each rollout returns. Through residual learning with directional task-error supervision and a task error model that drives sample selection, we achieve stable three-, four-, and five-ball juggling on anthropomorphic Barrett WAM arms. Despite planning and controlling through a simple, idealized stack, the system converges from the second attempt. The first attempt drops, after which task error decreases monotonically without further failures. In comparison, five-ball juggling typically takes humans years of practice. We compare residual learners across two ternary axes, the directional information in the learning feedback and the commitment of the analytic prior, spanning Newton-style Jacobian updates, Composite Bayesian Optimization, and stochastic search methods. Both axes prove necessary: neither directional feedback nor an informative prior suffices alone, and the simplest method that combines them, a fixed-Jacobian Newton update, is the most reliable. The learned residual tolerates substantial prior misalignment and degraded joint tracking, affecting mainly convergence speed. The bottleneck for residual learning on real robots is therefore the information content of the supervision signal and how the learner uses it, not the accuracy of the surrounding stack. Video documentation of all experiments is available at https://kai-ploeger.com/residual-juggling.

Kai Ploeger, Jan Peters

Technical University of Darmstadt, Germany · German Research Center for AI (DFKI), Germany · Hessian Center for Artificial Intelligence (hessian.AI), Germany

Robotic Control Robot Skill Learning Robotic RL Robotic Manipulation Data-Efficient Robot Learning Residual Policy Learning

Ask PDF

Jul 16, 2026

Catch, Throw, Repeat: Planning for Human-Robot Partner Juggling

Dynamic object exchange between humans and robots remains a challenging problem due to uncertainty in perception, timing, and contact-rich interaction. Human-robot juggling represents a particularly demanding instance of this problem, requiring precise real-time coordination, predictive motion planning with feedback control, and robustness to variability in human motion. Enabling such skills is of interest for advancing physical human-robot interaction and shared autonomy. We present a real-time planning and control architecture for human-robot partner juggling that enables a robot to reliably catch and throw balls in synchronized multi-ball patterns with a human partner. The system integrates predictive ball tracking, adaptive online trajectory optimization using a multiple-shooting formulation, and a state-machine-based coordination logic to enable synchronized multi-ball human-robot partner juggling. In a user study with 8 participants of varying juggling skill from beginner to expert, we demonstrate that our system can achieve three-ball cascades shared between the robot and the human. All participants exceeded previously reported best-case results within a 10-minute test session, with one participant extending the previous record for shared three-ball cascade juggling fivefold to 20 consecutive robot catches, and another participant achieving a 100% success rate with 40 consecutive catches in a single-ball catch-and-return setting. Video documentation can be found at https://kai-ploeger.com/partner-juggling

Jonathan Rainer Lippert, Kai Ploeger, Abir Chowdhury +3

Technical University of Darmstadt · Justus-Liebig University Gießen · German Research Center for AI (DFKI), 4Robotics Institute Germany +3

Dynamic Robotic Manipulation Human-Robot Interaction Robot Motion Planning

Ask PDF

Jun 4, 2026

Optimal Control Approach for Non-prehensile Ball Juggling Using a 7-DoF Manipulator

Non-prehensile object manipulation skills are important for real-world robot interactions, enabling highly dynamic tasks such as balancing a glass on a tray or the controlled sliding of items on a table. Among such tasks, those characterised by high-speed manipulation requirements and general sensitivity of the resulting hybrid dynamics are particularly hard to accomplish. Within these, juggling can be seen as a highly challenging maneuver to be solved. The key to robotic juggling is achieving dynamic stabilisation of an underactuated object. Since the object does not possess the ability of self-correction, its stability is entirely dependent on the forces applied to it. This creates a system that is sensitive to control inputs, where timing is critical to continuously counteract deviations and maintain the desired behavior. We develop a systematic method to control a 7-degree-of-freedom manipulator performing non-prehensile ball juggling with a tool. Our primary contribution is a model-based framework for generating juggling trajectories and stabilizing a periodic juggling motion for this hybrid system. The framework incorporates a two-stage optimal control approach to compute the underlying feasible motion patterns required for stable juggling. Offline-computed trajectories are then organised to enable real-time error correction without solving optimal control problems online. We demonstrate the effectiveness of the resulting controller by first evaluating its performance in a simulation environment and performing an experiment using a Franka Emika Panda robot.

Joel Ramadani, Vasilije Rakčević, Riddhiman Laha +4

Technical University of Munich · Technical University of Munich, Munich, Germany. · DLR Oberpfaffenhofen +3

Contact-Rich Manipulation Non-Prehensile Manipulation Robot Motion Generation Robotic Manipulation Optimal Control

Ask PDF

Abstract

Dynamic Robotic Manipulation

Explore similar work

Task-Error Residual Learning for Real-Robot Five-Ball Juggling

Catch, Throw, Repeat: Planning for Human-Robot Partner Juggling

Optimal Control Approach for Non-prehensile Ball Juggling Using a 7-DoF Manipulator