Differentiable simulation provides analytic gradients of robot dynamics, enabling fast and sample-efficient first-order policy optimization. However, obtaining smooth and informative gradients through rigid-body contact typically requires softened contact models, often at the expense of physical fidelity and thereby limiting learned policies largely to simulation. This trade-off becomes particularly consequential for dynamic humanoid motions, where accurate contact dynamics are critical for transferring policies to the real world. Increasing contact stiffness in rigid-body simulation improves the fidelity of interactions, but also makes the dynamics increasingly sensitive to small state perturbations, producing high-variance gradients that can destabilize first-order policy learning. To address this, we propose Bundled Contact Gradients (BCG), a contact-local randomized smoothing framework for differentiable policy learning. When stiff contact is detected, our method evaluates a local bundle of randomized perturbation rollouts around the stiff contact configuration and aggregates their gradient signal thereby reducing gradient variance. We demonstrate the effectiveness of our method by successfully training and transferring dynamic motions zero-shot onto a real-world Unitree G1 humanoid platform. Videos and supplementary information can be found at https://bundledcontactgradients.github.io/
Figures & tables
Fig. 1: Bundled Contact Gradients. (a) Dynamic, contact-rich Unitree G1 motions learned in differentiable simulation with stiff, deployable contact. (b) At stiff contact (time t ), B perturbations of the contacting foot’s position and velocity are sampled from a local sphere of radius σ and simulated in parallel for H differentiable steps, then aggregated into the bundled state (blue). In the right-to-left backward pass, per-step sample gradients agree out of contact but diverge sharply at stiff contact (red). Bundling concentrates them around a common direction (blue), stabilizing first-order policy optimization at deployable contact stiffness. (c) Zero-shot transfer of the learned policies to hardware.
Fig. 2: Policy learning with BCG in a SHAC-style framework and ADD. The policy drives parallel rollouts. At stiff contact, B perturbed branches advance for H steps under shared policy actions and are averaged to continue the rollout. ADD compares simulated and reference motion features to provide imitation rewards, whose gradients propagate through the dynamics to update the policy.
Run
Jump
Fight
Dance
κ
Falls ↓
Δd↓
Falls ↓
Δd↓
Falls ↓
Δd↓
Falls ↓
Δd↓
50
5/5
10.6
5/5
12.6
5/5
13.3
5/5
13.5
100
5/5
8.1
4/5
10.8
3/5
9.1
4/5
9.5
300
0/5
3.1
0/5
3.3
0/5
3.4
0/5
2.9
Table 1: Contact-stiffness ablation with BCG (Q1). All policies are trained with SHAC+BCG at the indicated stiffness κ and evaluated over 5 runs after transfer to MuJoCo. Falls: number of runs in which root height drops below 0.3 m, reported as n/5 . Δd : difference in mean foot penetration depth [mm] between the training simulator and MuJoCo; positive values indicate deeper penetration during training.
Fig. 3: Policy-gradient variance during training (Q2). Shading shows ± one standard deviation across 4,000 resamples of the 128 training environments, showing how much the estimated variance changes when the recorded gradients are resampled.
Fig. 5: Learning curves on the four reference motions (Rows) plotted against three budgets: training iterations (left), environment samples (middle), and wall-clock time (right). The sample axis is logarithmic
Motion
SHAC-Stiff + BCG [cm]
PPO [cm]
Run
34.1 ± 0.4
42.8 ± 0.7
Jump
18.6 ± 0.4
18.0 ± 0.2
Fight
13.5 ± 0.3
34.3 ± 1.4
Dance
18.5 ± 0.8
19.0 ± 0.1
Table 2: Tracking accuracy after sim-to-sim transfer (Q4). Global mean per-body position error Etrack [cm], reported as mean ± std over 5 runs.