Differentiable simulation provides analytic gradients of robot dynamics, enabling fast and sample-efficient first-order policy optimization. However, obtaining smooth and informative gradients through rigid-body contact typically requires softened contact models, often at the expense of physical fidelity and thereby limiting learned policies largely to simulation. This trade-off becomes particularly consequential for dynamic humanoid motions, where accurate contact dynamics are critical for transferring policies to the real world. Increasing contact stiffness in rigid-body simulation improves the fidelity of interactions, but also makes the dynamics increasingly sensitive to small state perturbations, producing high-variance gradients that can destabilize first-order policy learning. To address this, we propose Bundled Contact Gradients (BCG), a contact-local randomized smoothing framework for differentiable policy learning. When stiff contact is detected, our method evaluates a local bundle of randomized perturbation rollouts around the stiff contact configuration and aggregates their gradient signal thereby reducing gradient variance. We demonstrate the effectiveness of our method by successfully training and transferring dynamic motions zero-shot onto a real-world Unitree G1 humanoid platform. Videos and supplementary information can be found at https://bundledcontactgradients.github.io/
Figures & tables
Fig. 1: Bundled Contact Gradients. (a) Dynamic, contact-rich Unitree G1 motions learned in differentiable simulation with stiff, deployable contact. (b) At stiff contact (time t ), B perturbations of the contacting foot’s position and velocity are sampled from a local sphere of radius σ and simulated in parallel for H differentiable steps, then aggregated into the bundled state (blue). In the right-to-left backward pass, per-step sample gradients agree out of contact but diverge sharply at stiff contact (red). Bundling concentrates them around a common direction (blue), stabilizing first-order policy optimization at deployable contact stiffness. (c) Zero-shot transfer of the learned policies to hardware.
Fig. 2: Policy learning with BCG in a SHAC-style framework and ADD. The policy drives parallel rollouts. At stiff contact, B perturbed branches advance for H steps under shared policy actions and are averaged to continue the rollout. ADD compares simulated and reference motion features to provide imitation rewards, whose gradients propagate through the dynamics to update the policy.
Run
Jump
Fight
Dance
κ
Falls ↓
Δd↓
Falls ↓
Δd↓
Falls ↓
Δd↓
Falls ↓
Δd↓
50
5/5
10.6
5/5
12.6
5/5
13.3
5/5
13.5
100
5/5
8.1
4/5
10.8
3/5
9.1
4/5
9.5
300
0/5
3.1
0/5
3.3
0/5
3.4
0/5
2.9
Table 1: Contact-stiffness ablation with BCG (Q1). All policies are trained with SHAC+BCG at the indicated stiffness κ and evaluated over 5 runs after transfer to MuJoCo. Falls: number of runs in which root height drops below 0.3 m, reported as n/5 . Δd : difference in mean foot penetration depth [mm] between the training simulator and MuJoCo; positive values indicate deeper penetration during training.
Fig. 3: Policy-gradient variance during training (Q2). Shading shows ± one standard deviation across 4,000 resamples of the 128 training environments, showing how much the estimated variance changes when the recorded gradients are resampled.
Fig. 5: Learning curves on the four reference motions (Rows) plotted against three budgets: training iterations (left), environment samples (middle), and wall-clock time (right). The sample axis is logarithmic
Motion
SHAC-Stiff + BCG [cm]
PPO [cm]
Run
34.1 ± 0.4
42.8 ± 0.7
Jump
18.6 ± 0.4
18.0 ± 0.2
Fight
13.5 ± 0.3
34.3 ± 1.4
Dance
18.5 ± 0.8
19.0 ± 0.1
Table 2: Tracking accuracy after sim-to-sim transfer (Q4). Global mean per-body position error Etrack [cm], reported as mean ± std over 5 runs.
Accurate physics simulation is essential for robotic learning and control, yet analytical simulators often fail to capture complex contact dynamics, while learning-based simulators typically require large amounts of costly real-world data. To bridge this gap, we propose a few-shot real-to-sim approach that combines the physical consistency of analytical formulations with the representational capacity of graph neural network (GNN)-based models. Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions. On this foundation, we introduce a mesh-based GNN that implicitly models rigid-body forward dynamics and derive surrogate gradients for collision detection, achieving full differentiability. Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories. In addition, the differentiable design supports gradient-based optimization, which we validate through simulation-based policy learning in multi-object interaction scenarios. Extensive experiments show that our framework not only improves simulation fidelity with minimal supervision but also increases the efficiency of policy learning. Taken together, these findings suggest that differentiable simulation with few-shot real-world grounding provides a powerful direction for advancing future robotic manipulation and control.
Zhenhao Huang, Siyuan Luo, Bingyang Zhou +3
National University of Singapore, Singapore · Prana Lab, San Francisco, USA
Three properties determine whether a differentiable simulator can drive gradient-based optimization through contact: simulation accuracy, gradient reliability, and per-iteration cost. Tape-based engines such as MJX and Newton Semi-Implicit require timesteps small enough to keep contacts numerically tractable, and their backpropagation memory grows linearly with the number of timesteps T. Surrogate models bound memory by approximating contact away, but the resulting gradients lose the geometry the optimization depends on. We present Ostrich, a GPU-accelerated rigid-body simulator that resolves hard contacts and friction with non-smooth Newton iteration at large timesteps (h ~ 0.1 s), and differentiates the converged residual via the implicit function theorem, reusing the forward Schur complement to compute the adjoint at O(1) memory per timestep. On real-robot trajectories over a pallet obstacle, Ostrich holds MuJoCo's sim-to-real accuracy up to a 50x larger timestep. Its gradients converge from random initializations where MJX descends slowly and Newton Semi-Implicit stalls; a warm iteration runs 211x faster than MJX's and 4.7x faster than Semi-Implicit's. On the same scene Ostrich differentiates 8,192 parallel worlds on a single 24 GB GPU, sustaining 29x checkpointed MJX's optimization throughput; without checkpointing both baselines exhaust memory at far fewer worlds. We close with a gradient-based trajectory optimization demonstration over triangle-mesh terrain across a 10 s horizon, a setting where prior engines either restrict to primitive geometry or face the convergence and memory limits shown above.
Aleš Kučera, Karel Zimmermann
Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic
Generating intelligent robot behavior in contact-rich settings is a research problem where zeroth-order methods currently prevail. Developing methods that make use of first/second order information about rigid-body dynamics in the presence of contact holds great promise in terms of increasing the solution speed and computational efficiency. The main bottleneck in this research direction is the difficulty in obtaining gradients and Hessians that are actually useful for numerical optimization, due to pathologies in all three steps of a common simulation pipeline: i) collision detection, ii) contact dynamics, iii) time integration. This abstract proposes a method that aims to address the collision detection part of the puzzle, via a novel pipeline designed from scratch with smooth (i.e. twice) differentiability and massive vectorizability on GPUs as the main priorities. This is in contrast to standard collision detection routines that are instead optimized for runtime on CPUs and minimal memory footprint, but do employ logic and control flow that hinder differentiability and vectorization. The proposed pipeline consists of the following contributions: i) highly expressive and compute efficient SDF representations, ii) differentiable broad-phase and narrow-phase routines that use these representations to generate vertex-SDF and edge-SDF contacts, iii) a differentiable routine for convex decomposition based contact blending.