Recent work in reinforcement learning has shown that incorporating structural priors for articulated robots, such as link connectivity, into policy networks improves learning efficiency. However, dynamics properties, despite their fundamental role in determining how forces and motion propagate through the body, remain largely underexplored as an inductive bias for policy learning. To address this gap, we present the Articulated-Body Dynamics Network (ABD-Net), a novel graph neural network architecture grounded in the computational structure of forward dynamics. Specifically, we adapt the inertia propagation mechanism from the Articulated Body Algorithm, systematically aggregating inertial quantities from child to parent links in a tree-structured manner, while replacing physical quantities with learnable parameters. Embedding ABD-NET into the policy actor enables dynamics-informed representations that capture how actions propagate through the body, leading to efficient and robust policy learning. Through experiments with simulated humanoid, quadruped, and hopper robots, our approach demonstrates increased sample efficiency and generalization to dynamics shifts compared to transformer-based and GNN baselines. We further validate the learned policy on real Unitree G1 and Go2 robots, state-of-the-art humanoid and quadruped platforms, generating dynamic, versatile and robust locomotion behaviors through sim-to-real transfer with real-time inference.
Figures & tables
Fig. 1: Different approaches to computing node features for articulated robots. Conventional methods use link connectivity to define information flow via adjacency or attention mask, leaving the network to learn how to form node features from scratch. Our approach encodes the computational structure of forward dynamics by performing dynamics-inspired message passing, where learnable inertia-related quantities (denoted as Ia ) are propagated and aggregated from children to parents to form node features.
Fig. 2: Overview of ABD-Net on a quadruped robot. (a) Observation Encoding: Each link i has its own projection ϕi that transforms the observation s into a link-wise observation embedding zi (e.g., FR-Thigh, FR-Calf, FR-Foot for the front-right leg). (b) Dynamics-Informed Message Passing: Each link i is associated with learnable parameters (Wi,Bi) . Link j computes a contribution vja using Wj and sends it to its parent, which aggregates contributions from all children to form its link representation vi ( σ denotes softplus). (c) Action Decoding: For joint j connecting \textscpa(j) and link j , the action aj is computed from the parent’s representation v\textscpa(j) .
Fig. 3: Learning curves comparing ABD-Net (ours) and baselines: mean return versus the number of environment steps. All methods are trained with 5 seeds. Shaded regions indicate 95% standard confidence intervals.
Category
Reward Term
Go1
Go2
G1
T1
Vel. Track.
Lin. vel. tracking
1.5
1.0
1.0
1.0
Ang. vel. tracking
0.75
0.2
0.5
0.5
Base Reg.
Base height
12.5
50.0
10.0
20.0
Vertical lin. vel. ( ↓ )
2.0
1.0
2.0
1.0
Ang. vel. xy ( ↓ )
0.025
–
0.05
0.025
Joint Reg.
Action rate ( ↓ )
0.02
0.005
0.01
0.1
TABLE I: Reward functions and weights for Genesis environments. ( ↓ ) denotes penalty terms.
Genesis (T1 + G1 + Go1 + Go2)
Method
IQM
Median
Mean
ABD-Net
0.85 (0.73, 0.95)
0.86 (0.70, 0.97)
0.83 (0.75, 0.91)
BoT
0.69 (0.56, 0.85)
0.67 (0.54, 0.93)
0.70 (0.58, 0.82)
SWAT
0.79 (0.50, 0.94)
0.83 (0.39, 0.95)
0.75 (0.57, 0.89)
Rodrigues
0.45 (0.17, 0.71)
0.45 (0.04, 0.81)
0.45 (0.25, 0.65)
GNN
0.76 (0.65, 0.85)
0.76 (0.65, 0.87)
0.74 (0.67, 0.82)
TABLE II: Normalized final performance on Genesis and SAPIEN.
Algorithm
Humanoid
Hopper
Go2
T1
ABD-Net
91.1 ± 1.4
62.4 ± 0.5
82.4 ± 2.1
81.1 ± 3.5
BoT
17.0 ± 0.6
N/C
69.7 ± 2.3
57.0 ± 2.1
SWAT
88.9 ± 2.0
29.7 ± 1.0
81.4 ± 2.2
53.4 ± 1.6
Rodrigues
28.0 ± 1.7
9.1 ± 0.4
53.7 ± 3.1
N/C
GNN
54.9 ± 0.8
N/C
82.9 ± 2.2
69.9 ± 2.4
MLP
78.4 ± 2.9
55.4 ± 2.1
66.5 ± 2.7
N/C
TABLE III: Mass generalization performance (Retention, %). N/C indicates that the training policy did not converge.
Fig. 4: Recovery behavior under 2 × mass on Hopper Hop. Top: ABD-Net applies stronger torque to recover from the downward tilt. Bottom: SWAT fails to compensate and falls.
Fig. 5: Learned link representations on Go2 during trot gait. Left: Feature norm time series of hip joints (FL: front-left, RR: rear-right, etc.). Right: Correlation rank matrix between hip joint features, where lower rank indicates higher correlation.
Fig. 6: Comparison of ABD-Net and MLP as dynamics models. Left: Single-step validation loss on Double Pendulum and Hopper. Right: 1, 3, 5-step rollout prediction error.
Fig. 7: Learning curves comparing ABD-Net , ABD-Net w/o orth., and GNN across four tasks
Algorithm
Humanoid
Hopper
Go2
T1
ABD-Net
91.1 ± 1.4
62.4 ± 0.5
96.2 ± 0.6
81.1 ± 3.5
w/o orth.
59.8 ± 0.4
56.3 ± 0.8
77.4 ± 2.2
51.6 ± 2.2
GNN
54.9 ± 0.8
N/C
82.9 ± 2.2
69.9 ± 2.4
TABLE IV: Mass generalization performance
Fig. 8: Sim-to-real transfer on Unitree Go2 and G1 robots. Top two rows: Go2 velocity tracking across diverse terrains (packed dirt, asphalt, grass, indoor tile), with composite motion sequences showing lateral walking on grass and yaw tracking on indoor tile. Bottom: G1 velocity tracking with composite motion sequences showing forward and lateral walking (left) and motion tracking of a dance sequence (right).
Fig. 9: Effect of observation history length on G1. ABD-Net and MLP are compared across history lengths ∈{1,3,8,16} .
Fig. 10: Left: FLOPs per forward pass. Right: Inference time. All measured on an RTX 4090 with batch size 2048.
Reinforcement learning provides a framework for learning control policies that can reproduce diverse motions for simulated characters. However, such policies often exploit unnatural high-frequency signals that are unachievable by humans or physical robots, making them poor representations of real-world behaviors. Existing work addresses this issue by adding a reward term that penalizes a large change in actions over time. This term often requires substantial tuning efforts. We propose to use the action Jacobian penalty, which penalizes changes in action with respect to the changes in simulated state directly through auto differentiation. This effectively eliminates unrealistic high-frequency control signals without task specific tuning. While effective, the action Jacobian penalty introduces significant computational overhead when used with traditional fully connected neural network architectures. To mitigate this, we introduce a new architecture called a Linear Policy Net (LPN) that significantly reduces the computational burden for calculating the action Jacobian penalty during training. In addition, a LPN requires no parameter tuning, exhibits faster learning convergence compared to baseline methods, and can be more efficiently queried during inference time compared to a fully connected neural network. We demonstrate that a Linear Policy Net, combined with the action Jacobian penalty, is able to learn policies that generate smooth signals while solving a number of motion imitation tasks with different characteristics, including dynamic motions such as a backflip and various challenging parkour skills. Finally, we apply this approach to create policies for dynamic motions on a physical quadrupedal robot equipped with an arm.
Zhaoming Xie, Kevin Karol, Jessica Hodgins
Robotics AI Institute, Cambridge, Massachusetts, USA
Quadrupedal locomotion plays a critical role in enabling agile, versatile movement across complex terrains. Understanding and estimating the underlying physical dynamics are essential for achieving efficient and stable quadrupedal locomotion. We propose a novel training framework for quadrupedal locomotion that enables the Control Policy to understand and reason about physical dynamics. In simulation, we concurrently train an Intrinsic Dynamics (ID) Head that learns state-to-torque dynamics alongside the Control Policy, and we define a dynamics reward enabled by the ID Head that encourages the Policy toward more predictable dynamical behavior. We also provide a mechanism to tune the learned dynamics in the resulting Policy by controlling the training coefficients of the ID Head. Our simulation experiments show that this mechanism drives convergence to better optima across a wide range of standard quadrupedal locomotion rewards, yielding more efficient and smoother policies. Our real-robot experiments demonstrate sim-to-real transfer of these improvements, with significant gains in torque efficiency (16.8%), action rate (18.6%), and mechanical power (12.8%), while improving safe torque occupancy by 6.4%.
Aman Arora, Nalini Ratha
Department of Computer Science, University at Buffalo, The State University of New York, Buffalo, USA
State of the art quadrupedal locomotion approaches integrate Model Predictive Control (MPC) with Reinforcement Learning (RL), enabling complex motion capabilities with planning and terrain adaptive behaviors. However, they often face compounding errors over long horizons and have limited interpretability due to the absence of physical inductive biases. We address these issues by integrating Lagrangian Neural Networks (LNNs) into an RL MPC framework, enabling physically consistent dynamics learning. At deployment, our inverse dynamics infinite horizon MPC scheme avoids costly matrix inversions, improving computational efficiency by up to 4x with minimal loss of task performance. We validate our framework through multiple ablations of the proposed LNN and its variants. We show improved sample efficiency, reduced long-horizon error, and faster real time planning compared to unstructured neural dynamics. Lastly, we also test our framework on the Unitree Go1 robot to show real world viability.
the Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science, Bengaluru. · the Robert Bosch Center for Cyber-Physical Systems and the Department of Computer Science & Automation, Indian Institute of Science, Bengaluru.