Data-Driven Physics Embedded Dynamics with Predictive Control and Reinforcement Learning for Quadrupeds

Prakrut Kotecha; Aditya Shirwatkar; Shishir Kolathaya

cs.ROMar 15, 2026

Data-Driven Physics Embedded Dynamics with Predictive Control and Reinforcement Learning for Quadrupeds

Authors: Prakrut Kotecha, Aditya Shirwatkar, Shishir Kolathaya

Organizations: the Robert Bosch Center for Cyber-Physical Systems, Indian Institute of Science, Bengaluru. · the Robert Bosch Center for Cyber-Physical Systems and the Department of Computer Science & Automation, Indian Institute of Science, Bengaluru.

Abstract

State of the art quadrupedal locomotion approaches integrate Model Predictive Control (MPC) with Reinforcement Learning (RL), enabling complex motion capabilities with planning and terrain adaptive behaviors. However, they often face compounding errors over long horizons and have limited interpretability due to the absence of physical inductive biases. We address these issues by integrating Lagrangian Neural Networks (LNNs) into an RL MPC framework, enabling physically consistent dynamics learning. At deployment, our inverse dynamics infinite horizon MPC scheme avoids costly matrix inversions, improving computational efficiency by up to 4x with minimal loss of task performance. We validate our framework through multiple ablations of the proposed LNN and its variants. We show improved sample efficiency, reduced long-horizon error, and faster real time planning compared to unstructured neural dynamics. Lastly, we also test our framework on the Unitree Go1 robot to show real world viability.

Figures & tables

Fig. 1 : Overview of our inverse dynamics-based planner. At each control step, the method samples candidate joint trajectories, computes the corresponding torques, evaluates them using learned reward and value functions, and updates the distribution for the next sampling round.

Quadruped Locomotion Methods	Inference Speed (1-5)	Planning	Interpretable Dynamics	Multi-Terrain
Nonlinear MPC [ 1 , 2 , 3 ]	3 to 4	✓	✓	✗
DreamWaQ [ 11 ] , HIMLoco [ 12 ]	5	✗	None	✓
PIP-Loco [ 13 ]	4	✓	✗	✓
Ours	3	✓	✓	✓

TABLE I : Comparison of proprioceptive quadruped locomotion methods, including our approach. The table summarizes key trade-offs across inference speed (higher rank is faster), planning capability, model interpretability, and terrain generalization.

Fig. 2 : Our training framework where observation history is encoded into full-state estimates, and passed through a physics-informed Dreamer module to augment expert actor with future predictions for robust policy learning.

q^{(j)}_{1:H}\sim\mathcal{N}(\mu_{i-1},\sigma_{i-1}),\hskip 8.50012pt\forall j\in[1,M]

Algorithm 2 Inverse Dynamics-Based

\infty

Horizon MPC

Hyperparameters	Value
Training
Hidden layers for RL	$\begin{bmatrix}512,256,128\end{bmatrix}$
Hidden layers for Dreamer	$\begin{bmatrix}256,256\end{bmatrix}$
Activation function	tanh
Number of environments	4096
Number of mini batches	25

TABLE II : Hyperparameters used for training and deployment. Separate configurations are used for the RL and MPC modules to balance performance and efficiency.

Fig. 3 : Training loss curves across different dynamics models. Our method achieves strong sample efficiency while balancing compute cost, compared to LNN Forward and ONN baselines.

Fig. 4 : H-step prediction error of different models. Our approach maintains stable error across horizons, with significantly lower compounding error than ONN and CoM LNN.

Fig. 5 : Trade-off between MPC inference time and control performance across planning horizons. Our inverse-dynamics-based planner achieves competitive returns with significantly lower inference latency.

Method	Flat Ground	Slopes	Rough terrain	Stairs (12 cm)	Stairs (14 cm)	Stairs (16 cm)
HIMLoco	$9.19\pm 0.02$	$9.18\pm 0.05$	$8.72\pm 0.07$	$7.32\pm 0.24$	$7.01\pm 0.38$	$6.31\pm 0.58$
DeLaN (H=9)	$9.38\pm 0.1$	$9.38\pm 0.11$	$9.1\pm 0.11$	$8.33\pm 0.29$	$7.81\pm 0.28$	$7.31\pm 0.31$
CoM LNN (H=5)	$9.05\pm 0.26$	$8.26\pm 0.38$	$7.98\pm 0.62$	$6.83\pm 0.59$	Failed	Failed
ONN (H=9)	$9.19\pm 0.07$	$9.20\pm 0.12$	$8.80\pm 0.21$	$7.48\pm 0.46$	$7.21\pm 0.48$	$6.91\pm 0.78$
Ours (H=9)	$9.23\pm 0.17$	$8.67\pm 0.42$	$8.64\pm 0.42$	$7.74\pm 0.53$	$7.51\pm 0.58$	$7.18\pm 0.53$

TABLE III : Performance comparison across terrains in simulation. Reported values are average episodic rewards ± standard deviation. DeLaN achieves the highest rewards, while our method closely matches with much lower compute cost.

Fig. 6 : Hardware validation and absolute error tracking across 1200 time steps. Our method maintains significantly lower, more consistent rollout error than baselines (ONN, DeLaN, LNN variants) while navigating diverse terrain, including gravel, ramps, and stairs.

Explore similar work

Integrating Lagrangian Neural Networks into the Dyna Framework for Reinforcement Learning
Mar 9, 2026Shreya Das, Kundan Kumar, Muhammad Iqbal +4Model-Based RL
Department of Electrical Engineering and Automation, Aalto University, Finland · VTT Technical Research Centre of Finland · Department of Computer Science, University of Helsinki, Finland +1
Physics-Informed Policy Optimization via Analytic Dynamics Regularization
Mar 15, 2026Namai Chandra, Liu Mohan, Zhihao Gu +1Physics-Informed RL RL Control
Electronic Systems, IIT Madras, India · EmPACT Lab, Nanyang Technological University, Singapore
Dynamics Aware Quadrupedal Locomotion via Intrinsic Dynamics Head
May 2, 2026Aman Arora, Nalini RathaRobot Dynamics Learning RL for Legged Locomotion
Department of Computer Science, University at Buffalo, The State University of New York, Buffalo, USA

Mar 9, 2026J/K move · Enter open · S save

Integrating Lagrangian Neural Networks into the Dyna Framework for Reinforcement Learning

Shreya Das, Kundan Kumar, Muhammad Iqbal, Outi Savolainen+3

Department of Electrical Engineering and Automation, Aalto University, Finland · VTT Technical Research Centre of Finland · Department of Computer Science, University of Helsinki, Finland+1

Model-Based RL

Open paper AskPDF arXiv

Model-based reinforcement learning (MBRL) is sample-efficient but depends on the accuracy of the learned dynamics, which are often modeled using black-box methods that do not adhere to physical laws. Those methods tend to produce inaccurate predictions when presented with data that differ from the original training set. In this work, we employ Lagrangian neural networks (LNNs), which enforce an underlying Lagrangian structure to train the model within a Dyna-based MBRL framework. Furthermore, we train the LNN using stochastic gradient-based and state-estimation-based optimizers to learn the network's weights. The state-estimation-based method converges faster than the stochastic gradient-based method during neural network training. Simulation results are provided to illustrate the effectiveness of the proposed LNN-based Dyna framework for MBRL.

Abstract

Explore similar work

Integrating Lagrangian Neural Networks into the Dyna Framework for Reinforcement Learning

Physics-Informed Policy Optimization via Analytic Dynamics Regularization

Dynamics Aware Quadrupedal Locomotion via Intrinsic Dynamics Head