Learning-Based Robust Control: Unifying Exploration and Distributional Robustness for Reliable Robotics via Free Energy

Hozefa Jesawada; Giovanni Russo; Abdalla Swikir; Fares Abu-Dakka

cs.ROMar 6, 2026

Learning-Based Robust Control: Unifying Exploration and Distributional Robustness for Reliable Robotics via Free Energy

Authors: Hozefa Jesawada, Giovanni Russo, Abdalla Swikir, Fares Abu-Dakka

Organizations: Mechanical Engineering Program, New York University Abu Dhabi, Abu Dhabi, UAE. · Department of Information and Electrical Engineering & Applied Mathematics, University of Salerno, Fisciano, Italy. · Mohammed bin Zayed University for Artificial Intelligence, Abu Dhabi, UAE.

Abstract

A key challenge towards reliable robotic control is devising computational models that can both learn policies and guarantee robustness when deployed in the field. Inspired by the free energy principle in computational neuroscience, to address these challenges, we propose a model for policy computation that jointly learns environment dynamics and rewards, while ensuring robustness to epistemic uncertainties. Expounding a distributionally robust free energy principle, we propose a modification to the maximum diffusion learning framework. After explicitly characterizing robustness of our policies to epistemic uncertainties in both environment and reward, we validate their effectiveness on continuous-control benchmarks, via both simulations and real-world experiments involving manipulation with a Franka Research~3 arm. Across simulation and zero-shot deployment, our approach narrows the sim-to-real gap, and enables repeatable tabletop manipulation without task-specific fine-tuning.

Figures & tables

Fig. 1 : Modifying Maximum Diffusion RL with Distributionally Robust Free Energy Principle to address epistemic uncertainty, narrowing sim-to-real gaps for reliable robotics.

Panel of Figure 2 — Fig. 2 : overview of the proposed control loop. At each step, the agent collects data into a replay buffer, updates the dynamics and cost models, computes the maximally diffusive kernel $p_{\max}$ , and solves the min–max optimization. The resulting twisted kernel produces an action that is applied to the system, and the cycle repeats.

Fig. 3 : HalfCheetah results. Top: training performance of the proposed algorithm vs. MaxDiff RL on HalfCheetah-v5 (mean $\pm$ 1 STD over $n=20$ runs). Middle: rollout frames from the proposed algorithm showing a stable stride reaching the target. Bottom: rollout frames from MaxDiff RL illustrating an unstable gait leading to failure.

Panel of Figure 4 — Fig. 4 : Franka obstacle-avoidance results with the proposed method. Top: learning curves showing episode return and minimum distance to goal (success threshold 5 cm, mean $\pm$ 1 STD over $n=20$ runs). Bottom: i) a rollout demonstrating collision-free manipulation around the obstacle. ii) trajectory plot with goal orientation highlighted in cyan, magenta, and yellow, illustrating efficient training and task execution.

Fig. 5 : Deployment on the Franka Research 3 for cluttered tabletop pick-and-place. Top: grasp at the first goal and place at the second. Bottom: with an obstacle, the controller lifts to avoid collision and completes the placement.

Metric	$\varrho=0$	$0.5$	$1$	$5$	$100$	$1000$	$2000$
Normalized Avg. Episode Cost ( $\downarrow$ )	$1.87$	$1.01$	$\mathbf{1.00}$	$1.24$	$9.50$	$56.8$	$148$
Standard Deviation	$0.076$	$0.014$	$0.055$	$0.099$	$7.96$	$11.4$	$28.3$
Success Rate ( $\uparrow$ )	$100\%$	$100\%$	$100\%$	$80\%$	$60\%$	$20\%$	$0\%$

TABLE S-1 : Performance across

\varrho

over 5 trials. Episode cost is normalized to the best case (

\varrho{=}1

). A trial succeeds if the robot reaches the goal while staying

\geq 0.07

,m from all obstacle centers throughout the episode.

Explore similar work

Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shift
Apr 1, 2026Shaifalee Saxena, Rafael Fierro, Alexander ScheinkerRobotic Control Reinforcement Learning
Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM 87106, USA · Los Alamos National Lab, Los Alamos, NM 87547, USA
Distributionally Robust Control via Stein Variational Inference for Contact-Rich Manipulation
May 18, 2026Hrishikesh Sathyanarayan, Victor Vantilborgh, Harish Ravichandar +2Contact-Rich Robotic Manipulation Robust Control
Yale University, New Haven, CT, USA · Ghent University, Belgium · Georgia Institute of Technology, Atlanta, GA, USA +1
ManiDreams: An Open-Source Library for Robust Object Manipulation via Uncertainty-aware Task-specific Intuitive Physics
Mar 18, 2026Gaotian Wang, Kejia Ren, Andrew S. Morgan +1Robotic Manipulation Robot Task Planning
Department of Computer Science, Rice University, Houston, TX 77005, USA · Robotics and AI Institute, Cambridge, MA 02142, USA

Apr 1, 2026J/K move · Enter open · S save

Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shift

Shaifalee Saxena, Rafael Fierro, Alexander Scheinker

Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM 87106, USA · Los Alamos National Lab, Los Alamos, NM 87547, USA

Robotic Control Reinforcement Learning Contact-Rich Robotic Manipulation Generalization in Robotic Manipulation Feedback Control Robust RL

Open paper AskPDF arXiv

Reinforcement learning has shown strong performance in robotic manipulation, but learned policies often degrade in performance when test conditions differ from the training distribution. This limitation is especially important for reliable robot planning and control in contact-rich tasks such as pushing and pick-and-place, where changes in goals, contact conditions, or robot dynamics can drive the system out-of-distribution at inference time. In this paper, we investigate a hybrid controller that combines reinforcement learning with bounded extremum seeking (ES) to improve robustness under such conditions. In the proposed approach, deep deterministic policy gradient (DDPG) policies are trained under standard conditions on the robotic pushing and pick-and-place tasks, and are then combined with bounded ES during deployment. The RL policy provides fast manipulation behavior, while bounded ES ensures robustness of the overall controller to time variations when operating conditions depart from those seen during training. The resulting controller is evaluated under several out-of-distribution settings, including time-varying goals and spatially varying friction patches. Crucially, under reasonable assumptions, we prove finite time RL-based convergence of a robotic arm to an object after which bounded ES achieves finite-time convergence to a bounded velocity moving target. We provide analytic formulas for both convergence times, and an ability to tune the convergence times by our choice of gains.

Abstract

Explore similar work

Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shift

Distributionally Robust Control via Stein Variational Inference for Contact-Rich Manipulation

ManiDreams: An Open-Source Library for Robust Object Manipulation via Uncertainty-aware Task-specific Intuitive Physics