Reinforcement Learning for Optimal Experiment Design in Parameter Identification of Mechatronic Systems

Julian Langschwert; Georg Schaefer; Jakob Rehrl; Stefan Huber; Simon Hirlaender

cs.ROMay 19, 2026

Reinforcement Learning for Optimal Experiment Design in Parameter Identification of Mechatronic Systems

Authors: Julian Langschwert, Georg Schaefer, Jakob Rehrl, Stefan Huber, Simon Hirlaender

Organizations: Josef Ressel Centre for Intelligent and Secure Industrial Automation, Salzburg University of Applied Sciences, Salzburg, Austria · Paris Lodron University of Salzburg, Salzburg, Austria

Abstract

Informative excitation signals are critical for accurate system identification of mechatronic systems, yet classical system identification (SI) approaches require expert knowledge and hand-crafted signal design to respect hardware safety constraints, limiting their generalizability. We propose a reinforcement learning (RL) agent that learns optimal excitation signals for a Quanser Aero 2 testbed while autonomously enforcing safety constraints through reward shaping. Evaluated across 10 independent training seeds, our comprehensive agent achieves competitive estimation accuracy across all three identified parameters, outperforming classical baselines while incurring only 0.75% safety violations.

Explore similar work

Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance
May 11, 2026Adam Haroon, Erick J. Rodríguez-Seda, Cody Fleming +1Safety Constraints Lyapunov Function
Safe Reinforcement Learning using Ideas from Model Predictive Control
Jul 8, 2026Georg Schäfer, Jakob Rehrl, Stefan Huber +1Safety Constraints Iterative Reinforcement Learning
Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Exploration
May 12, 2026Youwei Yu, Jionghao Wang, Zhengming Yu +2Efficient Exploration Bayesian Experimental Design

May 11, 2026 · cs.LGJ/K move · Enter open · S save

Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance

Adam Haroon, Erick J. Rodríguez-Seda, Cody Fleming, Tristan Schuler

Department of Mechanical Engineering, Iowa State University, Ames, IA, USA · Navy Center for Applied Research in Artificial Intelligence (NCARAI), U.S. Naval Research Laboratory, Washington, D.C., USA · Department of Weapons, Robotics, and Control Engineering, United States Naval Academy, Annapolis, MD, USA

Safe reinforcement learning (RL) typically asks

\textit{what}

an agent should do. We ask

\textit{when}

it needs to act, and show that a single policy can jointly learn control inputs and communication-efficient timing decisions under a pointwise Lyapunov safety shield. We focus on stabilization around a known equilibrium, where CARE-based LQR backups, Lyapunov certificates, and classical Lyapunov-STC are well defined, enabling clean comparison against analytical baselines. A run-time assurance (RTA) layer overrides the policy via a one-step-ahead Lyapunov prediction and a precomputed LQR backup, providing a strictly stronger guarantee than constrained MDP methods that enforce safety only in expectation. On an inverted pendulum, cart--pole, and planar quadrotor, the learned policy achieves

1.91\times

1.45\times

, and

3.51\times

higher mean inter-sample interval (MSI) than a Lyapunov-triggered baseline; a fixed LQR controller at the same average rate is unstable on all three plants, showing that adaptive timing, not a lower average rate, makes sparsity safe. A CARE-derived Lyapunov reward transfers across environments without redesign, with a single weight

w_c

controlling the stability--communication tradeoff; ablations confirm the RTA shield is essential, with its removal reducing MSI by

1.27

1.84\times

and degrading state norms. A preference-conditioned extension recovers the full tradeoff frontier from one model at

\tfrac{2}{11}

of training compute, and SAC experiments show the results are algorithm-agnostic across discrete and continuous domains. A 12-state 3D quadrotor case study extends the framework to higher-dimensional systems where classical STC is intractable, and robustness to

\pm30\%

mass variation and disturbances shows graceful degradation, with the RTA absorbing what the learned policy cannot.

Safety Constraints Lyapunov Function Deep Reinforcement Learning Inference-Time Control Runtime Enforcement Quadrotor Flight Assurance

Open paper Ask about it PDF

Abstract

Explore similar work

Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance

Safe Reinforcement Learning using Ideas from Model Predictive Control

Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Exploration