Quasi-Periodic Gaussian Process Predictive Iterative Learning Control
Authors: Unnati Nigam, Radhendushka Srivastava, Faezeh Marzbanrad, Michael Burke
Organizations: IITB-Monash Research Academy, IIT Bombay, Mumbai, India · Department of Mathematics, IIT Bombay, Mumbai, India · Department of Electrical and Computer Systems Engineering, Monash University, Clayton, Melbourne, Australia
Repetitive motion tasks are common in robotics, but performance can degrade over time due to environmental changes and robot wear and tear. Iterative learning control (ILC) improves performance by using information from previous iterations to compensate for expected errors in future iterations. This work incorporates the use of Quasi-Periodic Gaussian Processes (QPGPs) into a predictive ILC framework to model and forecast disturbances and drift across iterations. Using a recent structural equation formulation of QPGPs, the proposed approach enables efficient inference with complexity O(p3) instead of O(i2p3), where p denotes the number of points within an iteration and i represents the total number of iterations, specially for larger i. This formulation also enables parameter estimation without loss of information, making continual GP learning computationally feasible within the control loop. By predicting next-iteration error profiles rather than relying only on past errors, the controller achieves faster convergence and maintains this under time-varying disturbances. We benchmark the method against both standard ILC and conventional Gaussian Process (GP)-based predictive ILC on three tasks, autonomous vehicle trajectory tracking, a three-link robotic manipulator, and a real-world Stretch robot experiment. Across all cases, the proposed approach converges faster and remains robust under injected and natural disturbances while reducing computational cost. This highlights its practicality across a range of repetitive dynamical systems.
Figures & tables
Fig. 1: Our key insight is that errors in iterative learning control evolve in a quasi-periodic manner. Incorporating predictive modeling exploiting this structure this into ILC (QPGP-PILC) results in noticeably reduced error magnitude and improved convergence behaviour over standard ILC.
Approach
Standard
QPGP
QPGP
Sparse
GP
(block)
(element)
GP
Time (secs.)
0.23
18.74
19.31
19.67
649.23
TABLE I: Total Computation time (in secs.) for 100 iterations for Standard ILC, GP-PILC, Sparse GP-PILC and QPGP-PILC (block and element-wise).
Fig. 2: Comparison of error convergence for vehicle trajectory tracking for Standard ILC, GP-PILC, Sparse GP-PILC, and QPGP-PILC (block and element-wise). Element-wise QPGP-PILC converges fastest, followed by block-based QPGP-PILC. GP-PILC shows similar convergence, while Sparse GP-PILC converges slightly slower. QPGP-PILC consistently achieves the most accurate tracking.
Fig. 3: Comparison of vehicle trajectories at the 50th iteration under Standard ILC and QPGP-PILC controllers. The QPGP-PILC trajectory closely follows the reference, whereas the Standard ILC trajectory shows visible deviations.
Fig. 4: Comparison of error convergence for manipulator between Standard ILC, QPGP-PILC, and GP-PILC. QPGP-PILC achieves the fastest convergence, followed by GP-PILC and then Standard ILC. In addition to its superior convergence rate, QPGP-PILC is computationally more efficient, as it requires only the most recent iteration’s errors for predictions.
Fig. 5: Response of the manipulator to a mid-iteration disturbance under Standard ILC and QPGP-PILC. Standard ILC reacts slowly, while QPGP-PILC anticipates and corrects deviations, recovering to pre-disturbance error levels much faster.
Fig. 6: Convergence of tracking error for Hello Robot Stretch 3 over iterations for standard ILC and QPGP-based PILC. The QPGP-PILC achieves faster convergence than Standard ILC.
Fig. 7: Tracking performance of Hello Robot Stretch 3 at 25th iteration under Standard ILC and QPGP-PILC controllers. The QPGP trajector is closer to the reference path compared to the Standard ILC trajectory, which exhibits noticeable deviations.
Robots executing iterative tasks in complex, uncertain environments require control strategies that balance robustness, safety, and high performance. This paper introduces a safe information-theoretic learning model predictive control (SIT-LMPC) algorithm for iterative tasks. Specifically, we design an iterative control framework based on an information-theoretic model predictive control algorithm to address a constrained infinite-horizon optimal control problem for discrete-time nonlinear stochastic systems. An adaptive penalty method is developed to ensure safety while balancing optimality. Trajectories from previous iterations are utilized to learn a value function using normalizing flows, which enables richer uncertainty modeling compared to Gaussian priors. SIT-LMPC is designed for highly parallel execution on graphics processing units, allowing efficient real-time optimization. Benchmark simulations and hardware experiments demonstrate that SIT-LMPC iteratively improves system performance while robustly satisfying system constraints.
Zirui Zang, Ahmad Amine, Nick-Marios T. Kokolakis +3
University of Pennsylvania, Philadelphia, PA 19104, USA · Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA · Lyric, Italia 152 Catania Italy
In this paper, we propose Lagrangian Gaussian Processes (LGPs) for probabilistic and data-efficient learning of dynamics via discrete forced Euler-Lagrange equations. Importantly, the geometric structure of the Lagrange-d'Alembert principle, which governs the motion of dynamical systems, is preserved by construction in the absence of external forces. This allows learning physically consistent models that overcome erroneous drift in the system's energy, thereby providing stable long-term predictions. At the core of our approach lie linear operators for Gaussian process conditioning, constructed from discrete forced Euler-Lagrange equations and variational discretization schemes. Thereby and unlike prior work, the method enables learning dynamics from discrete position snapshots, i.e., without access to a system's velocities or momenta. This is particularly relevant for a large class of practical scenarios where only position measurements are available, for instance, in motion capture or visual servoing applications. We demonstrate the data-efficiency and generalization capabilities of the LGPs in various synthetic and real-world case studies, including a real-world soft robot with hysteresis. The experimental results underscore that the LGPs learn physically consistent dynamics with uncertainty quantification solely from sparse positional data and enable stable long-term predictions.
A significant limitation of Deep Reinforcement Learning (DRL) is the stochastic uncertainty in actions generated during exploration-exploitation, which poses substantial safety risks during both training and deployment. In industrial process control, the lack of formal stability and convergence guarantees further inhibits adoption of DRL methods by practitioners. Conversely, Iterative Learning Control (ILC) represents a well-established autonomous control methodology for repetitive systems, particularly in batch process optimization. ILC achieves desired control performance through iterative refinement of control laws, either between consecutive batches or within individual batches, to compensate for both repetitive and non-repetitive disturbances. This study introduces an Iterative Learning Control-Informed Reinforcement Learning (IL-CIRL) framework for training DRL controllers in dual-layer batch-to-batch and within-batch control architectures for batch processes. The proposed method incorporates Kalman filter-based state estimation within the iterative learning structure to guide DRL agents toward control policies that satisfy operational constraints and ensure stability guarantees. This approach enables the systematic design of DRL controllers for batch processes operating under multiple disturbance conditions.
Runze Lin, Ziqi Zhuo, Junghui Chen +2
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China · Department of Chemical Engineering, Chung-Yuan Christian University, Taoyuan 32023, Taiwan, R.O.C.