cs.ROSep 17, 2026

Design of Adaptive PID Controller Based On Asynchronous Advantage Actor Critic Learning Method for QuadCopter Control

Authors: Ali Jokar, Aria Alasty

Organizations: Sharif AgRoLab School of Mechanical Engineering Sharif University of Technology Tehran, Iran

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Quadcopters offer great utility in many applications, but their nonlinear nature and disturbance sensitivity present great control challenges. Basic PID controllers are generally not sophisticated enough to cope with these complexities. This paper suggests a control system that integrates the Asynchronous Advantage Actor-Critic (A3C) algorithm with a PID controller for quadcopter attitude and trajectory tracking. The A3C controller uses parallel agents to optimize PID parameters dynamically using a neural network. A system identification module for the complementary system makes predictions about system states for optimal control policy. The proposed framework was compared with a standard actor-critic (A2C) model. Simulation results verify that they both track accurately. However, the A3C-based controller converges much more for the loss function, as evidenced by reward figures and loss curves, demonstrating better parameter optimization. This shows that A3C-based approach results in improved performance for the control of quadcopter, effectively integrating reinforcement learning and traditional control to achieve higher adaptability.

Figures & tables

Fig. 1: Schematic of a quadcopter with coordinates
Fig. 2: Self-tuning PID Neural Network Structure
Fig. 3: Actor Neural Network Structure
Fig. 4: Critic Neural Network Structure
 ψ\displaystyle\psi←ψ−ηψgψ,\displaystyle\leftarrow\psi-\eta_{\psi}g_{\psi}, 
 θ\displaystyle\theta←θ+ηθgθ.\displaystyle\leftarrow\theta+\eta_{\theta}g_{\theta}. 
Algorithm 1 A3C (Asynchronous Advantage Actor-Critic) [ 16 ]
Fig. 5: Comparison of tracking Desired Z path for each A2C and A3C Algorithms
Fig. 6: Comparison of tracking Desired X path for each A2C and A3C Algorithms
Fig. 7: Comparison of tracking Desired Y path for each A2C and A3C Algorithms
Fig. 8: Comparison of Roll ( ϕ\phi ) Angle Control Using A2C and A3C Algorithms
Fig. 9: Comparison of pitch ( θ\theta ) Angle Control Using A2C and A3C Algorithms
Fig. 10: Comparison of yaw ( ψ\psi ) Angle Control Using A2C and A3C Algorithms
Fig. 11: Comparison of Received rewards of each agent Using A2C and A3C Algorithms
Fig. 12: Comparison of Loss Function Variations in Each Agent’s Network Using A2C and A3C Algorithms
More in this topic

Quadrotor Control

Quadrotor control studies how to stabilize, guide, and adapt aerial robots under demanding flight conditions and physical constraints.

See all 126 papers

Explore similar work

CardsList

  1. Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics

    Jul 28, 2026Ya-Chia Shen, Woei-Leong ChanRL ControlQuadrotor Control

    Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Aeronautics and Astronautics, National Cheng Kung University

  2. Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning

    May 15, 2026Vishnu Saj, Sushil Vemuri, Dileep Kalathil +1RL ControlSim-to-Real Transfer

    Texas A&M University College Station, TX, USA

  3. A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions

    May 18, 2026Fausto Mauricio Lagos Suarez, Akshit Saradagi, Vidya Sumathy +1RL ControlRobot Trajectory Tracking

    Robotics and AI group, in the Department of Computer Science, Electrical and Space Engineering at Luleåa University of Technology, Sweden