cs.ROJun 3, 2025

Learned Controllers for Agile Quadrotors in Pursuit-Evasion Games

Authors: Alejandro Sanchez Roncero, Yixi Cai, Olov Andersson, Petter Ogren

Organizations: Robotics, Perception and Learning Lab., School of Electrical Engineering and Computer Science, Royal Institute of Technology (KTH), SE-100 44 Stockholm, Sweden.

Abstract

We study one-versus-one (1v1) quadrotor pursuit-evasion, where a pursuer and an evader are trained with reinforcement learning (RL). Direct adversarial RL tends to be unstable because opponent policies change during training, causing non-stationarity, forgetting, and strategy cycling. We propose Asynchronous Multi-Stage Population-Based training, inspired by Policy-Space Response Oracles (PSRO). Proximal Policy Optimization (PPO) trains responses against previously learned opponents held fixed during each stage. The uniform variant, AMSPB-U, samples opponents equally, while AMSPB-P combines hard prioritized fictitious self-play (PFSP) with uniform exploration to emphasize currently difficult opponents. In cross-evaluations against learned opponents, both variants improve pursuit-evasion balance over latest-opponent training. AMSPB-P scores highest while retaining competence against earlier strategies. Across varied starting positions and after larger-arena retraining, both population variants retain higher average success across the two roles than latest-opponent training. Hardware experiments with two 10 cm, approximately 40 g Crazyflie brushless quadrotors demonstrate zero-shot sim-to-real transfer against baseline controllers and learned adversaries across 42 flights. Recorded flights include captures of evaders reaching robust peak speeds of 3.31 m/s and accelerations of 17.95 m/s^2, with pursuer flight speeds up to 4.87 m/s.

Explore similar work

CardsList
  1. Intercepting an Agile Target with Net-Carrying Drones using Competitive Multi-Agent Reinforcement Learning

    Jul 7, 2026Timothée Gavin, Murat BronzRobotic RLCooperative MARL

  2. AgilePE: Autonomous UAV Pursuit-Evasion via Self-Play Reinforcement Learning

    Aug 14, 2026Wenhao Tang, Tianyang Chen, Zhejun Cui +9Robot Policy LearningSelf-Play RL

  3. Curriculum-Based Adversarial Heterogeneous Agent Reinforcement Learning for Autonomous Quad-Copter Landing in Maritime Settings

    Sep 11, 2026Allan Minh-Tam Nguyen, Sree Showrya Kotala, Stefan Banioi-Crijman +2Adversarial TrainingAerial Robotics