LARD 2.0: Enhanced Datasets and Benchmarking for Autonomous Landing Systems
Organizations: DGA · IRT Saint Exupéry, Toulouse, France · Thales, Toulouse, France · Airbus, Toulouse, France ·
Abstract
This paper addresses key challenges in the development of autonomous landing systems, focusing on dataset limitations for supervised training of Machine Learning (ML) models for object detection. Our main contributions include: (1) Enhancing dataset diversity, by advocating for the inclusion of new sources such as BingMap aerial images and Flight Simulator, to widen the generation scope of an existing dataset generator used to produce the dataset LARD; (2) Refining the Operational Design Domain (ODD), addressing issues like unrealistic landing scenarios and expanding coverage to multi-runway airports; (3) Benchmarking ML models for autonomous landing systems, introducing a framework for evaluating object detection subtask in a complex multi-instances setting, and providing associated open-source models as a baseline for AI models' performance.
Figures & tables
| Along-track distance (m) | Yaw range (°) | Roll range (°) |
| Input image RGB | shape (H,W,3) | |
| Runway BB | center of the BB | |
| Runway ODD indices (or flag) | indicate if the runway is in the ODD | |
| Predicted BB | is confidence score | |
| List of In ODD runways | ||
| List of out-of-ODD runways | ||
| List of predicted runways | can also use the NMS (Non Maximum Suppression) algorithm |
| Model: archi (Trained on) | Score | Test GT | GT | Detections | mAP | mAP50 | mAP75 |
| : yoloV8 (IN_ODD) | mAP | IN_ODD | 53610 | 80002 | 0.6974 | 0.9671 | 0.8122 |
| mAP | IN_ODD + EXTENDED_ODD | 61788 | 80002 | 0.6498 | 0.9084 | 0.7539 | |
| e-mAP | IN_ODD + EXTENDED_ODD | 58242 | 80002 | 0.6882 | 0.9654 | 0.7999 | |
| : yoloV8 (IN_ODD+ EXTENDED_ODD) | mAP | IN_ODD | 53610 | 100504 | 0.6265 | 0.8703 | 0.7256 |
| mAP | IN_ODD + EXTENDED_ODD | 61788 | 100504 | 0.6939 | 0.9669 | 0.8052 | |
| e-mAP | IN_ODD + EXTENDED_ODD | 61418 | 100504 | 0.6976 | 0.9685 | 0.8072 |
Aerial Robotics
Aerial robotics studies the design, perception, control, and coordination of flying robots for navigation, inspection, transport, and physical interaction.
See all 340 papers