Distributed 3D Leader-Follower Formation Control with Field-of-View Safety via Control Barrier Functions

Cooperation in multi-UAV systems requires reliable relative perception so that follower vehicles can maintain formation and continue their mission safely even when absolute positioning sensors degrade or fail. This paper presents a vision-based cooperative formation framework running on a follower UAV that uses a front-facing RGB-D camera to detect, track, and localize a leader UAV in real-time. A lightweight YOLO-based detector is trained on a dedicated drone dataset and deployed onboard to predict leader bounding boxes, which are then fused with depth information via a pinhole camera model to estimate the leader's relative pose. These estimates provide a leader-follower position controller and can also be used as a backup when GPS or external localization is unavailable. This framework is implemented as a set of ROS nodes and evaluated in a physics-based multi-UAV simulation built on XTDrone, with sensor noise and communication dropouts. We evaluate detection accuracy, runtime, and formation-keeping error under nominal conditions and under simulated failures of the positioning sensors. The results show that the proposed framework maintains stable leader-follower formations with reasonable computational cost and provides a practical basis for extending vision-based cooperative formation control to real-world multi-UAV systems.

Classical leader-follower formation control suffers from single points of failure and error propagation, and relies on absolute localization sensors that are ill-suited for GPS-denied environments. We present a learned, vision-only estimator that maps each robot's monocular image, together with messages exchanged over a communication graph, directly to its 6-DoF relative pose. Its key ingredient is the implicit virtual leader (IVL): a non-physical reference frame at the team centroid, implicitly learned inside a Transformer-based graph neural network, so that estimation has no privileged node and needs no absolute localization. The estimator additionally reports well-calibrated aleatoric (heteroscedastic GNLL) uncertainty alongside epistemic (MC~Dropout) uncertainty, compared systematically across simulation and real-world test sets. Trained only in simulation, the estimator generalizes to unseen scenes, to larger unseen team sizes, and to an external real-world benchmark. It exhibits no single point of failure: removing any one robot costs at most $1.24\times$ the median removal, and removing $71\%$ of the communication links costs $1.77\times$ in position error without retraining. Trained on real-robot data from a single platform, it transfers without modification to a heterogeneous team, estimating relative pose to $0.22$,m and $1.6^\circ$ on physical robots, where it drives closed-loop formation control.

Active perception in uncertain environments requires robots to navigate safely while acquiring informative observations to reduce map uncertainty. These objectives inherently conflict, as informative viewpoints often lie near uncertain regions with higher collision risk. To address this challenge, we develop a conflict-aware active perception and control framework for robotic systems operating in environments represented by 3D Gaussian Splatting (3DGS). Safety is enforced using a Control Barrier Function (CBF) derived from an Average Value-at-Risk AV@R collision-risk metric that accounts for geometric uncertainty and guarantees forward invariance of a safe set. To improve perception, we propose a risk-aware Expected Information Gain (EIG) formulation for selecting the next-best-view and introduce perception barrier functions that align the camera orientation with the local information-ascent direction. To obtain a tractable formulation for these conflicting safety and perception objectives, we propose a unified safety-critical, perception-aware quadratic program that enforces safety as a hard constraint while relaxing perception constraints through slack variables. Simulation results demonstrate that the proposed method improves both safety and information acquisition compared to existing 3DGS-based approaches.