cs.ROOct 29, 2026

Information-Driven Collaborative Autonomy for Heterogeneous UAV/UGV Teams

Authors: Joshua Yuriv Pilipovsky

Organizations: Senior Research Engineer, RTX Technology Research Center (RTRC), East Hartford, CT 06108, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper considers coordinating a heterogeneous team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) traversing a domain containing an unknown number of potentially hazardous targets. The UAVs provide aerial sensing using onboard optical cameras, adapting their trajectories to ground routes and target information. We formulate the aerial search problem using ergodic control, which distributes sensing effort according to a spatial information density to balance exploration and target localization. The formulation incorporates dynamic camera field-of-view footprints and pose-dependent probabilistic detection models. Angles-only measurements are processed through Bayesian inference and marginalization over data-association hypotheses to obtain a Poisson multi-Bernoulli (PMB) approximation of the target posterior, representing both detected and undetected targets. This belief directs aerial sensing toward unexplored regions along the ground routes and viewpoints that improve target localization. Ground planning combines chance-constrained target avoidance, enforced through confidence sets, with a void-probability objective that favors routes unlikely to contain threats. We solve each team's optimization problem in a fully decentralized manner using a nested SCvx*--C-ADMM scheme, with outer sequential convex programming iterations and inner distributed consensus optimization, accounting for nonlinearities, nonseparable objectives, and continuous-time inter-agent separation constraints. A closed-loop scheme updates both fleets' trajectories as information becomes available and permits ground motion when the remaining route risk falls below a prescribed threshold. Numerical case studies examine sensing geometry and target uncertainty, compare decentralized and centralized solvers, and demonstrate full closed-loop collaboration between multiple UAVs and UGVs.

Figures & tables

RFS ModelCardinality DistributionSpatial Distribution
BernoulliN∼Bernoulli⁡(p)\mathbf{N}\sim\operatorname{Bernoulli}(p)x∣N=1∼ρ(x)\mathbf{x}\mid\mathbf{N}=1\sim\rho(x)
Multi-BernoulliN=∑i=1MEi, Ei∼Bernoulli⁡(ri)\mathbf{N}=\displaystyle\sum_{i=1}^{M}\mathbf{E}_{i},\ \mathbf{E}_{i}\sim\operatorname{Bernoulli}(r_{i})xi∣Ei=1∼ρi(x)\mathbf{x}_{i}\mid\mathbf{E}_{i}=1\sim\rho_{i}(x)
PoissonN∼Poisson⁡(Λ)\mathbf{N}\sim\operatorname{Poisson}(\Lambda)x1,…,xN∣N∼i.i.d.ρ(x)=λ(x)Λ\mathbf{x}_{1},\ldots,\mathbf{x}_{N}\mid\mathbf{N}\overset{\mathrm{i.i.d.}}{\sim}\rho(x)=\dfrac{\lambda(x)}{\Lambda}
Table 1: Common random finite set models on a state space X\mathcal{X} .
Figure 1: High-level UAV–UGV collaborative autonomy architecture. UAV measurements update the PMB posterior, which informs risk-aware UGV planning, while the committed UGV routes and posterior components update the UAV search density; the PMB void probability gates UGV execution.
Figure 2: Geometry of quadrotor body-mounted optical sensor field of view, and its intersection with the target plane.
Figure 3: Pose-dependent sensing quality examples: (left) Gaussian spread factor G(p;x)G(p;x) with respect to footprint center; (middle) altitude/attitude trade-off βs(x)\beta_{s}(x) for a 2x2 sweep of representative heights and angles; (right) the combined field χ(p;x)=βs(x)G(p;x)\chi(p;x)=\beta_{s}(x)G(p;x) for a fixed vehicle pose.
Sensor ModelInstantaneous Sensing Measure νx\nu_{x}Spectral Response Ψk(x)\Psi_{k}(x)
Point Massνx=δg(x)\displaystyle\nu_{x}=\delta_{g(x)}Ψk(x)=Fk ⁣(g(x))\displaystyle\Psi_{k}(x)=F_{k}\!\left(g(x)\right)
Hard FoVνx(dξ)=1Ef(x)(ξ)∣Ef(x)∣ dξ\displaystyle\nu_{x}(\mathrm{d}\xi)=\frac{\mathbf{1}_{\mathcal{E}_{f}(x)}(\xi)}{|\mathcal{E}_{f}(x)|}\,\mathrm{d}\xiΨk(x)=1∣Ef(x)∣∫Ef(x)Fk(ξ) dξ\displaystyle\Psi_{k}(x)=\frac{1}{|\mathcal{E}_{f}(x)|}\int_{\mathcal{E}_{f}(x)}F_{k}(\xi)\,\mathrm{d}\xi
Smooth FoVνx(dξ)=K(ξ;x) dξ,∫ΩK(ξ;x) dξ=1\displaystyle\nu_{x}(\mathrm{d}\xi)=K(\xi;x)\,\mathrm{d}\xi,\int_{\Omega}K(\xi;x)\,\mathrm{d}\xi=1Ψk(x)=∫ΩK(ξ;x)Fk(ξ) dξ\displaystyle\Psi_{k}(x)=\int_{\Omega}K(\xi;x)F_{k}(\xi)\,\mathrm{d}\xi
Table 2: Comparison of sensor models used to construct the ergodic occupation measure on the search domain Ω\Omega .
Figure 4: Undetected target intensity λ+(ξ)\lambda^{+}(\xi) Poisson point process thinning during UAV execution and sensing.
Figure 5: RFS measurement update for one sensor.
Figure 6: Closed-loop Bernoulli-triggered re-planning flow: route-overlap evaluation, UGV route re-optimization, UAV search-density and trajectory updates, and joint PMB-gated UGV execution.
Figure 7: Three-UAV coverage and trajectories. Circles, squares, and triangles mark 00 , 55 , and 1010 s; translucent cones show the FoV, and dotted lines mark bounds.
Figure 8: UAV solver convergence. Panels (c)–(d) show the C-ADMM primal and dual residuals through convergence for the first outer attempt of the same solve. Dots mark accepted steps.
Figure 9: Nested SCvx*–C-ADMM convergence for the UAV fleet problem. Colors show the largest residual-to-tolerance ratio; row-end markers show inner termination and outer acceptance.
ParameterValue
Inner C-ADMM
Initial penalties, (ρτ,ρf,ρp)(\rho_{\tau},\rho_{f},\rho_{p})(0.04,0.05,0.02)(0.04,0.05,0.02)
Minimum tolerances, ϵmin⁡\epsilon_{\min}10−510^{-5}
Timing schedule, (c0,d)(c_{0},d)(100,0.8)(100,0.8)
Outer SCvx*
Initial AL penalty, ww2525
Table 7: UAV solver settings for the centralized versus decentralized computational comparisons.
Figure 10: Comparison of centralized and decentralized SCvx*–C-ADMM planning times: mean ±\pm one standard deviation over three cold solves per configuration.
ParameterValue
One Bernoulli track
Track existence probability, rℓr_{\ell}0.950.95
Track mean, μℓ\mu_{\ell}(0.54,0.48)(0.54,0.48)
Principal std., σ1,σ2\sigma_{1},\sigma_{2}0.10, 0.030.10,\ 0.03
Ellipse orientation135∘135^{\circ}
Confidence level, 1−δℓ1-\delta_{\ell}95%95\%
Table 8: Target belief used by UGV planner comparisons.
Figure 11: UGV planner comparison. The ellipse shows track uncertainty; thin outlines show swept corridors, and circles/stars mark start/goal.
PlannerPMB corridor probabilityControl effort EuE_{u}
Mean avoidance37.99%37.99\%0.02140.0214
Confidence-set avoidance7.27%7.27\%0.45160.4516
Full-PMB objective3.64%3.64\%0.73660.7366
Table 9: Risk and control effort for the routes in Figure 11 .
Figure 12: UAV base density updates. Dashed lines show the remaining ground routes.
Figure 13: Closed-loop mission execution: (a) PMB intensity and vehicle configurations; (b) corridor risk bounds; (c) UGV progress. Crosses mark targets; the vertical dashed line marks replanning, and shading marks authorized motion.
Figure 14: Two-UGV SCvx*–C-ADMM solver convergence. Inner residuals correspond to the first outer subproblem and terminate at convergence.
Figure 15: Nested SCvx*–C-ADMM convergence for the UGV fleet problem. Colors show the largest residual-to-tolerance ratio; row-end markers show inner termination and outer acceptance.
Figure 16: Closed-loop collaboration with two UGVs: (a) PMB intensity and vehicle configurations; (b) risk along the union of the remaining ground corridors; (c) ground progress. Vertical dashed lines mark ground replans, and shading marks authorized motion.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 17: Illustration of various geometric objects, properties, and deformation of path-centered corridor in shape calculus.
More in this topic

UAV Trajectory Optimization

UAV trajectory optimization plans feasible, safe flight paths that balance mission goals, energy use, sensing, and communication.

See all 48 papers

Explore similar work

CardsList
Jun 10, 2026

UGV-Conditioned Multi-UAV Informative Planning on a Shared Exposure Belief

Safe ground navigation in large, threat-augmented environments requires aerial support that actively reduces the risks that a ground vehicle faces along its route. Existing aerial reconnaissance systems focus on mapping or covering the environment, but do not direct sensing toward regions that are most relevant for ground vehicle safety. In this paper, we address the problem of coordinating a team of unmanned aerial vehicles (UAVs) to improve the safety of an unmanned ground vehicle (UGV) navigating through unknown threat zones. A key aspect of our approach is a risk-averse shared exposure belief for UGV routing and a mission-conditioned UAV reward that directs sensing toward uncertain, exposure-relevant regions of the current UGV corridor. UAV observations update the shared belief online, allowing the UGV to replan around newly revealed threats while the evolving route in turn reshapes UAV sensing priorities. For multiple UAVs, we use region-conditioned spatial assignment to reduce redundant sensing. Simulation experiments show that our approach reduces cumulative UGV exposure by 38% compared with terrain-semantic replanning, and reduces redundant aerial coverage from 38.8% to 3.7% under our multi-UAV coordination scheme.

Lars Oerlemans, Moji Shi, Marija Popovic

*Equal contribution. Authors are with the MAVLab, Faculty of Aerospace Engineering, TU Delft, The Netherlands.

Informative Path PlanningSafe Robot NavigationMulti-Robot CoordinationUAV Remote Sensing
AskPDF
Jul 7, 2026

Towards Reliable Aerial Ground Vehicle Collaboration: An Integrated Planning and Autonomy Framework for Field Deployment

Limited flight endurance significantly restricts the operational range of unmanned aerial vehicles (UAVs) in long duration missions such as surveillance and inspection, where multiple spatially distributed Areas of Interest (AOIs) must be visited. These tasks require efficient routing determining the sequence of visits which directly impacts mission time, energy consumption, and overall feasibility. Pairing UAVs with unmanned ground vehicles (UGVs) for mobile recharging offers a promising solution, but introduces a tightly coupled cooperative routing problem involving UAV route planning, UGV road constrained movement, energy management, and rendezvous scheduling under uncertainty. In this work, we present an integrated planning and autonomy framework for reliable field deployment. We formulate the problem as an energy constrained cooperative routing task and solve it using a Deep Reinforcement Learning (DRL) based planner that jointly optimizes the UAV visitation sequence and rendezvous locations with the UGV, outperforming baseline heuristics in minimizing total mission time. To bridge the gap between planning and execution, we introduce a standardized two layer YAML based mission API that captures environment states and structures lightweight, synchronized action sequences. This framework is supported by a complete autonomy stack using PX4/MAVSDK for UAV control and ROS 2/Nav2 for UGV navigation. Furthermore, we propose a lightweight Rendezvous Aware Replanner (RARP) that operates online to handle environmental uncertainties, reducing energy margin violations from 83.33% to 20.00%. The full system is validated through outdoor field experiments, demonstrating robust cooperative navigation and adaptability in dynamic tasks, including a search and rescue scenario with vision language model (VLM) based hazard detection

Md Safwan Mondal, Luca Russo, James D. Humann +2

Department of Mechanical and Industrial Engineering, University of Illinois Chicago, IL, 60607 USA · DEVCOM Army Research Laboratory, Minneapolis, Minnesota, USA · DEVCOM Army Research Laboratory, Aberdeen Proving Grounds, Aberdeen, MD 21005 USA

Vehicle RoutingMulti-Robot CoordinationDynamic Replanning
AskPDF
Mar 6, 2026

Collaborative Planning with Concurrent Synchronization for Operationally Constrained UAV-UGV Teams

Collaborative planning under operational constraints is an essential capability for heterogeneous robot teams tackling complex large-scale real-world tasks. Unmanned Aerial Vehicles (UAVs) offer rapid environmental coverage, but flight time is often limited by energy constraints, whereas Unmanned Ground Vehicles (UGVs) have greater energy capacity to support long-duration missions, but movement is constrained by traversable terrain. Individually, neither can complete tasks such as environmental monitoring. Effective UAV-UGV collaboration therefore requires energy-constrained multi-UAV task planning, traversability-constrained multi-UGV path planning, and crucially, synchronized concurrent co-planning to ensure timely in-mission recharging. To enable these capabilities, we propose Collaborative Planning with Concurrent Synchronization (CoPCS), a learning-based approach that integrates a heterogeneous graph transformer for operationally constrained task encoding with a transformer decoder for joint, synchronized co-planning that enables UAVs and UGVs to act concurrently in a coordinated manner. CoPCS is trained end-to-end under a unified imitation learning paradigm. We conducted extensive experiments to evaluate CoPCS in both robotic simulations and physical robot teams. Experimental results demonstrate that our method provides the novel multi-robot capability of synchronized concurrent co-planning and substantially improves team performance. More details of this work are available on the project website: https://hcrlab.gitlab.io/project/CoPCS.

Zihao Deng, Qianhuang Li, Peng Gao +4

Zihao Deng, Qianhuang Li, Donghyun Kim, and Hao Zhang are with the University of Massachusetts Amherst, Amherst, MA, 01002, USA. · Peng Gao is with North Carolina State University, Raleigh, NC, 27695, USA. · Maggie Wigness and John Rogers are with the U.S. Army DEVCOM Army Research Laboratory (ARL), Adelphi, MD, 20783, USA.

Multi-Robot CoordinationMulti-Robot Motion PlanningMulti-Robot Task Allocation
AskPDF