KC-BFPRL: Knowledge-Guided Multi-UAV Collaboration for Grassland Restoration via Bilevel Formerpointer-Based Reinforcement Learning
Authors: Dongbin Jiao, Xianyi Wang, Yuchen Yuan, Weibo Yang, Peng Yang, Peng Zhao, Zhanhuan Shang, Shi Yan
Organizations: School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, P. R. China · Key Laboratory of Tourism Information Fusion Processing and Data Ownership Protection, Ministry of Culture and Tourism, Lanzhou University, Lanzhou, 730000, P. R. China · School of Automobile, Chang’an University, Xi’an, 710064, P. R. China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen 518055, P. R. China · State Key Laboratory of Grassland Agro-Ecosystem, College of Ecology, Lanzhou University, Lanzhou, 730000, P. R. China
Multi-unmanned aerial vehicle (UAV) systems provide scalable service platforms for large-scale environmental tasks, such as grassland ecosystem restoration. However, coordinating fleet operations requires solving the restoration area maximization problem (RAMP). This non-linear combinatorial optimization challenge is complicated by payload-dependent energy dynamics and heterogeneous ecological degradation. We propose a novel knowledge-guided collaborative bilevel formerpointer reinforcement learning framework (KC-BFPRL) to address this complexity. Using a hierarchical paradigm, KC-BFPRL decomposes RAMP into global task allocation and local restoration planning, with the latter further divided into upper-level trajectory planning and lower-level restoration area allocation. Our specialized architecture pairs featuring a Transformer-based encoder that fuses static environmental features with dynamic UAV states, and a Pointer Network decoder trained via a robust actor-critic framework. By embedding ecological priority rules and heuristic logic, KC-BFPRL achieves a structured warm-start, solving the RL cold-start problem while ensuring strict constraint satisfaction. Extensive experiments demonstrate that KC-BFPRL consistently outperforms state-of-the-art baselines, achieving superior objective values and efficiency. It maintains a 0.00% optimality gap in the most complex scenarios U8-R160 and operates nearly three times faster than MAPDP, validating its robustness, scalability, and real-time applicability for large-scale automated ecological restoration.
Figures & tables
Fig. 1: An example of multi-restored regions by multi-UAV collaborative.
Fig. 4: Comparison of training convergence curves for the three learning-based algorithms in scenario U4-R60.
Fig. 5: Optimal trajectories generated by the four algorithms for 4, 6, and 8 UAVs across three small-to-medium-scale scenarios: (a)-(c) CAMP, (d)-(f) CHAPBILM, (g)-(i) MAPDP, and (j)-(l) KC-BFPRL.
Areas Pending Restoration
Metric
Algorithms
CAMP
CHAPBILM
MAPDP
KC-BFPRL
60
Obj.
675.60
798.51
862.34
945.80
Areas
612
742
789
863
Time (s)
11.21
892.49
45.79
15.81
Gap (%)
28.59
15.57
8.83
0.00
80
Obj.
878.00
1065.42
1125.51
1248.27
TABLE I: Comparison of Average Performance Across Four Algorithms for 4-UAV Collaborative Grassland Restoration in All Scenarios.
Areas Pending Restoration
Metric
Algorithms
CAMP
CHAPBILM
MAPDP
KC-BFPRL
60
Obj.
1010.59
1198.18
1306.18
1425.49
Areas
918
1089
1187
1295
Time (s)
16.82
1340.78
68.98
22.74
Gap (%)
29.10
15.95
8.37
0.00
80
Obj.
1317.09
1528.79
1898.60
1834.28
TABLE II: Comparison of Average Performance Across Four Algorithms for 6-UAV Collaborative Grassland Restoration in All Scenarios.
Areas Pending Restoration
Metric
Algorithms
CAMP
CHAPBILM
MAPDP
KC-BFPRL
60
Obj.
1347.36
1428.41
1789.27
1718.95
Areas
1224
1298
1625
1562
Time (s)
69.77
1643.51
82.11
22.64
Gap (%)
24.69
20.16
0.00
3.93
80
Obj.
1756.63
1868.25
2076.01
2298.61
TABLE III: Comparison of Average Performance Across Four Algorithms for 8-UAV Collaborative Grassland Restoration in All Scenarios.
Fig. 6: Comparative performance of the four algorithms on four key metrics.
Fig. 7: Visualization of optimal KC-BFPRL trajectories for 4, 6, and 8 UAVs across three medium-to-large-scale scenarios.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Description
Value
nlayers
number of Transformer layers
3
din
input dimension
4 ( x , y , l , R )
de
embedding dimension
128
C
attention scaling factor
10
Nepoch
number of epochs
100
B
batch size
256
Appendix
TABLE IV: Transformer model training parameters
Fig. 8: Evolution of total restoration areas across training epochs for the three learning-based algorithms in scenario U4-R60.
Fig. 9: Visualization of optimal KC-BFPRL trajectories for 4, 6, and 8 UAVs across three small-to-medium-scale scenarios.
Metric
Algorithm
Mean
Std
Median
Min
Max
Q75
Restoration Areas
CAMP
700
359
625
203
2216
838
CHAPBILM
866
449
725
292
2263
1067
MAPDP
994
487
872
262
2510
1232
KC-BFPRL
1109
544
971
322
2941
1395
Energy Efficiency
CAMP
0.457
0.043
0.437
0.411
0.570
0.477
CHAPBILM
0.494
0.052
0.469
0.431
0.582
0.547
Appendix
TABLE V: Comparison of average performance metrics across the four algorithms.
Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.
Limited flight endurance significantly restricts the operational range of unmanned aerial vehicles (UAVs) in long duration missions such as surveillance and inspection, where multiple spatially distributed Areas of Interest (AOIs) must be visited. These tasks require efficient routing determining the sequence of visits which directly impacts mission time, energy consumption, and overall feasibility. Pairing UAVs with unmanned ground vehicles (UGVs) for mobile recharging offers a promising solution, but introduces a tightly coupled cooperative routing problem involving UAV route planning, UGV road constrained movement, energy management, and rendezvous scheduling under uncertainty. In this work, we present an integrated planning and autonomy framework for reliable field deployment. We formulate the problem as an energy constrained cooperative routing task and solve it using a Deep Reinforcement Learning (DRL) based planner that jointly optimizes the UAV visitation sequence and rendezvous locations with the UGV, outperforming baseline heuristics in minimizing total mission time. To bridge the gap between planning and execution, we introduce a standardized two layer YAML based mission API that captures environment states and structures lightweight, synchronized action sequences. This framework is supported by a complete autonomy stack using PX4/MAVSDK for UAV control and ROS 2/Nav2 for UGV navigation. Furthermore, we propose a lightweight Rendezvous Aware Replanner (RARP) that operates online to handle environmental uncertainties, reducing energy margin violations from 83.33% to 20.00%. The full system is validated through outdoor field experiments, demonstrating robust cooperative navigation and adaptability in dynamic tasks, including a search and rescue scenario with vision language model (VLM) based hazard detection
Md Safwan Mondal, Luca Russo, James D. Humann +2
Department of Mechanical and Industrial Engineering, University of Illinois Chicago, IL, 60607 USA · DEVCOM Army Research Laboratory, Minneapolis, Minnesota, USA · DEVCOM Army Research Laboratory, Aberdeen Proving Grounds, Aberdeen, MD 21005 USA
This paper addresses multi-objective conflicts and catastrophic forgetting in uncrewed aerial vehicle (UAV) networks across dynamic spatiotemporal environments. Conventional multi-agent reinforcement learning (MARL) algorithms suffer from severe policy degradation during sequential task transitions. We propose a spatiotemporal continual federated learning (SCFL) framework driven by the group-decoupled multi-agent proximal policy optimization (G-MAPPO) algorithm. SCFL incorporates a three-stage geometric alignment mechanism: it resolves local gradient conflicts via group-decoupled policy optimization (GDPO), mitigates spatial Non-Independent and Identically Distributed (non-IID) client drift through adaptive cosine aggregation, and suppresses inter-task interference via global temporal orthogonal projection without raw experience replay. Evaluations show that SCFL achieves superior robustness over federated baselines, maintaining spatial service reliability above 0.95 and a load balancing index of approximately 0.95 during non-stationary transitions. A longitudinal self-degradation analysis further shows that SCFL preserves historical knowledge with near-zero performance variation in spatial reliability and QoS under moderate loads from 40 to 120 users, while revealing its operating boundary under extreme congestion with 140 users due to hard projection constraints. The framework provides a scalable, communication-efficient approach for autonomous aerial network orchestration.
Chuan-Chi Lai
Department of Communications Engineering, National Chung Cheng University, Minxiong Township, Chiayi County 621301, Taiwan · Advanced Institute of Manufacturing with High-tech Innovations (AIM-HI), National Chung Cheng University, Minxiong Township, Chiayi County 621301, Taiwan
This paper investigates a multi-Unmanned Aerial Vehicle (UAV) joint base station-assisted Internet of Vehicles (IoV) task offloading system in dense urban environments. To minimize system delay and energy consumption under strict coupling constraints, the complex non-convex optimization problem is decoupled into a hierarchical execution framework. First, a sequential distributed optimization algorithm based on Second-Order Cone Programming (SOCP) is proposed to optimize the 3D flight trajectory of each UAV, ensuring adaptive network coverage. Second, a novel hybrid resource scheduling paradigm synergizing Deep Reinforcement Learning (DRL) and Large Language Models (LLMs) is developed. Within this framework, the DRL agent dictates the initial resource allocation, while the LLM acts as a semantic macro-scheduler to rectify long-tail allocation imbalances for failed and surplus tasks. Crucially, a reward decoupling mechanism is introduced to isolate DRL training from external LLM interventions, thereby ensuring policy convergence. Finally, the task offloading ratios are precisely determined via Linear Programming (LP) within an alternating optimization loop. Simulation results demonstrate that the proposed method significantly outperforms traditional multi-agent reinforcement learning baselines in terms of task success rate and system efficiency.
Maoxin Ji, Qiong Wu, Pingyi Fan +4
School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China · School of Information Engineering, Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications, Nanchang University, Nanchang 330031, China · Department of Electronic Engineering, State Key laboratory of Space Network and Communications, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China +4