cs.LGMar 1, 2026

Intent-Context Synergy Reinforcement Learning for Autonomous UAV Decision-Making in Air Combat

Authors: Jiahao Fu, Feng Yang

Organizations: National Elite Institute of Engineering,Northwestern Polytechnical University, Xi’an, Shanxi, China · School of Automation, Northwestern Polytechnical University, Xi’an, Shanxi, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Autonomous UAV infiltration in dynamic contested environments remains a significant challenge due to the partially observable nature of threats and the conflicting objectives of mission efficiency versus survivability. Traditional Reinforcement Learning (RL) approaches often suffer from myopic decision-making and struggle to balance these trade-offs in real-time. To address these limitations, this paper proposes an Intent-Context Synergy Reinforcement Learning (ICS-RL) framework. The framework introduces two core innovations: (1) An LSTM-based Intent Prediction Module that forecasts the future trajectories of hostile units, transforming the decision paradigm from reactive avoidance to proactive planning via state augmentation; (2) A Context-Analysis Synergy Mechanism that decomposes the mission into hierarchical sub-tasks (safe cruise, stealth planning, and hostile breakthrough). We design a heterogeneous ensemble of Dueling DQN agents, each specialized in a specific tactical context. A dynamic switching controller based on Max-Advantage values seamlessly integrates these agents, allowing the UAV to adaptively select the optimal policy without hard-coded rules. Extensive simulations demonstrate that ICS-RL significantly outperforms baselines (Standard DDQN) and traditional methods (PSO, Game Theory). The proposed method achieves a mission success rate of 88% and reduces the average exposure frequency to 0.24 per episode, validating its superiority in ensuring robust and stealthy penetration in high-dynamic scenarios.

Figures & tables

Figure 1: Mission Scenario Diagram.
Figure 2: UAV Kinematics model.
Figure 3: Fundamental framework of reinforcement learning.
Figure 4: The structure diagram of air combat decision model.
ParameterDDQNCA-DDQNICS-RL (Ours)
Replay Buffer Size5×1055\times 10^{5}5×1055\times 10^{5}5×1055\times 10^{5}
Training Episodes40,00040,00040,000
Learning Rate1×10−51\times 10^{-5}1×10−51\times 10^{-5}1×10−51\times 10^{-5}
Discount Factor ( γ\gamma )0.990.990.99
Batch Size512512512
Table 1: Hyperparameter Settings for Comparative Algorithms
Figure 5: Comparison of cumulative reward curves between CA-DDQN and Standard DDQN.
Figure 6: Comparison of cumulative reward curves between ICS-DDQN and CA-DDQN.
Figure 7: Frequency of decision adoption for each specialized agent during training.
Figure 8: Comparison of positive reward steps, indicating mission completion speed.
Figure 9: Statistical distribution of total rewards over 50 test episodes.
MethodSRAECPA
PSO69.0%1.87N/A
Game Theory77.0%1.41N/A
Standard DDQN64%1.56N/A
CA-DDQN (Ours)80%1.15N/A
ICS-RL (Ours)88%0.2480.2%
Table 2: Performance Comparison with Traditional and DRL Algorithms
Figure 10: Detail of a trajectory performed by ICS-DDQN
More in this topic

Reinforcement Learning

Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.

See all 1,717 papers

Explore similar work

CardsList
Aug 14, 2026

AgilePE: Autonomous UAV Pursuit-Evasion via Self-Play Reinforcement Learning

Autonomous pursuit-evasion is a fundamental challenge for Unmanned Aerial Vehicles (UAVs), requiring rapid decision-making under tightly coupled dynamics and continuously changing opponent behaviors. Traditional rule-based or differential-game approaches often struggle with high-dimensional aerial interactions and agile maneuvering. We present AgilePE, a complete system for autonomous UAV pursuit-evasion via self-play reinforcement learning. AgilePE integrates agile low-level control, competitive policy optimization, and sim-to-real deployment in a unified framework. The policy directly maps onboard state observations to Collective Thrust and Body Rates (CTBR) commands, enabling end-to-end agile maneuvering without intermediate trajectory planners or waypoint controllers. For training, we use competitive self-play with Prioritized Fictitious Self-Play (PFSP) and a diversified opponent pool, enabling agents to improve against historical policies while stabilizing optimization and reducing policy oscillation. This process leads to the emergence of sophisticated pursuit and evasion strategies. For real-world deployment, we develop a hardware-aligned simulation pipeline that models actuator-response dynamics, communication latency, and domain randomization. The learned policies transfer zero-shot to real quadrotors without task-specific tuning. Real-world experiments reproduce pursuit-evasion tactics observed in simulation, including rapid dodging and flanking, and demonstrate interactive two-agent zero-shot deployment.

Wenhao Tang, Tianyang Chen, Zhejun Cui +9

Tsinghua University, Beijing, 100084, China. · Zhejiang University, Hangzhou, 310058, China. · Chongqing University, Chongqing, 400044, China.

Robot Policy LearningSelf-Play RLMulti-Agent Reinforcement Learning
AskPDF
Jul 2, 2026

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

With the rapid development of autonomous aerial systems, Unmanned Aerial Vehicles (UAVs) are increasingly deployed in applications such as inspection, environmental monitoring, and rescue, creating growing demand for reliable autonomous navigation. However, autonomous UAV navigation in dense environments remains challenging under sparse perception and dynamic constraints. Most reinforcement learning (RL) methods lack explicit safety mechanisms, leading to unsafe exploration, unstable training, and risky behaviors, especially during high-speed flight. Even in safe RL approaches, safety is often enforced by projecting policy outputs onto a safe action set, which may introduce instability. Meanwhile, many learning-based methods rely on dense inputs or large networks, increasing computational burden and limiting lightweight onboard deployment. Facing the above challenges, we propose a safety-constrained perception-control integrated framework for UAV navigation. A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions. We formulate the task as a constrained Markov decision process within a hierarchical control architecture and solve it using a Lagrangian-based safe PPO algorithm. Curriculum learning further improves training stability. Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency than existing reinforcement learning baselines.

Shenghui Zhang, YuXuan Gao, Songwei Zhao +3

Robot Policy LearningUAV NavigationConstrained RLSafe RL
AskPDF
May 3, 2026

Zero-Shot, Safe and Time-Efficient UAV Navigation via Potential-Based Reward Shaping, Control Lyapunov and Barrier Functions

Autonomous navigation and obstacle avoidance remain a core challenge of modern Unmanned Aerial Vehicles (UAVs). While traditional control methods struggle with the complexity and variability of the environment, reinforcement learning (RL) enables UAVs to learn adaptive behaviors through interaction with the environment. Existing research with RL prioritizes the mission success at the expense of mission time and safety of UAVs. This study integrates Potential Based Reward Shaping (PBRS) with Control Lyapunov Functions (CLF) and Control Barrier Functions (CBF) to simultaneously optimize mission time and ensure formal safety guarantees. An RL model is trained in a generalized simple environment, then used in complex scenarios incorporating a CLF-CBF-QP filter without further training. Experimental results in simulated environments demonstrate a significant reduction in mission time and outstanding performance in complex environment.

Ashik Abrar Naeem, Mohammad Ariful Haque

RL ControlControl Lyapunov FunctionsUAV NavigationSafe Robot Navigation
AskPDF