Interruption-Aware Recurrent Reinforcement Learning for UAS Navigation Under Temporary Overrides

Christopher Conrad; Deepak Panda; Weisi Guo; Antonios Tsourdos

cs.AIOct 20, 2026

Interruption-Aware Recurrent Reinforcement Learning for UAS Navigation Under Temporary Overrides

Authors: Christopher Conrad, Deepak Panda, Weisi Guo, Antonios Tsourdos

Organizations: Centre for Assured and Connected Autonomy, Cranfield University, Bedford, UK

Abstract

Safe autonomous navigation of UASs requires tactical deconfliction to address conflicts that strategic coordination cannot prevent. This task is challenging under partial observability, when sensor or traffic data are missing, delayed, or degraded. Geometry-based methods can offer separation assurances but face limitations in dense encounters. Recurrent reinforcement learning has therefore been proposed to learn navigation policies that integrate observation-action histories to optimise long-term safety-efficiency trade-offs. However, operational safeguards may require a remote pilot, onboard detect-and-avoid system, or runtime assurance mechanism to temporarily override the learned policy. A research gap remains in understanding how these interruptions affect recurrent policies when the agent regains authority. Benchmark evidence shows that interruptions make task variables less decodable from the recurrent hidden state and increase post-interruption collision risk. The findings suggest that altered recurrent context prolongs post-handback degradation through an action-observation feedback loop, in which changed actions alter the flight path and subsequent recurrent updates. To address this vulnerability, we propose three interruption-aware architectures that condition recurrent processing on interruption context through separate feature streams, dedicated interruption memory, or gated environmental features. Across simulations, all three architectures preserve more decodable task information than a standard recurrent policy. Moreover, the strongest architecture reduces post-interruption collisions by 74% and interruption-induced mission failures by 69% relative to the recurrent baseline, without degrading interruption-free performance. These results show that safe interruption does not guarantee safe handback, and support interruption-aware recurrent design for resilient UAS autonomy.

Figures & tables

Research area	Primary focus	Key references	Research Gap
UAS supervision and transport handovers	Focuses on interruption timing, override conditions, operator readiness, and situational awareness.	[ 3 , 4 , 7 , 8 , 9 , 25 , 26 , 27 , 5 ]	Establishes the operational basis for temporary supervisor overrides, but is primarily concerned with system-level safety and human performance.
Recurrent RL for UAS navigation	Uses recurrent policies for navigation and collision avoidance under partial observability.	[ 23 , 28 ]	Establishes the relevance of recurrent policies to UAS navigation, but assumes that the nominal policy selects all executed actions.
Safe RL and shielded RL	Incorporates safety constraints into learning and prevents unsafe actions from being executed.	[ 29 , 30 , 31 ]	Learns safe policies and prevents unsafe action execution. Although shielding may replace policy actions, recurrent handback is not considered.
Corrigibility and safe interruptibility	Ensures that agents accept external correction without seeking, resisting, or manipulating interruptions.	[ 32 , 33 , 34 , 35 ]	Examines behavioural incentives and the policy learned when interruptions occur during training, but does not consider deployment-time interruptions.
Modified-action and human-in-the-loop RL	Learns when executed actions differ from policy-selected actions, and uses interruptions as corrective feedback.	[ 36 , 37 ]	Directly represents action substitution, but focuses on its effect on learning rather than recurrent state degradation and post-interruption performance.
Recurrent states and modular memory	Measures information encoded in recurrent states, and structures memory to improve robustness.	[ 24 , 38 , 39 , 40 ]	Provides diagnostic and architectural foundations, but does not consider the effect of interruptions.

TABLE I: Summary of existing literature relevant to interrupting recurrent navigation policies.

Fig. 1: Operational context and problem overview for interrupting a recurrent UAS navigation policy.

Function and temporal scope	Representative approaches	Impact of interruptions and handback
Stabilisation (milliseconds, low-level control)	PID, model-based control and physics- informed RL [ 41 ]	Not relevant . Commands can be recomputed from aircraft state without traffic history.
Strategic coordination (pre-flight)	Scheduling, optimisation and RL/MARL [ 42 ]	Not relevant . Plans can be revised without interrupting an active navigation policy.
Short-horizon collision avoidance (seconds, local manoeuvres)	Velocity obstacles, potential fields and feedforward RL/MARL [ 15 ]	Not relevant . A new manoeuvre can be computed from traffic estimates without recurrent context.
Long-horizon tactical navigation (minutes-hours, mission execution)	Recurrent and history-dependent RL [ 43 ]	Primary scope . Safe handback requires retained traffic and mission context, which is impacted by interruptions.

TABLE II: Impact of interruptions across UAS navigation functions

Operational element	Simulation abstraction
Tactical UAS navigation	One ego UAS follows a start-destination mission while avoiding traffic and respecting operational boundaries, representing a vehicle- or operator-side navigation policy.
Incomplete traffic information	Sensor range is limited and a restricted track set is refreshed intermittently, held between updates with an age indicator, and stochastically blanked, representing incomplete or delayed onboard and networked information.
Temporary overrides	An external supervisor replaces the nominal yaw-rate command for a bounded interval, representing a DAA, RTA, remote-pilot or harmful interruption.
Handback	The recurrent policy resumes from the aircraft and memory states reached during an interruption, exposing the post-handback reliability problem.
Operational constraints and objectives	Boundary clamping approximates an operational-volume or geozone limit, and the reward defines safety and efficiency objectives for a finite learning task.

TABLE III: Operational Interpretation of Simulation Abstractions

Panel of Figure 5 — Fig. 2: Overview of proposed interruption-aware recurrent architectures: (a) Dual-feature architecture. (b) Dual-memory architecture. (c) Gated architecture.

Component	Parameter	Setting
Environment	Operating region	$20\times 20$ m planar region; start and destination randomised by episode.
	Ego motion	Constant ground speed of $1.5$ m/s; normalised yaw-rate action $a_{t}\in[-1,1]$ ; maximum turn rate of $60^{\circ}$ /s.
	Temporal resolution	$\Delta t=0.1$ s; maximum training-episode horizon of 500 steps.
	Goal and collision geometry	Goal radius of $0.5$ m; ego and intruder radii of $0.15$ m.
Traffic	Intruder count	Training curriculum of $\{3,5,7,8,9,10\}$ intruders; ten intruders during evaluation.
	Intruder motion	Constant ground speed of $0.5$ m/s with reflection at the operating-region boundary.

TABLE IV: Numerical Settings for Simulation and Training

Fig. 3: Reference, interrupted and transplanted arms used to evaluate the impact of interruptions and interruption-aware recurrent architectures.

Fig. 4: Training curves of memoryless (pink) and recurrent (gray) PPO agents for the simulated UAS navigation task.

Fig. 5: Absolute decodability score of the reference arm hidden states for different task-relevant variables, averaged across a 45-step window.

Fig. 6: Memory degradation caused by interruptions, measured as a reduction in decodability of task-relevant variables from the recurrent hidden state averaged across a 45-step post-handback window.

Fig. 7: Lost-success rate for episodes with different corrected decoding gaps, confirming that greater memory degradation results in poorer post-handback task performance.

Fig. 8: Contribution of different signals to command differences, with carried cell content showing the largest contribution.

Fig. 9: Contraction of the reference and interrupted arm recurrent states after handback with shared inputs (blue) and closed-loop feedback (orange).

Fig. 10: Lost success rate for different supervisor behaviours and durations, showing that longer and less optimal supervisors are generally more harmful.

Fig. 11: Lost success rate for episodes with multiple interruption segments, demonstrating that longer contiguous interruptions are more harmful.

Fig. 12: Naive mitigations fail to improve post-handback performance.

Fig. 13: Training curves showing convergence of interruption-aware and recurrent baseline policies.

Fig. 14: Belief degradation against behavioural harm for each architecture, confirming that the interruption-aware architectures perform better than the recurrent baseline.

Fig. 15: Post-handback performance improvements with interruption-aware architectures.

Fig. 16: Example trajectories generated by each architecture for the same environmental scenario.

Fig. 17: Performance improvements of interruption-aware architectures with state-triggered interruptions.

Fig. 18: Interface used for the human-in-the-loop demonstration, based on the physics-informed PyBullet simulation framework.

Fig. 19: Performance improvements of interruption-aware architectures with human operator-based interruptions.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation
Jul 2, 2026Shenghui Zhang, YuXuan Gao, Songwei Zhao +3Robot Policy Learning UAV Navigation
LNN-Fly: Continuous-Time UAV Navigation for Robust Obstacle Avoidance under Timing Mismatch
Jun 27, 2026Yulin Huang, Shaojie Chen, Di Feng +3Aerial Robotics Robot Navigation
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, Guangdong 518000, China
Runtime Safety Filtering for Learned Small UAS Separation Policies under GNSS Degradation
Jul 10, 2026Alex Zongo, Peng WeiCollision Avoidance Safety Filtering
Department of Mechanical and Aerospace Engineering · George Washington University · Washington, D.C., USA

Jul 2, 2026J/K move · Enter open · S save

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

Shenghui Zhang, YuXuan Gao, Songwei Zhao, Jifeng Hu+2

Robot Policy Learning UAV Navigation Constrained RL Safe RL

Open paper AskPDF arXiv

With the rapid development of autonomous aerial systems, Unmanned Aerial Vehicles (UAVs) are increasingly deployed in applications such as inspection, environmental monitoring, and rescue, creating growing demand for reliable autonomous navigation. However, autonomous UAV navigation in dense environments remains challenging under sparse perception and dynamic constraints. Most reinforcement learning (RL) methods lack explicit safety mechanisms, leading to unsafe exploration, unstable training, and risky behaviors, especially during high-speed flight. Even in safe RL approaches, safety is often enforced by projecting policy outputs onto a safe action set, which may introduce instability. Meanwhile, many learning-based methods rely on dense inputs or large networks, increasing computational burden and limiting lightweight onboard deployment. Facing the above challenges, we propose a safety-constrained perception-control integrated framework for UAV navigation. A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions. We formulate the task as a constrained Markov decision process within a hierarchical control architecture and solve it using a Lagrangian-based safe PPO algorithm. Curriculum learning further improves training stability. Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency than existing reinforcement learning baselines.

Abstract

UAV Navigation

Explore similar work

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

LNN-Fly: Continuous-Time UAV Navigation for Robust Obstacle Avoidance under Timing Mismatch

Runtime Safety Filtering for Learned Small UAS Separation Policies under GNSS Degradation