eess.SYMar 29, 2026

Fundamental Limits of Man-in-the-Middle Attack Detection in Model-Free Reinforcement Learning

Authors: Rishi Rani, Massimo Franceschetti

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We consider the problem of learning-based man-in-the-middle (MITM) attacks in cyber-physical systems (CPS), and extend our previously proposed Bellman Deviation Detection (BDD) framework for model-free reinforcement learning (RL). We refine the standard MDP attack model by allowing the reward function to depend on both the current and subsequent states, thereby capturing reward variations induced by errors in the adversary's transition estimate. We also derive an optimal system-identification strategy for the adversary that minimizes detectable value deviations. Further, we prove that the agent's asymptotic learning time required to secure the system scales linearly with the adversary's learning time, and that this matches the optimal lower bound. Hence, the proposed detection scheme is order-optimal in detection efficiency. Finally, we extend the framework to asynchronous and intermittent attack scenarios, where reliable detection is preserved.

Figures & tables

Fig. 1: (a) Adversary Learning Phase: During this phase, the attacker eavesdrops and learns the system dynamics without altering the feedback signal to the agent. (b) Adversary Attack Phase: During this phase, the attacker intercepts the feedback loop and provides a falsified signal to the agent to induce a target policy or cause value deviation.
Fig. 2: This figure shows the absolute value of the maximum BDPM for the cases when an attack occurs (in orange) and when the system is secure (in blue). The security bound (in dotted blue) upper bounds the BDPM when no attack occurs and lower bounds the BDPM when attack occurs, if the information advantage condition is met.
Fig. 3: This figure shows the probability of the agent detecting an attack, calculated based on 1000 trials for varying amounts of agent and adversary learning times.
More in this topic

CPS Security

See all 25 papers

Explore similar work

CardsList

  1. Model-Free Reinforcement Learning Control for Resilient Cyber-Physical Systems

    Jun 17, 2026Hugo O. Garcés, Alejandro J. Rojas, Bernardo A. Hernández +5RL ControlCyber-Physical Systems

    Departamento de Ingeniería Informática y Ciencias de la Computación, Universidad de Concepción, Concepción, Chile · Electrical Engineering Departament, Universidad de Talca · Department of Electrical & Computer Engineering, University of Alberta, Edmonton, T6G 1H9, Alberta, AB, Canada +2

  2. Attack Detection using Time Series Foundation Models

    Jun 4, 2026Sribalaji C. Anand, Anh Tung Nguyen, George J. PappasCyber-Physical SystemsCybersecurity

    University of Pennsylvania, United States · Uppsala University, Sweden

  3. Identifying Adversary Characteristics from an Observed Attack

    Mar 5, 2026Soyon Choi, Scott Alfeld, Meiyi MaQuery IdentifiabilityAdversarial Attacks

    Vanderbilt University Nashville, TN 37240 · Amherst College Amherst, MA 01002