cs.LGNov 2, 2026

Beyond L∞L_\infty-Norm Sets: Synthesizing Interpretable Non-Uniform Uncertainty Sets for Evaluation-Time State-Adversarial Robustness

Authors: Xulin Chen, Ruipeng Liu, Garrett E. Katz

Organizations: Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13244 USA

Abstract

Deep reinforcement learning (DRL) agents are vulnerable to perturbations of their observed state inputs. Prior works often constrain state adversaries with LpL_p-norm uncertainty sets and evaluate policy performance under the resulting perturbations. However, in continuous control tasks like robotics control, different state dimensions have distinct physical meanings and influence policy behavior to different degrees, which LpL_p-norm sets fail to capture or explain. Moreover, it is important to identify when and along which state dimensions a policy is vulnerable, independently of the algorithm used to train it. In this paper, we focus on evaluation-time attacks applied to pretrained policies. We propose Performance-Degradation Non-Uniform Uncertainty Set (\method{}), an interpretable formulation of uncertainty sets for state perturbations. \method{} is state-dependent, dimension-wise non-uniform, and can be synthesized online during policy evaluation. We first formulate an ideal constrained optimization problem for \method{} computation, then introduce a surrogate constraint to make the problem tractable. Experiments across multiple continuous-control environments, policy algorithms, and state attacks show that \method{} reveals variation in policy sensitivity across states and dimensions, while its performance degradation is moderately controlled.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Training Verifiably Robust Agents Using Set-Based Reinforcement Learning

    Aug 17, 2024Manuel Wendl, Lukas Koller, Tobias Ladner +1RL ControlReinforcement Learning

  2. Robust Policy Optimization via Adversarial Importance Sampling

    Sep 11, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouRobust RLPolicy Learning

  3. A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning

    Sep 11, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouKL-Regularized RLAdversarial Robustness