cs.LGNov 5, 2026

Monotone Viability Learning for Offline Safe Reinforcement Learning

Authors: Ziqian Wang, Zhen Zhang

Organizations: Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China.

Abstract

Offline safe reinforcement learning requires reliable long-horizon safety estimates from fixed data. Learning Hamilton-Jacobi (HJ) viability in this setting calls for semantic preservation, convergence, and numerical stability. To address these challenges, we propose Monotone Viability Learning (MoViL), which learns viable sets from offline transitions through monotone finite-horizon set recursion. The recursion preserves finite-horizon safety semantics and converges to the infinite-horizon viability kernel. Binary supervision maintains a fixed target scale across horizons, avoiding discount-induced numerical attenuation. We further establish sufficient conditions for recovering behavior-supported viable subsets. The learned viability estimates are used to certify actions at deployment. Dubins car experiments validate the numerical accuracy of the learned viable sets. Evaluations on safe RL benchmarks demonstrate state-of-the-art reward-cost performance. MoViL satisfies the average cost constraint on every task and achieves the lowest average cost among the compared methods. These results highlight MoViL's potential to extend long-horizon HJ viability analysis to safety-critical systems where reliable dynamics models are unavailable and online exploration is unsafe.

Figures & tables

Explore similar work

CardsList
  1. Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies

    Mar 16, 2026Mumuksh Tayal, Manan Tayal, Ravi PrakashFlow MatchingSafe Offline RL

  2. Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learning

    Mar 8, 2026Janaka Chathuranga Brahmanage, Akshat KumarSafe RLSafe Offline RL

  3. EpiFlow: Bridging Safety and Performance in Offline Reinforcement Learning via Epigraph-Guided Flow Matching

    Feb 8, 2026Manan Tayal, Mumuksh Tayal, Ravi PrakashSafe Offline RLOffline RL