cs.LGNov 2, 2026

A Universal Policy Wrapper With Guarantees

Authors: Georgiy Malaniya, Anton Bolychev, Pavel Osinenko

Organizations: Center for Engineering Systems and Sciences, 30 Bolshoy Boulevard, bld. 1, Moscow, 121205, Russia · Central University, 7 Gasheka St., bld. 1, Moscow, 123056, Russia · Sirius University of Science and Technology, 1 Olimpiyskiy Avenue, Sirius, 354340, Russia

Abstract

Many control tasks require driving the system into a prescribed goal set and subsequently keeping it there. The considered setting assumes that a working controller is available that provably reaches and maintains the goal, albeit with suboptimal performance. This controller is referred to as the fallback policy. Conversely, although state-of-the-art reinforcement-learning (RL) algorithms can discover policies with impressive empirical performance, the neural networks at their core act as opaque black boxes learned through data-driven exploration and, as such, provide no theoretical mechanism to guarantee that the system will ever reach---and stay within---the prescribed goal set. This article presents CALF-Wrapper, a method that combines the fallback policy with a pretrained RL base policy and uses the critic estimate associated with the base policy to select between them during deployment. The theoretical analysis establishes that CALF-Wrapper inherits the goal-reaching property of the fallback policy under the stated assumptions and provides probability bounds for the settling time and the maximum distance from the goal set over a prescribed time interval. The experimental study evaluates CALF-Wrapper in four simulated control environments of different complexity. Comparisons include the base policy, fallback policy, and constrained reinforcement-learning methods formulated for the same objective. Sensitivity and ablation studies provide direct hyperparameter recommendations. The experiments show how CALF-Wrapper can provide effective task performance while attaining the maximum possible observed goal-reaching rate. CALF-Wrapper requires no additional dynamics model, online learning, or constrained optimization during execution.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. An Agency-Transferring Model-Free Policy Enhancement Technique

    Jun 8, 2026Anton Bolychev, Georgiy Malaniya, Sinan Ibrahim +1RL ControlTransfer Learning

  2. SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning

    Apr 10, 2026Maksim Anisimov, Francesco Belardinelli, Matthew WickerSafe RLContinual RL