Safe Offline RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

12 papers0 papers in the last four weeks, down 100% on the four weeks before. 0.0% of all new papers.

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning
Sep 28, 2026Shengchao Hu, Peng Wang, Jifeng Hu +5Reinforcement Learning Transformer-Based RL
Shanghai Jiao Tong University, China · Shenzhen Campus of Sun Yat-sen University, China · Jilin University, China +1
OneBid: A Unified Auto-Bidding Foundation Model for Diverse oCPX Advertising Scenarios
Sep 18, 2026Yewen Li, Peng Jiang, Yitian Li +4Safe Offline RL Automated Bidding
Kuaishou Technology Beijing, China
Certified Safety Curation: Distribution-Free Guarantees for Safe Offline Reinforcement Learning
Sep 10, 2026Adam Haroon, Cody FlemingSafe Offline RL Offline RL
Iowa State University Ames, IA, USA
Redistribution-based Cost Inference Improves Sparse Safe Offline RL
Aug 12, 2026Ebenezer Gelo, Geraud Nangue Tasse, Steven James +1Credit Assignment in RL Step-Level Credit Assignment in RL
University of the Witwatersrand
Individual Control Barrier Functions-Guided Diffusion Model for Safe Offline Multi-Agent Reinforcement Learning
Jun 10, 2026Qingyun Guo, Junyi Shi, Jianuo Huang +1Safe Offline RL Control Barrier Functions
Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland · School of Computing and Data Science, Xiamen University Malaysia, Sepang, Malaysia · Department of Computer Science, University of Toronto, Toronto, Canada
Robust Probabilistic Shielding for Safe Offline Reinforcement Learning
May 11, 2026Maris F. L. Galesloot, Thomas Rhemrev, Nils JansenOffline RL Safe RL
Radboud University · Nijmegen, The Netherlands · Ruhr University & Radboud University +1
Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learning
May 4, 2026Rufeng Chen, Zhaofan Zhang, Zhejiang Yang +2Diffusion-Based Planning Safe Offline RL
1The Hong Kong University of Science and Technology (Guangzhou) · 2Jilin University
Lyapunov-Guided Self-Alignment: Test-Time Adaptation for Offline Safe Reinforcement Learning
Apr 29, 2026Seungyub Han, Hyungjin Kim, Jungwoo LeeReinforcement Learning Offline RL
Seoul National University
Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies
Mar 16, 2026Mumuksh Tayal, Manan Tayal, Ravi PrakashFlow Matching Safe Offline RL
Indian Institute of Science, India
Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learning
Mar 8, 2026Janaka Chathuranga Brahmanage, Akshat KumarSafe RL Safe Offline RL
School of Computing and Information Systems, Singapore Management University.
LexiSafe: Offline Safe Reinforcement Learning with Lexicographic Safety-Reward Hierarchy
Feb 19, 2026Hsin-Jung Yang, Zhanhong Jiang, Prajwal Koirala +3Safe Offline RL Reinforcement Learning
Dept. of Mechanical Engineering Iowa State University Ames, USA · Translational AI Center Iowa State University Ames, USA · Sibley School of Mechanical and Aerospace Engineering Cornell University Ithaca, USA
EpiFlow: Bridging Safety and Performance in Offline Reinforcement Learning via Epigraph-Guided Flow Matching
Feb 8, 2026Manan Tayal, Mumuksh Tayal, Ravi PrakashSafe Offline RL Offline RL
TAU Intelligence

Sep 28, 2026J/K move · Enter open · S save

Q-learning Penalized Transformer for Safe Offline Reinforcement Learning

Shengchao Hu, Peng Wang, Jifeng Hu, Qiyang Zhou+4

Shanghai Jiao Tong University, China · Shenzhen Campus of Sun Yat-sen University, China · Jilin University, China+1

Reinforcement Learning Transformer-Based RL Safe Offline RL Constrained RL Offline RL

Open paper AskPDF arXiv

This paper addresses the problem of safe offline reinforcement learning, which involves training a policy to satisfy safety constraints using an offline dataset. This problem is inherently challenging as it requires balancing three highly interconnected and competing objectives: satisfying safety constraints, maximizing rewards, and adhering to the behavior regularization imposed by the offline dataset. To tackle this trilogy challenge, we propose Q-learning Penalized Transformer policy (QPT), a training--inference consistent framework that bridges conditional sequence modeling with constraint-aware value estimation. QPT trains a Transformer policy that generates actions conditioned on trajectory context and target return/cost, retaining strong behavior regularization. To inject explicit safety semantics during learning, we augment sequence-model training with a Q-shaped penalty using learned reward and cost Q-functions to favor high return under low constraint violation. At inference, the same Q-functions enforce the cost threshold and choose the highest-reward feasible action, closing the loop between training and deployment. We provide a principled analysis under stylized near-deterministic CMDPs, characterizing how Q-penalized conditional generation improve safety and performance. Empirically, QPT consistently outperforms strong safe offline RL baselines across 38 tasks on the DSRL benchmark, and exhibits robust zero-shot adaptation to different constraint thresholds.