Offline RL

RL: Reinforcement Learning

Ask about this topic

Overview

Offline reinforcement learning studies how to learn policies from previously collected transitions or demonstrations, which is useful when online trial and error is costly, unsafe, or unavailable. Value-based methods such as Conservative Q-Learning and Implicit Q-Learning, behavior-regularized policy optimization, learned world models, and trajectory or sequence models are common approaches; goal-conditioned and generative methods address long-horizon tasks and complex action distributions. Methods are often compared on benchmarks such as D4RL and OGBench, alongside applications in robotics, healthcare, and control. The central tension is improving beyond the logged behavior while avoiding unreliable value estimates and unsupported actions, so data coverage, uncertainty, and trustworthy evaluation remain key challenges.

Latest papers

CardsList

304 papers48 papers in the last four weeks, up 66% on the four weeks before. 0.3% of all new papers.

  1. GUI Agents with Reinforcement Learning: Toward Digital Inhabitants

    Apr 30, 2026Junan Hu, Jian Liu, Jingxiang Lai +6Reinforcement LearningRL for GUI Agents

    1Shandong University · 2The Hong Kong University of Science and Technology · 3The Hong Kong University +2

  2. Lyapunov-Guided Self-Alignment: Test-Time Adaptation for Offline Safe Reinforcement Learning

    Apr 29, 2026Seungyub Han, Hyungjin Kim, Jungwoo LeeReinforcement LearningOffline RL

    Seoul National University

  3. TSN-Affinity: Similarity-Driven Parameter Reuse for Continual Offline Reinforcement Learning

    Apr 28, 2026Dominik Żurek, Kamil Faber, Marcin Pietron +2Continual LearningRobotic RL

    AGH University of Krakow, Poland · American University, Washington, DC, USA

  4. Improving Zero-Shot Offline RL via Behavioral Task Sampling

    Apr 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudZero-Shot RLOffline RL

    Institut des Systèmes Intelligents et de Robotique, ISIR, Sorbonne Université, Paris, France.

  5. DLM: Unified Decision Language Models for Offline Multi-Agent Sequential Decision Making

    Apr 26, 2026Zhuohui Zhang, Bin Cheng, Bin HeSequential MARLOffline RL

    Department of Control Science and Engineering, Tongji University, Shanghai 201804, China · Shanghai Institute of Intelligent Science and Technology; Shanghai Research Institute for Intelligent Autonomous Systems; State Key Laboratory of Autonomous Intelligent Unmanned Systems, Shanghai 200120, China

  6. Learning from Demonstration with Failure Awareness for Safe Robot Navigation

    Apr 25, 2026Xianghui Wang, Siwei Cheng, Shanze Wang +3Safe Robot NavigationOffline RL

    Department of Mechanical Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong · College of Information Science and Technology, Eastern Institute of Technology, Ningbo, China · School of Computer Science and Technology, University of Science and Technology of China, Hefei, China +1

  7. CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning

    Apr 25, 2026Marcel Hedman, Kale-ab Abebe Tessera, Juan Claude Formanek +5Multi-Agent CoordinationOffline RL

    Inephany Ltd · Department of Statistics, University of Oxford · The University of Edinburgh +2

  8. SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning

    Apr 24, 2026Jichao Wang, Liuyang Bian, Yufeng Zhou +9RL for GUI AgentsCredit Assignment in RL

    1vivo AI Lab · 2Zhejiang Lab · 5Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences +2

  9. Preserve Support, Not Correspondence: Dynamic Routing for Offline Reinforcement Learning

    Apr 24, 2026Zhancun Mu, Guangyu Zhao, Yiwu Zhong +1Reinforcement LearningOffline RL

    School of Intelligence Science and Technology, Peking University

  10. When Policies Cannot Be Retrained: A Unified Closed-Form View of Post-Training Steering in Offline Reinforcement Learning

    Apr 23, 2026Elias Hossain, Mohammad Jahid Ibna Basher, Ivan Garibay +2Offline RLRL Post-Training

    College of Engineering and Computer Science University of Central Florida, Orlando, FL, USA

  11. Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness

    Apr 23, 2026Zihan Liang, Ziwen Pan, Ruoxuan XiongClinical Outcome PredictionOffline RL

    Emory University, Atlanta, USA

  12. Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning

    Apr 22, 2026Aravind Venugopal, Jiayu Chen, Xudong Wu +3Reward ShapingTemporal Credit Assignment

    Carnegie Mellon University · The University of Hong Kong · INFIFORCE Intelligent Technology +1

  13. Lever: Inference-Time Policy Reuse under Support Constraints

    Apr 22, 2026Ihor Vitenko, Noha Ibrahim, Sihem Amer-YahiaOffline RLPolicy Learning

    Anhalt University of Applied Sciences Germany · Université Grenoble Alpes Grenoble, France · CNRS, Univ. Grenoble Alpes Grenoble, France

  14. Fisher Decorator: Refining Flow Policy via a Local Transport Map

    Apr 20, 2026Xiaoyuan Cheng, Haoyu Wang, Wenxuan Yuan +4Flow MatchingOffline RL

    University College London · Shanghai Jiao Tong University · Imperial College London +3

  15. Reinforcement Learning via Value Gradient Flow

    Apr 15, 2026Haoran Xu, Kaiwen Hu, Somayeh Sojoudi +1Regularized RLReinforcement Learning

    University of Texas at Austin · University of California, Berkeley

  16. Pareto-Optimal Offline Reinforcement Learning via Smooth Tchebycheff Scalarization

    Apr 14, 2026Aadyot Bhatnagar, Peter Mørch Groth, Sebastian Ibarraran +1Multi-Objective Reinforcement LearningOffline RL

    Profluent Bio, Inc. · Stanford University

  17. Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers

    Apr 14, 2026Snehal Jauhri, Vignesh Prasad, Georgia ChalvatzakiMobile ManipulationWhole-Body Robotic Manipulation

    PEARL Lab, Computer Science Department, Technische Universität Darmstadt, Germany · Hessian.AI, Darmstadt, Germany · Robotics Institute, Germany

  18. Efficient Hierarchical Implicit Flow Q-learning for Offline Goal-conditioned Reinforcement Learning

    Apr 10, 2026Zhiqiang Dong, Teng Pang, Rongjian Xu +1Offline RLHierarchical RL

    School of Software, Shandong University, Jinan, Shandong, China · Luke EI, Jinan, China

  19. Epistemic Robust Offline Reinforcement Learning

    Apr 8, 2026Abhilash Reddy Chenreddy, Erick DelageUncertainty Estimation in RLOffline RL

    GERAD, Department of Decision Sciences, HEC Montréal, Montréal, Québec H3T 2A7, Canada

  20. Beyond Pessimism: Offline Learning in KL-regularized Games

    Apr 8, 2026Yuheng Zhang, Claire Chen, Nan JiangKL-Regularized RLZero-Sum Games

    University of Illinois Urbana-Champaign. · California Institute of Technology.

  21. Cross-fitted Proximal Learning for Model-Based Reinforcement Learning

    Apr 6, 2026Nishanth Venkatesh, Andreas A. MalikopoulosCross-FittingReinforcement Learning

    Department of Systems Engineering, Cornell University, Ithaca, NY, USA · Applied Mathematics, Systems Engineering, Mechanical Engineering, Electrical & Computer Engineering, and School of Civil & Environmental Engineering, Cornell University, Ithaca, NY, USA

  22. Offline RL for Adaptive Policy Retrieval in Prior Authorization

    Apr 6, 2026Ruslan Sharifullin, Maxim Gorshkov, Hannah ClayAdaptive RetrievalRL for IR

    Stanford University.

  23. ReinVBC: A Model-based Reinforcement Learning Approach to Vehicle Braking Controller

    Apr 6, 2026Haoxin Lin, Junjie Zhou, Daheng Xu +1RL ControlModel-Based RL

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · Polixir Technologies, Nanjing, China

  24. Mitigating Data Scarcity in Spaceflight Applications for Offline Reinforcement Learning Using Physics-Informed Deep Generative Models

    Apr 2, 2026Alex E. Ballentine, Nachiket U. Bapat, Raghvendra V. CowlagiPhysics-Informed Generative ModelingVariational Autoencoders

    Graduate Research Assistant, Aerospace Engineering Department, Worcester Polytechnic Institute, MA. · Associate Professor, Aerospace Engineering Department. AIAA Senior Member. Corresponding Author.

  25. Residuals-based Offline Reinforcement Learning

    Apr 1, 2026Qing Zhu, Xian YuOffline RLDeep Q-Learning

    Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH, USA

  26. GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetes

    Apr 1, 2026Saman Khamesian, Sri Harini Balaji, Di Yang Shi +5Reinforcement LearningType 1 Diabetes

    College of Health Solutions, Arizona State University, Phoenix, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, USA · School for Engineering of Matter Transport and Energy, Arizona State University, Tempe, USA +2

  27. Offline Constrained RLHF with Multiple Preference Oracles

    Mar 31, 2026Brenden Latham, Mehrdad MoharramiConstrained RLReinforcement Learning

    Computer Science Department, University of Iowa, Iowa City, IA, USA

  28. GraSP-STL: A Graph-Based Framework for Zero-Shot Signal Temporal Logic Planning via Offline Goal-Conditioned Reinforcement Learning

    Mar 31, 2026Ancheng Hou, Ruijia Liu, Xiang YinSTL PlanningGraph-Based Planning

    School of Automation & Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China.

  29. A Pontryagin Method of Model-based Reinforcement Learning via Hamiltonian Actor-Critic

    Mar 30, 2026Chengyang Gu, Yuxin Pan, Hui Xiong +1Model-Based RLActor-Critic Methods

    Information Hub, HKUST (Guangzhou), Guangzhou, China · the Department of Computer Science, City University of Hong Kong, Hong Kong, China · the Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada

  30. Functional Natural Policy Gradients

    Mar 30, 2026Aurelien Bibaut, Houssam Zenati, Thibaud Rahier +1Cross-FittingNatural Policy Gradient