Offline RL

RL: Reinforcement Learning

Ask about this topic

Overview

Offline reinforcement learning studies how to learn policies from previously collected transitions or demonstrations, which is useful when online trial and error is costly, unsafe, or unavailable. Value-based methods such as Conservative Q-Learning and Implicit Q-Learning, behavior-regularized policy optimization, learned world models, and trajectory or sequence models are common approaches; goal-conditioned and generative methods address long-horizon tasks and complex action distributions. Methods are often compared on benchmarks such as D4RL and OGBench, alongside applications in robotics, healthcare, and control. The central tension is improving beyond the logged behavior while avoiding unreliable value estimates and unsupported actions, so data coverage, uncertainty, and trustworthy evaluation remain key challenges.

Latest papers

CardsList

304 papers48 papers in the last four weeks, up 66% on the four weeks before. 0.3% of all new papers.

  1. Escaping Path Mirages in Offline Goal-Conditioned Reinforcement Learning

    Oct 29, 2026Junhyeon Bae, Jaebak Hwang, Gwanwoo Choi +2Offline RLGoal-Conditioned RL

    Graduate School of Artificial Intelligence, UNIST, Ulsan, South Korea

  2. Generating Critic-Guided Synthetic Experience in Latent World Models

    Oct 29, 2026Jorge de Freitas, Christos Ziakas, Alessandra RussoModel-Based RLOffline RL

    Imperial College London

  3. Transportable Offline Reinforcement Learning in Presence of Unmeasured Confounding

    Oct 28, 2026Junyi Liao, Hangqi Ren, Zhuoran YangOffline RLCausal Inference with Unobserved Confounding

    Department of Electrical and Computer Engineering, Duke University · Department of Electrical and Computer Engineering, Vanderbilt University · Department of Statistics and Data Science, Yale University

  4. Variational Consequence-Driven Offline Reinforcement Learning

    Oct 28, 2026Wen Jiang, Ke Jiang, Xiaoyang TanOffline RLConstrained RL

    College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence · Graduate School of Information Science and Technology, The University of Osaka

  5. CORMA: Constraint-Routed Decision Mamba for Generalizable Multi-Constraint Offline Safe Reinforcement Learning

    Oct 25, 2026Changxu Ni, Feiran You, Yongan Guo +5Safe RLReinforcement Learning

    Southeast University · The University of Hong Kong · Nanjing University of Posts and Telecommunications

  6. Conditional Flow Models for Offline Maximum Entropy Reinforcement Learning

    Oct 23, 2026Hao Cheng, Yuling Jiao, Lican Kang +2Offline RLMaximum Entropy RL

    School of Mathematics and Statistics, Wuhan University, Wuhan, 430072, China. · School of Artificial Intelligence, and Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, 430072, China. · Institute for Math and AI, Hubei Key Laboratory of Computational Science, and School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China. +1

  7. MA-UITD: Offline Multi-Agent Reward Inference for Characterizing Continuous Social Behavior

    Oct 21, 2026Zeyuan Zhang, Yiyun Yang, Yuan Cheng +2Reinforcement LearningOffline RL

    Institute of Neuroscience, Chinese Academy of Sciences · Nankai University · Icahn School of Medicine at Mount Sinai

  8. Value-Rectified Distillation for Flow-based Offline Reinforcement Learning

    Oct 21, 2026Ke Jiang, Wen Jiang, Yoshinobu Kawahara +1Offline RLPolicy Distillation

    College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence · Graduate School of Information Science and Technology, The University of Osaka +1

  9. Offline Apprenticeship Learning for Human-Centric Environments

    Oct 20, 2026Md Mirajul IslamOffline RLReinforcement Learning

  10. AWLS: Advantage-Weighted Latent Steering for One-Step Generative Robotic Manipulation Policies

    Oct 20, 2026Jiahong Tang, Jiaju He, Zhiyuan Yang +2Robot Policy LearningRobotic Manipulation

    Nanyang Technological University, Singapore · Singapore Institute of Manufacturing Technology (SIMTech), Agency for Science, Technology and Research (A*STAR), Singapore

  11. Ask to Learn: Query-Based Collaborative Reinforcement Learning

    Oct 20, 2026Kaicheng Zhang, Hongqian Niu, Laixi Shi +3Reinforcement LearningSample-Efficient RL

    University of North Carolina at Chapel Hill · Johns Hopkins University · Carnegie Mellon University

  12. VPFlow: Value-Predictive Planning with Imperfect World Models for Offline Goal Reaching

    Oct 19, 2026Yunzhe Hu, Dong XuOffline RLModel-Based Planning

    School of Computing and Data Science, The University of Hong Kong

  13. Next Best Action: Causal Constrained Offline RL for Business Action Targeting

    Oct 19, 2026Surya Ramachandiran, Yiming Li, Juan Mancilla-Caceres +2Offline RLConstrained RL

    Lyft, Inc., San Francisco, CA, USA

  14. Anatomy of Off-Policy Policy Gradient: Importance Sampling, KL Regularization, and Baselines

    Oct 19, 2026Haoqun Cao, Yurun Yuan, Tengyang XiePolicy Gradient MethodsOff-Policy RL

    University of Wisconsin–Madison

  15. ClinSDT: LLM-Encoded Clinical Semantic Guidance for Sepsis Treatment via Decision Transformer

    Oct 18, 2026Yanyan Wu, Guanyu Zhu, Changbin Zhao +7HealthcareClinical Decision-Making

    South China Agricultural University, Guangzhou, China · The Ohio State University, Columbus, Ohio, United States

  16. Generative Policy Improvement from Observational Data

    Oct 16, 2026Jonas Schweisthal, Dennis Frauen, Stefan Feuerriegel +1Offline RLRL for Generative Models

    LMU Munich · Munich Center for Machine Learning (MCML) · Department of Computer Science, Stanford University

  17. Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents

    Oct 15, 2026Hanzhen Wang, Boxiang Lyu, Zhongxiang Dai +3Reinforcement LearningRisk-Sensitive RL

    City University of Hong Kong Hong Kong, China · University of Chicago Chicago, IL, USA · The Chinese University of Hong Kong, Shenzhen Shenzhen, Guangdong, China +1

  18. Value-Guided Repulsion for Offline Reinforcement Learning

    Oct 14, 2026Yixue Hao, Jiahao Zhang, Jiesong Lian +5Offline RLPolicy Learning

    School of Computer Science and Technology, Huazhong University of Science and Technology

  19. Reinforcement Learning via Advantage Guidance

    Oct 14, 2026Zhiyuan Zhou, Sergey LevineReinforcement LearningOffline RL

    UC Berkeley

  20. Fast Rates for Offline Q-Learning with KL Regularization

    Oct 13, 2026Yize Hao, Guangxin Jiang, Shuang QiuOffline RLQ-Learning

    Harbin Institute of Technology. · City University of Hong Kong.

  21. From Suboptimal to Optimal: One-Step Latent Flow Policies for Offline Reinforcement Learning

    Oct 12, 2026Shenghua Lu, Zixuan Yang, Rongxu Zhao +3Offline RLFlow-Based Policy Learning

    Jilin University

  22. Recovering Low-Bit Failures in Vision-Language-Action Policies via Quantization-Conditioned Reinforcement Learning

    Oct 11, 2026Seongmin Park, Jaewon Lee, Yujun Heo +4VLM QuantizationEfficient VLA Model Inference

    Hanyang University, Republic of Korea · LG Electronics, Republic of Korea

  23. Counterfactual Rollouts for Offline Contrastive Goal-Conditioned Reinforcement Learning

    Oct 10, 2026Tingrui Huang, Ai Bo, Junzhe ZhangReinforcement LearningOffline RL

    Eindhoven University of Technology, Eindhoven, The Netherlands · Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY, USA

  24. Compositional Average Velocity Modeling for Efficient Diffusion Offline Reinforcement Learning

    Oct 10, 2026Thanh Nguyen, Abu Hanif Muhammad Syarubany, Hwanhee Kim +1Offline RLDiffusion Policy

    The authors are with the School of Electrical Engineering, KAIST, Daejeon, Republic of Korea.

  25. CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance

    Oct 10, 2026Shutong Ding, Zhaoyang Liu, Yimiao Zhou +3Diffusion PolicyRobotic RL

    ShanghaiTech University, Shanghai, China · InstAdapt, Shanghai, China

  26. OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models

    Oct 9, 2026Xule Gao, Xi Wang, Zehua Zang +3Vision-Language-Action ModelsRobotic Manipulation

    National Key Laboratory of Space Integrated Information System · Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences +1

  27. Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target

    Oct 9, 2026Du Cheng, Junyu Lu, Jingfan Zhang +3Policy OptimizationRL for Language Model Reasoning

  28. Higher-Order Action Supervision Makes A Strong Policy Class

    Oct 8, 2026Peng Cheng, Yunxian Hou, Zhi Zhou +3Policy LearningRobust RL

    Institute for AI Industry Research (AIR), Tsinghua University · Zenfex AI Lab · University of Electronic Science and Technology of China +1

  29. DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Tasks

    Oct 8, 2026Xinfei Wang, Shanchen Pang, Chenhao Zhang +5Reinforcement LearningOffline RL

  30. World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning

    Oct 7, 2026Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart +1Goal-Conditioned RLOffline RL

    University of Toronto · University of Toronto, Vector Institute