Offline RL

RL: Reinforcement Learning

Overview

Offline reinforcement learning studies how to learn policies from previously collected transitions or demonstrations, which is useful when online trial and error is costly, unsafe, or unavailable. Value-based methods such as Conservative Q-Learning and Implicit Q-Learning, behavior-regularized policy optimization, learned world models, and trajectory or sequence models are common approaches; goal-conditioned and generative methods address long-horizon tasks and complex action distributions. Methods are often compared on benchmarks such as D4RL and OGBench, alongside applications in robotics, healthcare, and control. The central tension is improving beyond the logged behavior while avoiding unreliable value estimates and unsupported actions, so data coverage, uncertainty, and trustworthy evaluation remain key challenges.

Latest papers

CardsList

328 papers33 papers in the last four weeks, down 30% on the four weeks before. 0.3% of all new papers.

  1. Latent Action Policies from Suboptimal Data

    Nov 18, 2026Donghyeon Ki, Xiaoyu Chen, Jun Ho Seo +4Latent Action LearningOffline RL

  2. Action-outcome coupling: a pre-investment feasibility audit for offline reinforcement learning in supply chain operations

    Nov 17, 2026Tung Dang, The Hung Phung, Son Lam Nguyen +1Offline RLAlgorithmic Auditing

  3. RIFT: Reward Improvement Field Transfer for Cross-Domain Offline RL

    Nov 15, 2026Jinhee Won, Mardavij Roozbehani, Munther DahlehOffline RLTransfer Learning in RL

  4. Support-Constrained Offline Policy Improvement for Pediatric ECMO from Limited Clinical Data

    Nov 12, 2026Fateme Golivand Darvishvand, Saurabh Mathur, Nikhilesh Prabhakar +6Fitted Q-IterationOffline RL

  5. Uncertainty-Aware Hybrid Offline Reinforcement Learning with Online Action Selection Enabled by Sparse Gaussian-Process Critics

    Nov 10, 2026Marco Chan, Christos Evripidou, Liqiu Dong +2Offline RLOffline-To-Online RL

  6. An Augmented Lagrangian Approach to Lifted Bellman Linear Programming

    Nov 10, 2026Jongchan Park, Hyukjun Yang, Narim Jeong +1Offline RLLinear Programming

  7. FaST: Factored Synthesis of Transitions for Offline Reinforcement Learning

    Nov 9, 2026Daniel Campa, Spandan Das, Srinjoy DasOffline RLSynthetic Data Augmentation

  8. Uncertainty-Guided Reward Labeling for Reinforcement Learning under Limited Feedback

    Nov 8, 2026Renhao Zhang, Shreyas Chaudhari, Bruno Castro da SilvaReinforcement LearningSequential Decision Making

  9. Horizon-Scalable Q-Learning

    Nov 7, 2026Haoran Xu, Pragya Srivastava, Xinjie Liu +2Q-LearningTemporal-Difference Learning

  10. Bayesian transfer for offline Q-learning with multi-task Gaussian processes

    Nov 7, 2026Wenxing GuoOffline RLTransfer Learning in RL

  11. Reinforcement Learning based control of Cancer Chemotherapy treatments

    Nov 6, 2026Jayakody Kankanamalage Chamani ShiranthikaRL ControlOncology

  12. Learning When to Think: Dual-Reference Offline Optimization for Adaptive VLM Reasoning

    Nov 6, 2026Hongbin Lin, Sizhe Zou, Juangui Xu +6Reasoning for VLMsVLM Reasoning

  13. GIST: Gibbs-Targeted Hierarchical Data Selection for Offline RLVR

    Nov 4, 2026Xuliang Huang, Tianxiang Zhao, Chengwei Qin +1Offline RLRL for Language Model Reasoning

  14. Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning

    Nov 2, 2026Xinbo Zhao, Yuhang Liu, Xin Zhang +3Meta-Reinforcement LearningOffline RL

  15. Value Forcing: Aligning Return Distributions at All Noise Levels

    Nov 1, 2026Chongyi Zheng, Zhiyuan Zhou, Qiyang Li +2Value Function EstimationSample-Efficient RL

  16. Direct Knowledge Transfer for Offline Q-Learning

    Nov 1, 2026Zhanye Luo, Guangyan Gan, Lin William CongFitted Q-IterationTransfer Learning in RL

  17. Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective

    Oct 30, 2026Yuxin Chen, Yuejie Chi, Jianqing Fan +3Sample-Efficient RLModel-Based RL

  18. Escaping Path Mirages in Offline Goal-Conditioned Reinforcement Learning

    Oct 29, 2026Junhyeon Bae, Jaebak Hwang, Gwanwoo Choi +2Offline RLGoal-Conditioned RL

  19. Generating Critic-Guided Synthetic Experience in Latent World Models

    Oct 29, 2026Jorge de Freitas, Christos Ziakas, Alessandra RussoModel-Based RLOffline RL

  20. Transportable Offline Reinforcement Learning in Presence of Unmeasured Confounding

    Oct 28, 2026Junyi Liao, Hangqi Ren, Zhuoran YangCausal TransportabilityOffline RL

  21. Variational Consequence-Driven Offline Reinforcement Learning

    Oct 28, 2026Wen Jiang, Ke Jiang, Jin Wang +1Offline RLConstrained RL

  22. CORMA: Constraint-Routed Decision Mamba for Generalizable Multi-Constraint Offline Safe Reinforcement Learning

    Oct 25, 2026Changxu Ni, Feiran You, Yongan Guo +5Safe RLReinforcement Learning

  23. Conditional Flow Models for Offline Maximum Entropy Reinforcement Learning

    Oct 23, 2026Hao Cheng, Yuling Jiao, Lican Kang +2Offline RLMaximum Entropy RL

  24. MA-UITD: Offline Multi-Agent Reward Inference for Characterizing Continuous Social Behavior

    Oct 21, 2026Zeyuan Zhang, Yiyun Yang, Yuan Cheng +2Reinforcement LearningOffline RL

  25. Value-Rectified Distillation for Flow-based Offline Reinforcement Learning

    Oct 21, 2026Ke Jiang, Wen Jiang, Yoshinobu Kawahara +1Offline RLPolicy Distillation