Offline RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 222% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 189

All topics

Higher-Order Action Supervision Makes A Strong Policy Class
Oct 8, 2026Peng Cheng, Yunxian Hou, Zhi Zhou +3Policy Learning Robust RL
Institute for AI Industry Research (AIR), Tsinghua University · Zenfex AI Lab · University of Electronic Science and Technology of China +1
DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Tasks
Oct 8, 2026Xinfei Wang, Shanchen Pang, Chenhao Zhang +5Reinforcement Learning Offline RL
World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning
Oct 7, 2026Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart +1Goal-Conditioned RL Offline RL
University of Toronto · University of Toronto, Vector Institute
Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving
Oct 7, 2026Mahmoud Selim, Cristina Cipriani, Karl Henrik JohanssonOffline RL Distribution Shift
KTH Royal Institute of Technology
An Informational Curse of Horizon in Goal-Conditioned Policy Learning
Oct 7, 2026John L. Zhou, Yuxuan Dong, Jonathan C. KaoGoal-Conditioned RL Generalization in RL
University of California, Los Angeles
LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
Oct 6, 2026Songyuan Zhang, Oswin So, Eric Yang Yu +3Reinforcement Learning Offline RL
MIT · MIT Lincoln Laboratory
Directed Temporal Representations for Offline Visual Control
Oct 6, 2026Chenyang Yuan, Haoyu Wang, Zhuo Sun +1Visual RL Trajectory Representation Learning
University of Sheffield · Shanghai Jiao Tong University · Shanghai University of Finance and Economics +1
Variance-Averse $n$ -Step Offline Reinforcement Learning for Sparse Long-Horizon Environments
Oct 6, 2026Guhyeon Kang, Minhae KwonDistributional RL Risk-Sensitive RL
Department of Electrical and Computer Engineering Sungkyunkwan University, Republic of Korea
Pessimistic Minimax Learning for Public-Private Information Games under Unilateral Coverage
Oct 4, 2026Shuze Daniel Liu, Claire Chen, Jiuqi Wang +1Imperfect-Information Games Zero-Sum Games
Massachusetts Institute of Technology · Purdue University · California Institute of Technology +1
Decision Titan: Test-Time Training for Long-Term Memory in Offline Reinforcement Learning
Oct 1, 2026Jude Waide, Robert LieckReinforcement Learning Transformer-Based RL
Department of Computer Science, University of Durham, Durham, United Kingdom.
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Oct 1, 2026Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira +1Reinforcement Learning Offline RL
Universidade Federal de Minas Gerais · University of Alberta · Alberta Machine Intelligence Institute (Amii) +1
Learning Goal-Reaching Quasimetric Geometry From Finite-Time Reachability
Sep 30, 2026Daisuke Yamada, Travis Pence, Vikas SinghMetric Learning Offline RL
Department of Computer Science University of Wisconsin–Madison · Department of Biostatistics and Medical Informatics University of Wisconsin–Madison
Role-Adaptive Policy Optimization for Offline Reinforcement Learning
Sep 30, 2026Seonvin Cho, Soohyun Choi, Songnam HongPolicy Optimization Offline RL
Department of Electronic Engineering, Hanyang University Seoul, Republic of Korea
Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
Sep 30, 2026Syed Nazmus Sakib, Abdul Monaf Chowdhury, Nafiul Haque +2Offline RL Representation Learning for RL
Department of Robotics and Mechatronics Engineering, University of Dhaka, Bangladesh · Department of Computer Science, University of Oxford, UK
In-Distribution Imagination for Model-Based Offline Reinforcement Learning
Sep 29, 2026Mintae Kim, Koushil SreenathOffline RL
Hybrid Robotics, BAIR, UC Berkeley
HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL
Sep 29, 2026JunHyeok Oh, Zian Jang, Byung-Jun LeeHierarchical Planning Trajectory Generation
Korea University. · Gauss Labs Inc., Seoul, Republic of Korea.
Diffusion Policy Improvement with Proposal-Conditioned Refinement Flows
Sep 29, 2026Junhyun Ha, Juho Lee, Byoungwoo ParkPolicy Optimization Offline RL
KAIST
Deep Weighted Bellman Residual Minimization for $Q^*$ Estimation
Sep 28, 2026Lican Kang, Jerry Zhijian Yang, Cheng Yuan +1Off-Policy Evaluation Value Function Estimation
Institute for Math and AI, Wuhan University, Wuhan, 430072, China · Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, 430072, China · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China +1
Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
Sep 28, 2026Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen +1Diffusion-Based Planning Hierarchical RL
School of Information and Control Engineering, China University of Mining and Technology · School of Computer Science and Engineering, South China University of Technology
QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement Learning
Sep 28, 2026Yuehu Gong, Shutong Ding, Mokai Pan +4Flow Matching MeanFlow
School of Data Science, Fudan University · ShanghaiTech University · Shanghai Innovation Institute
Q-learning Penalized Transformer for Safe Offline Reinforcement Learning
Sep 28, 2026Shengchao Hu, Peng Wang, Jifeng Hu +5Reinforcement Learning Transformer-Based RL
Shanghai Jiao Tong University, China · Shenzhen Campus of Sun Yat-sen University, China · Jilin University, China +1
GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences
Sep 27, 2026Abdul Monaf Chowdhury, MD Sameer Iqbal Chowdhury, Shifat E Arman +1Reinforcement Learning Temporal-Difference Learning
Department of Robotics and Mechatronics Engineering, University of Dhaka, Bangladesh · Department of Computer Science, Texas State University, USA · Department of Computer Science, University of Oxford, UK
Trust Guided Decision Transformer
Sep 25, 2026Chainesh Gautam, Raghuram Bharadwaj Diddigi, Chandramouli Kamanchi +3Transformer-Based RL Decision Transformer
International Institute of Information Technology Bangalore · IBM Research Bangalore
Learning from Mixed-Quality Deployment Experience for Robot Manipulation
Sep 24, 2026Yangang Ren, Yujie Yan, Zirui Li +6Robot Policy Learning Offline RL
School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 · Chongqing Changan Automobile Co., Ltd, Chongqing 400023, China · Guangzhou Cloudbutterfly Technology Co., Ltd., Guangzhou 510220, China
Offline Reinforcement Learning for Distribution-Grid Protection
Sep 21, 2026Julian Oelhaf, Alexander Luce, Christian Bergler +2Fault Detection Offline RL
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen–Nürnberg, Erlangen, Germany · Department of Electrical Engineering, Media and Computer Science, Ostbayerische Technische Hochschule Amberg-Weiden, Amberg, Germany
Lifted Bellman Linear Programming for Offline Reinforcement Learning
Sep 21, 2026Hyukjun Yang, Jongchan Park, Narim Jeong +1Reinforcement Learning Offline RL
Korea Advanced Institute of Science and Technology(KAIST) · NAVER LABS
Learning Reliable Parking Policies via Offline Reinforcement Learning with Quantized Action Representations
Sep 17, 2026Zewei Yang, Zengqi Peng, Jun MaAutonomous Parking Offline RL
Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR, China
Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulation
Sep 16, 2026Jing ZhangReward Shaping Offline RL
The Hong Kong University of Science and Technology
Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport
Sep 14, 2026Jaehun Shon, Jinha Choi, Jongwook Jeon +1Offline RL
Department of Artificial Intelligence, Yonsei University
Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs
Sep 14, 2026Abhinav Anand, Sanjana Reddy Pachika, Shweta Verma +1RL for Code Generation Offline RL
TU Darmstadt · Hessian Center for Artificial Intelligence, Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE

Oct 8, 2026, cs.ROJ/K move · Enter open · S save

Higher-Order Action Supervision Makes A Strong Policy Class

Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang+2

Institute for AI Industry Research (AIR), Tsinghua University · Zenfex AI Lab · University of Electronic Science and Technology of China+1

Policy Learning Robust RL Offline RL

Open paper AskPDF arXiv

Modern data-driven decision-making methods, such as imitation learning (IL) and reinforcement learning (RL), have achieved great success in solving many complex tasks. However, these methods often suffer from serious control instability and robustness issues when applied in real-world applications such as robotics and autonomous driving, posing notable challenges for their practical deployment. We argue that this instability issue stems largely from their limitations in solely supervising and optimizing zeroth-order actions (i.e., the action labels), failing to account for higher-order action dynamics and temporal consistency. In this paper, we show that simultaneously supervising both zeroth- and first-order actions can dramatically enhance policies' performance and control robustness. To achieve this, we introduce a novel and elegant loss scheme supported by formal theoretical guarantees that can equip any off-the-shelf policy model (e.g., deterministic, stochastic, or flow policies) with the capability for higher-order action supervision, without requiring any structural modifications. Moreover, our proposed method can serve as a lightweight plug-and-play module that seamlessly integrates with a broad spectrum of existing offline RL frameworks. Extensive evaluations on OGBench and D4RL demonstrate that our approach yields substantial performance and robustness improvements across a wide range of continuous control environments. Notably, our method can also enhance policies' out-of-distribution (OOD) generalization capability in the challenging low-data regime, making it an ideal tool in tackling many real-world control problems.

Momentum

Related topics

Latest papers 189