Implicit Q Learning

Implicit Q-learning (IQL) is an offline reinforcement learning algorithm that learns a value function from a fixed dataset of experiences, focusing on efficiently utilizing limited data and handling out-of-distribution actions. Current research emphasizes improving IQL's policy extraction methods, addressing its limitations in handling sparse rewards and noisy data, and exploring its application in diverse domains like robotics and natural language processing. These advancements aim to enhance the robustness and sample efficiency of IQL, making it a more practical and powerful tool for real-world applications where collecting extensive data is costly or impossible.

Papers

October 15, 2024

DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation
Jaehyun Park, Yunho Kim, Sejin Kim, Byung-Jun Lee, Sundong Kim
Offline Reinforcement Learning Adaptive Importance Sparse Reward Environment Adaptive Decision Making Implicit Q Learning

October 10, 2024

VerifierQ: Enhancing LLM Test Time Compute with Q-Learning-based Verifiers
Jianing Qi, Hao Tang, Zhigang Zhu
Reinforcement Learning Q Learning Implicit Q Learning

June 20, 2024

Equivariant Offline Reinforcement Learning
Arsh Tangri, Ondrej Biza, Dian Wang, David Klee, Owen Howell, Robert Platt
Online Reinforcement Learning Conservative Q Learning Robot Policy Learning Implicit Q Learning

May 28, 2024

October 19, 2023

Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, Tong Zhang
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Robust Offline Reinforcement Learning Data Corruption Implicit Q Learning

June 1, 2023

IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control
Rohan Chitnis, Yingchen Xu, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu, Olivier Delalleau
Model Based Reinforcement Learning Implicit Language Td MPC Implicit Q Learning Based Offline RL

April 20, 2023

IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies
Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, Sergey Levine
Actor Critic Diffusion Policy Implicit Language Semantic Actor Offline RL Implicit Q Learning

June 5, 2022

Offline RL for Natural Language Generation with Implicit Language Q Learning
Charlie Snell, Ilya Kostrikov, Yi Su, Mengjiao Yang, Sergey Levine
Large Language Model Language Model Reinforcement Learning Language Generation Language Model Generation Implicit Q Learning