Contrastive Reinforcement Learning

Contrastive reinforcement learning (CRL) aims to improve reinforcement learning agents' ability to learn effectively from limited data and complex environments by leveraging contrastive learning techniques. Current research focuses on developing stable and efficient CRL algorithms, often incorporating infoNCE objectives or modifications of policy gradient methods, and exploring their application in diverse domains such as robotics, recommendation systems, and large language model fine-tuning. This approach shows promise in enhancing sample efficiency and enabling the learning of complex behaviors from limited rewards or even without explicit rewards, potentially leading to more robust and adaptable AI systems across various applications.

Papers

August 20, 2024

Accelerating Goal-Conditioned RL Algorithms and Research
Michał Bortkiewicz, Władek Pałucki, Vivek Myers, Tadeusz Dziarmaga, Tomasz Arczewski, Łukasz Kuciński, Benjamin Eysenbach
Reinforcement Learning Reinforcement Learning Algorithm DH Research Self Supervision Goal Conditioned Reinforcement Learning Contrastive Reinforcement Learning

August 11, 2024

A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals
Grace Liu, Michael Tang, Benjamin Eysenbach
Environment Exploration Reward Function Manipulation Task Noisy Demonstration Active Exploration Pseudo Goal Robust Skill Contrastive Reinforcement Learning RL Algorithm

June 27, 2024

Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashion
Yannis Flet-Berliac, Nathan Grinsztajn, Florian Strub, Eugene Choi, Chris Cremer, Arash Ahmadian, Yash Chandak, Mohammad Gheshlaghi Azar, Olivier Pietquin, Matthieu Geist
Large Language Model Reinforcement Learning Policy Gradient Contrastive Reinforcement Learning Policy Policy Gradient

October 25, 2023

Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation
Chengpeng Li, Zhengyi Yang, Jizhi Zhang, Jiancan Wu, Dingxian Wang, Xiangnan He, Xiang Wang
Reinforcement Learning Sequential Recommendation Contrastive Reinforcement Learning

October 2, 2023

All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization
Pablo Barros, Alessandra Sciutti
Cooperative Game Contrastive Reinforcement Learning Diverse Opponent

June 6, 2023

Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data
Chongyi Zheng, Benjamin Eysenbach, Homer Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, Sergey Levine
Reinforcement Learning Self Supervised Learning Self Supervised Barzilai Borwein Technique Offline Data Self Supervised Reinforcement Learning Robot Goal Contrastive Reinforcement Learning

October 14, 2022

Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning
Louis Castricato, Alexander Havrilla, Shahbuland Matiana, Michael Pieler, Anbang Ye, Ian Yang, Spencer Frazier, Mark Riedl
Story Generation Narrative Text Contrastive Reinforcement Learning Robust Preference Contrastive Reward

June 17, 2022

CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer
Yao Mu, Shoufa Chen, Mingyu Ding, Jianyu Chen, Runjian Chen, Ping Luo
Transformer Based State Representation Visual Control Contrastive Reinforcement Learning

June 15, 2022

Contrastive Learning as Goal-Conditioned Reinforcement Learning
Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, Sergey Levine
Reinforcement Learning Contrastive Learning Goal Conditioned Reinforcement Learning Contrastive Reinforcement Learning

June 10, 2022

Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
Xiang Li, Jinghuan Shang, Srijan Das, Michael S. Ryoo
Reinforcement Learning Self Supervised Learning Tetromino Pixel Online Reinforcement Learning Self Supervised Loss Contrastive Reinforcement Learning

Contrastive Reinforcement Learning

Papers

Accelerating Goal-Conditioned RL Algorithms and Research

A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals

Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashion

Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation

All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization

Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data

Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning

CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer

Contrastive Learning as Goal-Conditioned Reinforcement Learning

Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?