Unsupervised Reinforcement Learning

Unsupervised reinforcement learning (URL) aims to train agents to learn effective behaviors without relying on explicit reward signals, focusing instead on intrinsic motivations like exploration and skill discovery. Current research emphasizes developing algorithms that efficiently explore state spaces, often employing techniques like empowerment, entropy maximization/minimization, and curiosity-driven exploration, sometimes within model-based or ensemble frameworks. These advancements are significant because they promise more data-efficient and generalizable agents, potentially impacting robotics, personalized medicine (e.g., EEG analysis), and other fields requiring adaptable, self-improving systems.

Papers

October 21, 2024

Unsupervised Replay Strategies for Continual Learning with Limited Data
Anthony Bazhenov, Pahan Dewasurendra, Giri P. Krishnan, Jean Erik Delanois
Continual LEArning Limited Data Continuous Learning Memory Replay Unsupervised Reinforcement Learning Unsupervised Version Hebbian Learning Rule

October 10, 2024

Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning
Vassil Atanassov, Wanming Yu, Alexander Luis Mitchell, Mark Nicholas Finean, Ioannis Havoutis
Skill Discovery Latent Space Optimization Unsupervised Reinforcement Learning Quadruped Locomotion

August 27, 2024

Unsupervised-to-Online Reinforcement Learning
Junsu Kim, Seohong Park, Sergey Levine
Offline Reinforcement Learning Unsupervised Reinforcement Learning Offline Policy Learning Offline to Online Reinforcement Learning

August 22, 2024

Emotion-Agent: Unsupervised Deep Reinforcement Learning with Distribution-Prototype Reward for Continuous Emotional EEG Analysis
Zhihao Zhou, Qile Liu, Jiyuan Wang, Zhen Liang
Emotion Recognition Brain Computer Interface Brain Electroencephalography Unsupervised Reinforcement Learning

June 24, 2024

Uncertainty-Aware Reward-Free Exploration with General Function Approximation
Junkai Zhang, Weitong Zhang, Dongruo Zhou, Quanquan Gu
Reinforcement Learning Unsupervised Reinforcement Learning Reward Free Reinforcement Learning Reward Free Exploration

May 27, 2024

Surprise-Adaptive Intrinsic Motivation for Unsupervised Reinforcement Learning
Adriana Hugessen, Roger Creus Castanyer, Faisal Mohamed, Glen Berseth
Emergent Behavior Stable Entropy Unsupervised Reinforcement Learning Entanglement Entropy Entropy Based

May 25, 2024

Constrained Ensemble Exploration for Unsupervised Skill Discovery
Chenjia Bai, Rushuai Yang, Qiaosheng Zhang, Kang Xu, Yi Chen, Ting Xiao, Xuelong Li
Skill Discovery Unsupervised Reinforcement Learning Ensemble Based Reward Free Entropy Exploration

January 8, 2024

Curiosity & Entropy Driven Unsupervised RL in Multiple Environments
Shaurya Dewan, Anisha Jain, Zoe LaLena, Lifan Yu
Entropy Regularized Unsupervised Reinforcement Learning Curiosity Inducing Situation Curiosity Driven Exploration Alpha MEPOL Agnostic Exploration

December 14, 2023

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
Taewook Nam, Juyong Lee, Jesse Zhang, Sung Ju Hwang, Joseph J. Lim, Karl Pertsch
Vision Language Model Foundation Model Good Teacher Language Conditioned Skill Discovery Unsupervised Reinforcement Learning Floor Lift Reward Feedback Non Cognitive Skill

November 16, 2023

Augmenting Unsupervised Reinforcement Learning with Self-Reference
Andrew Zhao, Erle Zhu, Rui Lu, Matthieu Lin, Yong-Jin Liu, Gao Huang
Reinforcement Learning Benchmark Finetune Paradigm Unsupervised Reinforcement Learning Unsupervised Fine Tuning

October 13, 2023

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
Seohong Park, Oleh Rybkin, Sergey Levine
Reinforcement Learning Metric Aware Abstraction Unsupervised Reinforcement Learning Locomotion Behavior

September 29, 2023

CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning
Tianyu Li, Hyunyoung Jung, Matthew Gombolay, Yong Kwon Cho, Sehoon Ha
High Dimensional Legged Robot Human Motion Robot Motion Motion Control Unsupervised Reinforcement Learning

June 12, 2023

On the Amplification of Linguistic Bias through Unintentional Self-reinforcement Learning by Generative Language Models -- A Perspective
Minhyeok Lee
Human Language Visual Perspective Generative Language Model Linguistic Diversity Language Bias Unsupervised Reinforcement Learning Linguistic Landscape

June 6, 2023

Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning
Peggy Tang, Junbin Gao, Lei Zhang, Zhiyong Wang
High Efficiency Unsupervised Reinforcement Learning Compressive Summarisation

May 21, 2023

Unsupervised Discovery of Continuous Skills on a Sphere
Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka
Spherical Surface Unsupervised Discovery Unsupervised Reinforcement Learning Multi Agent MuJoCo Diverse Skill Latent Skill

February 21, 2023

Reinforcement Learning in a Birth and Death Process: Breaking the Dependence on the State Space
Jonatha Anselmi, Bruno Gaujal, Louis-Sébastien Rebuffi
Reinforcement Learning State Space Unsupervised Reinforcement Learning Activity Dependence

February 8, 2023

Predictable MDP Abstraction for Unsupervised Model-Based RL
Seohong Park, Sergey Levine
Model Based Reinforcement Learning Unsupervised Setting Unsupervised Reinforcement Learning Model Based Planning

January 31, 2023

CRC-RL: A Novel Visual Feature Representation Architecture for Unsupervised Reinforcement Learning
Darshita Jain, Anima Majumder, Samrat Dutta, Swagat Kumar
Reinforcement Learning Loss Function End to End Policy Learning Feature Representation Unsupervised Reinforcement Learning Action Description

October 19, 2022

Palm up: Playing in the Latent Manifold for Unsupervised Pretraining
Hao Liu, Tom Zahavy, Volodymyr Mnih, Satinder Singh
Reinforcement Learning Representation Learning Unsupervised Representation Learning Unsupervised Pre Training Latent Manifold Unsupervised Reinforcement Learning Palm ID

October 13, 2022

A Mixture of Surprises for Unsupervised Reinforcement Learning
Andrew Zhao, Matthieu Gaetan Lin, Yangguang Li, Yong-Jin Liu, Gao Huang
Mixture Component Intrinsic Reward Fast Adaptation Unsupervised Reinforcement Learning Reward Free Surprise Bound