Self Predictive

Self-predictive learning in reinforcement learning aims to learn effective representations by training models to predict their own future states or latent representations. Current research focuses on understanding why and when these methods succeed, analyzing the dynamics of different algorithms like BYOL and its variants (including action-conditional versions), and investigating the role of optimization techniques such as stop-gradient methods. This research is significant because improved representation learning is crucial for the performance of reinforcement learning agents, potentially leading to more efficient and robust AI systems across various applications.

Papers

June 25, 2024

When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning
Claas Voelcker, Tyler Kastner, Igor Gilitschenski, Amir-massoud Farahmand
Reinforcement Learning Learning Dynamic Auxiliary Task Auxiliary Learning Self Predictive

June 4, 2024

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires, Yunhao Tang, Clare Lyle, Mark Rowland, Nicolas Heess, Diana Borsa, Arthur Guez, Will Dabney
Reinforcement Learning Unifying Framework Self Predictive

January 17, 2024

Bridging State and History Representations: Understanding Self-Predictive RL
Tianwei Ni, Benjamin Eysenbach, Erfan Seyedsalehi, Michel Ma, Clement Gehring, Aditya Mahajan, Pierre-Luc Bacon
Deep Reinforcement Learning Markov Decision Process Bridging Text Abstraction Learning History Representation Self Predictive

February 9, 2023

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick
Pierre H. Richemond, Allison Tam, Yunhao Tang, Florian Strub, Bilal Piot, Felix Hill
Extreme Edge Best View Post Hoc Orthogonalization Linear Predictor Orthonormal Basis Prediction Network Self Predictive

December 6, 2022

Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Ávila Pires, Yash Chandak, Rémi Munos, Mark Rowland, Mohammad Gheshlaghi Azar, Charline Le Lan, Clare Lyle, András György, Shantanu Thakoor, Will Dabney, Bilal Piot, Daniele Calandriello, Michal Valko
Reinforcement Learning Learning Dynamic Bidirectional Training Self Predictive

Self Predictive

Papers

When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning

Bridging State and History Representations: Understanding Self-Predictive RL

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

Understanding Self-Predictive Learning for Reinforcement Learning