Hindsight Instruction

Hindsight instruction leverages past experiences, often relabeling data or goals after an action sequence is completed, to improve the learning efficiency and performance of artificial intelligence agents, particularly in reinforcement learning and large language models. Current research focuses on developing algorithms that effectively utilize this hindsight information, including adaptations of reinforcement learning methods like Decision Transformers and the exploration of novel architectures for handling diverse data sources and complex tasks. This approach holds significant promise for enhancing the sample efficiency of training AI agents, leading to improved performance in robotics, natural language processing, and other domains where obtaining labeled data is costly or difficult.

Papers

November 12, 2024

Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPs
Chao Han, Debabrota Basu, Michael Mangan, Eleni Vasilaki, Aditya Gilra
Markov Decision Process Hindsight Instruction Causal System Dynamical Variational Markovian Sequence

November 7, 2024

Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
Joey Hong, Jessica Lin, Anca Dragan, Sergey Levine
Reinforcement Learning Dialogue Agent Hindsight Instruction

August 12, 2024

Hierarchical in-Context Reinforcement Learning with Hindsight Modular Reflections for Planning
Chuanneng Sun, Songjun Huang, Dario Pompili
Reinforcement Learning Task Planning Hierarchical Reinforcement Learning Hindsight Instruction LLM Policy

May 29, 2024

AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data
Zifan Song, Yudong Wang, Wenwei Zhang, Kuikun Liu, Chengqi Lyu, Demin Song, Qipeng Guo, Hang Yan, Dahua Lin, Kai Chen, Cairong Zhao
Large Language Model Code Generation Open Source Large Language Model Multi Source Hindsight Instruction Version Control Eliciting Code Capability

April 14, 2024

April 12, 2024

Hindsight PRIORs for Reward Learning from Human Preferences
Mudit Verma, Katherine Metcalf
Reinforcement Learning Policy Learning Human Preference Reward Learning Hindsight Instruction Reward Engineering

February 28, 2024

Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation
Tonghe Zhang, Yu Chen, Longbo Huang
Reinforcement Learning High Efficiency Efficient Reinforcement Learning Observable Markov Decision Process Risk Sensitive Reinforcement Learning Hindsight Instruction

July 21, 2023

Hindsight-DICE: Stable Credit Assignment for Deep Reinforcement Learning
Akash Velu, Skanda Vaidyanath, Dilip Arumugam
Reinforcement Learning Deep Reinforcement Learning Policy Evaluation Reinforcement Learning Agent Hindsight Instruction

July 6, 2023

Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight
Jiacheng Guo, Minshuo Chen, Huan Wang, Caiming Xiong, Mengdi Wang, Yu Bai
Reinforcement Learning Markov Decision Process Sample Efficient Partially Observable Markov Decision Process Hindsight Instruction

June 26, 2023

CEIL: Generalized Contextual Imitation Learning
Jinxin Liu, Li He, Yachen Kang, Zifeng Zhuang, Donglin Wang, Huazhe Xu
Imitation Learning Feature Imitation Communication Efficient Hindsight Instruction Expert Retrieval

May 26, 2023

Emergent Agentic Transformer from Chain of Hindsight Experience
Hao Liu, Pieter Abbeel
Reinforcement Learning Transformer Based Model Side Chain Hindsight Instruction Transformer Based Policy Transformer Agent

March 3, 2023

Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning
Simon Guist, Jan Schneider, Alexander Dittrich, Vincent Berenz, Bernhard Schölkopf, Dieter Büchler
Reinforcement Learning Robotics Domain Soft Robot Efficient Reinforcement Learning Real World Task Hindsight Instruction

February 10, 2023

November 18, 2022

Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments
Daniel Jarrett, Corentin Tallec, Florent Altché, Thomas Mesnard, Rémi Munos, Michal Valko
Sparse Reward Intrinsic Reward Stochastic Environment Curiosity Inducing Situation Hindsight Instruction Reward Free Intrinsic Exploration Curiosity Driven Learning

October 17, 2022

Symbol Guided Hindsight Priors for Reward Learning from Human Preferences
Mudit Verma, Katherine Metcalf
Reward Function Preference Feedback Human Preference Reward Learning Hindsight Instruction

September 26, 2022

Understanding Hindsight Goal Relabeling from a Divergence Minimization Perspective
Lunjun Zhang, Bradly C. Stadie
Imitation Learning Suboptimal Demonstration Hindsight Instruction Multi Goal Reinforcement Learning Divergence Optimization Hindsight Goal Relabeling

July 13, 2022

Hindsight Learning for MDPs with Exogenous Inputs
Sean R. Sinclair, Felipe Frujeri, Ching-An Cheng, Luke Marshall, Hugo Barbalho, Jingling Li, Jennifer Neville, Ishai Menache, Adith Swaminathan
Markov Decision Process Sequential Decision Hindsight Instruction Domain Specific Heuristic Exogenous Variable

May 24, 2022

Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections
Dustin Morrill, Ryan D'Orazio, Marc Lanctot, James R. Wright, Michael Bowling, Amy R. Greenwald
LeArning Abstract Video Game Language Correction Regret Minimization Hindsight Instruction Least Absolute Deviation Regret Dynamic Counterfactual Regret Minimization