State Action Pair

State-action pairs represent the fundamental units of interaction in reinforcement learning (RL), encompassing the agent's state and chosen action at a given time step. Current research focuses on improving the representation and utilization of state-action pairs, particularly through advanced model architectures like diffusion models and ensemble methods, and algorithms such as those employing bisimulation or contrastive learning to enhance sample efficiency and robustness. This work is crucial for advancing RL's capabilities in complex environments, particularly in offline settings and safety-critical applications, by enabling more accurate value function estimation, improved policy learning, and better generalization.

Papers

June 20, 2023

The Unintended Consequences of Discount Regularization: Improving Regularization in Certainty Equivalence Reinforcement Learning
Sarah Rathnam, Sonali Parbhoo, Weiwei Pan, Susan A. Murphy, Finale Doshi-Velez
Optimal Policy State Action Pair Certainty Equivalent Regularization Penalty

May 24, 2023

Matrix Estimation for Offline Reinforcement Learning with Low-Rank Structure
Xumei Xi, Christina Lee Yu, Yudong Chen
Offline Reinforcement Learning Inner Structure State Action Pair Offline Policy Evaluation Matrix Estimation

May 4, 2023

Masked Trajectory Models for Prediction, Representation, and Control
Philipp Wu, Arjun Majumdar, Kevin Stone, Yixin Lin, Igor Mordatch, Pieter Abbeel, Aravind Rajeswaran
Human Prediction External Control Individual Representation State Representation State Action Pair Offline RL Algorithm

April 4, 2023

Risk-Aware Distributed Multi-Agent Reinforcement Learning
Abdullah Al Maruf, Luyao Niu, Bhaskar Ramasubramanian, Andrew Clark, Radha Poovendran
Multi Agent Reinforcement Learning State Action Pair Risk Management Autonomous Cyber

March 15, 2023

On the Benefits of Leveraging Structural Information in Planning Over the Learned Model
Jiajun Shen, Kananart Kuwaranancharoen, Raid Ayoub, Pietro Mercati, Shreyas Sundaram
Task Planning Sample Complexity Model Based Reinforcement Learning Complementary Benefit Q Function State Action Pair Structural Information Learned Model Grid World

January 26, 2023

Model-based Offline Reinforcement Learning with Local Misspecification
Kefan Dong, Yannis Flet-Berliac, Allen Nie, Emma Brunskill
State Action Pair Model Based Offline Reinforcement Learning Model Misspecification Offline Policy Policy Selection

January 3, 2023

DADAgger: Disagreement-Augmented Dataset Aggregation
Akash Haridas, Karim Hamadeh, Samarendra Chandan Bindu Dash
Action Prediction State Action Pair Based Imitation Balanced Dataset Aggregated Data Random Baseline

December 30, 2022

Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
Trevor Ablett, Bryan Chan, Jonathan Kelly
LeArning Abstract Imitation Learning Multi Task Environment Exploration Online Reinforcement Learning State Action Pair Adversarial Imitation Learning Adversarial Imitation

December 29, 2022

An Instrumental Variable Approach to Confounded Off-Policy Evaluation
Yang Xu, Jin Zhu, Chengchun Shi, Shikai Luo, Rui Song
Markov Decision Process Policy Evaluation Instrumental Variable State Action Pair Single Stage

December 4, 2022

Automata Learning meets Shielding
Martin Tappler, Stefan Pranger, Bettina Könighofer, Edi Muškardin, Roderick Bloem, Kim Larsen
Reinforcement Learning Markov Decision Process Q Learning State Action Pair Learning Automaton

November 15, 2022

Explainable Action Advising for Multi-Agent Reinforcement Learning
Yue Guo, Joseph Campbell, Simon Stepputtis, Ruiyu Li, Dana Hughes, Fei Fang, Katia Sycara
Reinforcement Learning Multi Agent Reinforcement Learning State Action Pair Suggestion System Broad Persistent Advice

October 28, 2022

Using Contrastive Samples for Identifying and Leveraging Possible Causal Relationships in Reinforcement Learning
Harshad Khadilkar, Hardik Meisheri
Reinforcement Learning Causal Relationship Policy Reinforcement Learning State Action Pair Contrastive Data Proxy Based Contrastive Replay

September 28, 2022

Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees
Daniil Tiapkin, Denis Belomestny, Daniele Calandriello, Eric Moulines, Remi Munos, Alexey Naumov, Mark Rowland, Michal Valko, Pierre Menard
Reinforcement Learning Posterior Sampling State Action Pair Tight Guarantee

September 14, 2022

Using Forwards-Backwards Models to Approximate MDP Homomorphisms
Augustine N. Mavor-Parker, Matthew J. Sargent, Christian Pehle, Andrea Banino, Lewis D. Griffin, Caswell Barry
Action Space State Action Space State Action Pair MDP Model MDP Homomorphism

August 1, 2022

Safe Policy Improvement Approaches and their Limitations
Philipp Scholl, Felix Dietrich, Clemens Otte, Steffen Udluft
Offline Reinforcement Learning Fundamental Limitation State Action Pair Policy Improvement Safe Algorithm

June 2, 2022

May 23, 2022

Efficient Reinforcement Learning from Demonstration Using Local Ensemble and Reparameterization with Split and Merge of Expert Policies
Yu Wang, Fang Liu
Reinforcement Learning Policy Gradient Split Learning Textual Demonstration Efficient Reinforcement Learning State Action Pair Reparameterization Method Suboptimal Demonstration Expert Policy

March 22, 2022

Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos
Tomáš Souček, Jean-Baptiste Alayrac, Antoine Miech, Ivan Laptev, Josef Sivic
Self Supervised Past Action Pre Change Information State Action Pair Untrimmed Video Object State

March 17, 2022

Investigating Compounding Prediction Errors in Learned Dynamics Models
Nathan Lambert, Kristofer Pister, Roberto Calandra
Model Based Reinforcement Learning Robot Control State Action Pair Prediction Error Learned Dynamic Data Driven Control