Predictive State Representation

Predictive State Representations (PSRs) aim to model dynamical systems, particularly partially observable ones, by representing states based on predictions of future observations. Current research focuses on developing computationally efficient algorithms, such as UCB-type approaches and singular value decomposition methods, to learn these representations, often within the context of reinforcement learning. This work is significant because it enables more robust and efficient decision-making in complex environments with incomplete information, impacting fields like robotics and control systems through improved model-based learning and parameter estimation from visual data.

Papers

July 1, 2023

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations
Ruiquan Huang, Yingbin Liang, Jing Yang
Markov Decision Process Sequential Decision Making Problem Partially Observable Markov Decision Process Near Optimal Policy Upper Confidence Bound Predictive State Representation

May 1, 2023

Representations and Exploration for Deep Reinforcement Learning using Singular Value Decomposition
Yash Chandak, Shantanu Thakoor, Zhaohan Daniel Guo, Yunhao Tang, Remi Munos, Will Dabney, Diana L Borsa
Deep Reinforcement Learning Representation Learning Meaningful Representation Environment Exploration Singular Value Decomposition Deep Reinforcement Learning Agent Exploration Problem Predictive State Representation

September 13, 2022

Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory
Sridhar Mahadevan
Reinforcement Learning Causal Inference Causal Model Predictive State Representation

July 12, 2022

PAC Reinforcement Learning for Predictive State Representations
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
Markov Decision Process Online Reinforcement Learning C POMDPs Thermodynamic Observables Predictive State Representation

May 11, 2022

RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation
Pingchuan Ma, Tao Du, Joshua B. Tenenbaum, Wojciech Matusik, Chuang Gan
Imitation Learning Cross Domain Differentiable Rendering Differentiable Simulation Dynamic Motion Predictive State Representation

Predictive State Representation

Papers

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations

Representations and Exploration for Deep Reinforcement Learning using Singular Value Decomposition

Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory

PAC Reinforcement Learning for Predictive State Representations

RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation