Contextual MDPs

Contextual Markov Decision Processes (CMDPs) extend standard reinforcement learning by incorporating context-dependent dynamics and rewards, enabling more realistic modeling of time-varying environments. Current research focuses on developing efficient algorithms with provable guarantees, often leveraging function approximation techniques like linear models or employing offline oracles for density estimation or regression. These advancements aim to improve sample efficiency and reduce computational complexity, addressing key challenges in applying reinforcement learning to complex real-world scenarios. The resulting algorithms find applications in diverse fields such as personalized recommendations and robotics, where adapting to changing contexts is crucial for optimal performance.

Papers

May 28, 2024

Offline Oracle-Efficient Learning for Contextual MDPs via Layerwise Exploration-Exploitation Tradeoff
Jian Qian, Haichen Hu, David Simchi-Levi
Contextual Bandit Action Free Offline Exploration Exploitation Trade Near Optimal Algorithm Reward Free Reinforcement Learning Contextual Markov Decision Process Contextual MDPs

February 5, 2024

Sample Complexity Characterization for Linear Contextual MDPs
Junze Deng, Yuan Cheng, Shaofeng Zou, Yingbin Liang
Sample Complexity Linear Ordered Data MDP Model Contextual Markov Decision Process Contextual MDPs Time Varying Environment

June 5, 2023

A Study of Global and Episodic Bonuses for Exploration in Contextual MDPs
Mikael Henaff, Minqi Jiang, Roberta Raileanu
Study Feature Environment Exploration World Event Novelty Search Unifying Theory Contextual MDPs Episodic Bonus

March 2, 2023

Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximation
Orin Levy, Alon Cohen, Asaf Cassel, Yishay Mansour
Regret Minimization Adversarial Text Adversarial Markov Decision Process Regret Optimal Algorithm Contextual MDPs Rate Optimal Regret

February 4, 2023

Reinforcement Learning with History-Dependent Dynamic Contexts
Guy Tennenholtz, Nadav Merlis, Lior Shani, Martin Mladenov, Craig Boutilier
Reinforcement Learning Non Markovian Contextual Markov Decision Process Contextual MDPs

November 27, 2022

Eluder-based Regret for Stochastic Contextual MDPs
Orin Levy, Asaf Cassel, Alon Cohen, Yishay Mansour
Optimal Regret Stochastic Way Regret Minimization Contextual Markov Decision Process Contextual MDPs RL Algorithm

July 22, 2022

Optimism in Face of a Context: Regret Guarantees for Stochastic Contextual MDP
Orin Levy, Yishay Mansour
Context Information Human Face Reachability Analysis Regret Guarantee Contextual MDPs

March 2, 2022

Learning Efficiently Function Approximation for Contextual MDP
Orin Levy, Yishay Mansour
LeArning Abstract Supervised Learning Function Approximation Contextual MDPs