Overview

Markov decision processes describe how an agent chooses actions as a system changes state probabilistically and rewards accumulate over time. Dynamic programming methods such as value iteration and policy iteration use Bellman equations to plan, while reinforcement learning methods such as Q-learning, policy gradients, and posterior sampling learn when transition models are unknown. Current work extends these tools to settings with constraints, risk, partial observability, non-stationarity, and large or structured state spaces, using methods such as robust optimization, state abstraction, and function approximation. A central tension is obtaining reliable safety and performance guarantees while keeping learning and planning computationally and statistically efficient beyond small tabular problems.

Latest papers

CardsList

177 papers15 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. Bayesian κκ-Explorer for Minimax Model Estimation in MDPs

    Nov 18, 2026Xihe Gu, Urbashi Mitra, Tara JavidiRL ExplorationMarkov Decision Processes

  2. Approximate Dynamic Programming with Basis Function Learning

    Nov 18, 2026Mohammadreza Fathi, Chi-Guhn LeeValue Function EstimationDynamic Programming

  3. GP--RL--MPC: Uncertainty-Aware Reinforcement Learning-Augmented Model Predictive Control with Gaussian Process Postdecision State-Value Functions

    Nov 17, 2026Jon Li, Jay H. LeeModel Predictive ControlMarkov Decision Processes

  4. Subprocess-Constrained Markov Decision Processes

    Nov 16, 2026Jiarui Gan, Debmalya MandalMarkov Decision ProcessesConstrained RL

  5. Lossless First-Action Menus for Linear-Reward Planning

    Nov 15, 2026Yukun YangSequential Decision MakingMarkov Decision Processes

  6. Improved Regret via Variance-Aware Policy Optimization in Adversarial MDPs

    Nov 12, 2026Ofir Schlisselberg, Alon Cohen, Yishay MansourRegret Minimization in RLMarkov Decision Processes

  7. Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal S\sqrt S and K\sqrt K Dependencies in Tabular and Linear AMDPs

    Nov 12, 2026Aarush Kulkarni, Khang Nguyen, William Chang +1Regret Minimization in RLMarkov Decision Processes

  8. Understanding the Asymptotic Transience of Q-Learning via Distributional Expansion

    Nov 11, 2026Shengbo WangQ-LearningTemporal-Difference Learning

  9. An Analysis of Reinforcement Learning with Function Approximation

    Nov 7, 2026Francisco S. Melo, Sean P. Meyn, M. Isabel RibeiroQ-LearningFunction Approximation in RL

  10. Provably Efficient Representation Learning for Low-Rank CMDPs

    Nov 5, 2026Kaixuan Liu, Guojun Xiong, Shengpu Tang +2Primal-Dual RLMarkov Decision Processes

  11. LLMs as MDP Designers: A Dependency-Aware Agentic Framework for Automated Robot Policy Generation

    Nov 3, 2026Hong Je-Gal, Jinwoo Kim, Chanbin Yi +1LLM-Guided RLRobotic RL

  12. When Is a Game a Markov Decision Process? Correspondences Between Game Theory and Reinforcement Learning

    Oct 30, 2026Eduardo C. Garrido-MerchánGame-Theoretic RLGame Theory

  13. FB-MOAC: A Reinforcement Learning Algorithm for Forward-Backward Markov Decision Processes

    Oct 29, 2026Mohsen Amidzadeh, Mario Di FrancescoMarkov Decision ProcessesMulti-Objective Reinforcement Learning

  14. Game-Theoretic Foundations for Undiscounted Bisimulation Metrics

    Oct 28, 2026Matthew Zurek, Yudong Chen, Josiah P. HannaMarkov Decision ProcessesState Abstraction

  15. Primal-Dual Representation Learning for Low-Rank Constrained MDPs

    Oct 22, 2026Chenhao Zhou, Chao Zhang, Wensong Bai +2Markov Decision ProcessesConstrained RL

  16. Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions

    Oct 21, 2026Xun Shen, Yuepeng Wang, Daniyal Sajedi-Hosseini +6RL ControlMarkov Decision Processes

  17. Towards instance-dependent regret optimality in Episodic MDPs with Posterior Sampling

    Oct 19, 2026Victor Boone, Cyrille Kone, Waris Radji +2Regret Minimization in RLMarkov Decision Processes

  18. Contracting with an Agent under Model Mismatch

    Oct 16, 2026Elif Yılmaz, Jakub Tłuczek, Victor Villin +1Principal-Agent TheoryModel Misspecification

  19. Robust Counterfactuals for Markov Decision Processes under Statistical and Causal Uncertainties

    Oct 14, 2026Jessica Lally, Milad Kazemi, Nicola PaolettiCounterfactual ExplanationsMarkov Decision Processes

  20. Certifying Almost-Sure Reachability in Unknown MDPs

    Oct 13, 2026S Akshay, Suguman Bansal, Gaurav MahlaMarkov Decision ProcessesReachability Analysis

  21. Convex-Analytic View of Query Selection for Safety Verification of Markov Decision Processes

    Oct 13, 2026Han Bao, Shinsaku Sakaue, Kazuki WatanabeFormal VerificationMarkov Decision Processes

  22. Closing the Horizon Gap in Policy Optimization for Adversarial MDPs

    Oct 8, 2026Mingyi Li, Taira TsuchiyaRegret Minimization in RLMarkov Decision Processes

  23. Toward Optimal Regret in Adversarial MDPs with Stochastic Hard Constraints

    Oct 8, 2026Qian Zuo, Francesco Emanuele StradiMarkov Decision ProcessesConstrained RL

  24. Policy Synthesis for Finite Populations of MDP Agents under Aggregate Reach-Avoid Chance Constraints

    Oct 8, 2026Jie Fu, Anamika DubeyMulti-Agent ControlChance-Constrained Control

  25. Average-Reward Reinforcement Learning for Multichain MDPs: A Hierarchical Decomposition Approach

    Oct 7, 2026Huizhen Yu, Isaiah HeidtMarkov Decision ProcessesAverage-Reward RL