Overview

Markov decision processes describe how an agent chooses actions as a system changes state probabilistically and rewards accumulate over time. Dynamic programming methods such as value iteration and policy iteration use Bellman equations to plan, while reinforcement learning methods such as Q-learning, policy gradients, and posterior sampling learn when transition models are unknown. Current work extends these tools to settings with constraints, risk, partial observability, non-stationarity, and large or structured state spaces, using methods such as robust optimization, state abstraction, and function approximation. A central tension is obtaining reliable safety and performance guarantees while keeping learning and planning computationally and statistically efficient beyond small tabular problems.

Latest papers

CardsList

177 papers15 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing

    Aug 21, 2026Huiling Meng, Ningyuan Chen, Xuefeng GaoMarkov Decision ProcessesContinuous-Time RL

  2. Boundary-Seeking Policy Gradient for Safe Reinforcement Learning

    Aug 10, 2026Chenhua Fan, Jiahui Zhu, Yuhang Zhang +1Markov Decision ProcessesConstrained RL

  3. Path-dependent Discrete Amortized Inference

    Aug 9, 2026Tiago da Silva, Esmeralda S. Whitammer, Salem LahlouMarkov Decision ProcessesLatent Dynamics Modeling

  4. Finite Constant Frontiers and Auditable Regret Certificates for Average-Reward Reinforcement Learning

    Aug 7, 2026Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus SwaqeebRegret Minimization in RLMarkov Decision Processes

  5. Sub-Quadratic Bisimulation Metrics via Approximate Nearest Neighbors: Coverage-Augmented Guarantees and Computable Two-Sided Certificates

    Aug 7, 2026Ibne Farabi Shihab, Joyanta Jyoti MondalMarkov Decision ProcessesApproximation Algorithms

  6. Stochasticity Is Not the Hard Part: Reduction and Complexity in Instructional Sequencing over Prerequisite DAGs

    Aug 5, 2026Zonglin Han, Yichen Chen, Jiawen Jiang +2Markov Decision ProcessesEducational Technology

  7. Optimal Constrained sc-LTL Planning in MDPs via Switching Policies

    Aug 5, 2026Zetong Xuan, Yu WangMarkov Decision ProcessesLinear Temporal Logic

  8. Robust Counterfactual Policy Optimisation via Nondeterministic Causal Models

    Aug 3, 2026Jessica Lally, Milad Kazemi, Nicola Paoletti +2Markov Decision ProcessesStructural Causal Models

  9. Improved Quantum Algorithms for Reinforcement Learning Under a Generative Model

    Aug 3, 2026Joao F. DoriguelloMarkov Decision ProcessesReinforcement Learning

  10. Optimizing Minimax Regret in Uncertain MDPs with Small Sets of Policies

    Aug 3, 2026Sterre Lutz, Daniël Vos, Matthijs T. J. Spaan +1Markov Decision ProcessesBranch and Bound

  11. PAC Approximation and DIRECT Optimization for Parametric Markov Models

    Aug 3, 2026Zhiming Chi, Ying Liu, Andrea Turrini +2Robust Markov Decision ProcessesMarkov Decision Processes

  12. Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints

    Aug 2, 2026Francesco Cordiano, Kanghui He, Bart De SchutterChance-Constrained OptimizationMarkov Decision Processes

  13. Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes

    Jul 29, 2026Zuyuan Zhang, Yongshan Chen, Mahdi Imani +1Markov Decision ProcessesPartially Observable RL

  14. Property-driven Causal Abstractions for Markov Decision Processes

    Jul 29, 2026Jule Schmidt, Maximilian Weininger, Clemens Dubslaff +2Markov Decision ProcessesMarkov Models

  15. dtControl2+ε\varepsilon: Trading Optimality for Explainability in MDPs via Decision Trees

    Jul 28, 2026Tereza Kinská, Jan Křetínský, Tobias Meggendorfer +2Markov Decision ProcessesMarkov Models

  16. Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes

    Jul 25, 2026Asha Barua, Sajad KhodadadianPolicy GradientMarkov Decision Processes

  17. Active Inference as a Convex Markov Decision Process

    Jul 22, 2026Nikola Milosevic, Nicolás Hinrichs, Nico ScherfMarkov Decision ProcessesReinforcement Learning

  18. Long-Term Sequential Decision Making under Risk

    Jul 22, 2026Irmaan, Mirzanejad, Nadjet Bourdache +1Markov Decision ProcessesDynamic Programming

  19. Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence

    Jul 22, 2026Runlong Zhou, Zihan Zhang, Maryam Fazel +1Regret Minimization in RLMarkov Decision Processes

  20. Generalised Bellman recurrence and three dualities in sequential decision-making

    Jul 20, 2026Fernando E. Rosas, David Hyland, Daniel PolaniMarkov Decision ProcessesSequential Decision Making

  21. Learning-Driven Adaptive Audit Scheduling: A Sequential Decision Approach to Off-Chain Data Integrity

    Jul 19, 2026Changting Lin, Fan Li, Weihang Yu +4Markov Decision ProcessesDeep Q-Learning

  22. Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning

    Jul 19, 2026Joseph Lazzaro, Alessio Russo, Aldo PacchianoMarkov Decision ProcessesRL Exploration

  23. Risk-Aware General-Utility Markov Decision Processes

    Jul 10, 2026Pedro P. Santos, Fábio Vital, Alberto Sardinha +1Markov Decision ProcessesMarkov Models

  24. Mathematical methods of reinforcement learning

    Jul 8, 2026Denis Belomestny, Alexander Gasnikov, Egor Gladin +5Markov Decision ProcessesReinforcement Learning

  25. Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

    Jun 30, 2026Mingyi Li, Taira Tsuchiya, Kenji YamanishiRegret Minimization in RLMarkov Decision Processes

  26. Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guarantees

    Jun 29, 2026Ryohei Oura, Georgios Fainekos, Hideki Okamoto +1Reachability AnalysisMarkov Decision Processes

  27. Confidence Sequences for Online Statistical Model Checking of Markov Decision Processes

    Jun 24, 2026Konstantin Kueffner, Tobias Meggendorfer, Maximilian Weininger +1Markov Decision ProcessesUncertainty Quantification