Overview

Markov decision processes describe how an agent chooses actions as a system changes state probabilistically and rewards accumulate over time. Dynamic programming methods such as value iteration and policy iteration use Bellman equations to plan, while reinforcement learning methods such as Q-learning, policy gradients, and posterior sampling learn when transition models are unknown. Current work extends these tools to settings with constraints, risk, partial observability, non-stationarity, and large or structured state spaces, using methods such as robust optimization, state abstraction, and function approximation. A central tension is obtaining reliable safety and performance guarantees while keeping learning and planning computationally and statistically efficient beyond small tabular problems.

Latest papers

CardsList

177 papers15 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. Decision-Focused Learning in MDPs: An Occupancy Measure Approach

    Oct 6, 2026Zihao Zhao, Ashwath K. Karunakaram, Ali Eshragh +2Markov Decision ProcessesDecision-Focused Learning

  2. Taylor Representations for Model-Free RL in Networked MDPs

    Oct 4, 2026Salah Chikhi, Abdelhaq Chaoui, Asuman Ozdaglar +1Markov Decision ProcessesRepresentation Learning for RL

  3. On the Convergence of Success Conditioning for Policy Optimization

    Oct 2, 2026Matthew Brun, Xu Andy SunPolicy OptimizationMarkov Decision Processes

  4. Linear Programming Representations and Strongly Polynomial Algorithms for Robust Markov Decision Processes

    Oct 1, 2026Han Zhong, Yinyu YeRobust Markov Decision ProcessesMarkov Decision Processes

  5. Q-Learning for Reachability in MEC-Free MDPs

    Oct 1, 2026Lu-Chin Chang, Suguman BansalMarkov Decision ProcessesReinforcement Learning

  6. Towards Optimal Policy Improvement

    Oct 1, 2026Yaniv Oren, Viliam Vadocz, Wiktor Zabka +7Markov Decision ProcessesPolicy Optimization

  7. Rate-Optimal Algorithm for Adversarial Linear CMDPs

    Oct 1, 2026Kihyun Yu, Honghao Wei, Dabeen LeeRegret Minimization in RLMarkov Decision Processes

  8. Policy Iteration Is Not Strongly Polynomial for Deterministic Markov Decision Processes: The Price of Algorithmic Anarchy

    Sep 30, 2026Han Zhong, Yinyu YeMarkov Decision ProcessesPolicy Iteration

  9. Fast Regularized Policy Mirror Descent with One-Step TD Updates

    Sep 30, 2026Qipei Chen, Wenye Li, Yule Sun +1Markov Decision ProcessesPolicy Optimization

  10. Learning Infinite-Horizon Average-Reward CMDPs via State Augmentation

    Sep 30, 2026Kihyun Yu, Seoungbin Bae, Dabeen LeeRegret Minimization in RLMarkov Decision Processes

  11. Achieving an O(1/N)O(1/N) Optimality Gap in Average-Reward Weakly-Coupled MDPs

    Sep 29, 2026Yige Hong, Xiangcheng Zhang, Qiaomin Xie +2Markov Decision ProcessesAverage-Reward RL

  12. Optimal Multi-Reward Reinforcement Learning

    Sep 29, 2026Zijun Chen, Zihan ZhangMarkov Decision ProcessesSample-Efficient RL

  13. Fair Policy Optimization in Major-Minor Weakly Coupled Markov Decision Processes

    Sep 28, 2026Xiaohui Tu, Yossiri Adulyasak, Erick DelageFair Resource AllocationMarkov Decision Processes

  14. Learning Chance-Constrained MDPs with Bellman Distributional Certificates

    Sep 25, 2026Chenbei Lu, Hongyu YiMarkov Decision ProcessesConstrained RL

  15. Vector Bellman Theory for Multichain Robust Average-Reward Markov Decision Processes

    Sep 23, 2026Yue Wang, George AtiaRobust Markov Decision ProcessesMarkov Decision Processes

  16. A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing

    Sep 22, 2026Xiaoxing Ren, Thomas Parisini, Andreas A. MalikopoulosMarkov Decision ProcessesLinear Markov Decision Processes

  17. Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective

    Sep 21, 2026Larry Preuett, Qiuyi Zhang, Muhammad Aurangzeb AhmadDistributional RLMarkov Decision Processes

  18. Provably Efficient Reinforcement Learning in Continuous-Time Episodic MDPs with Poisson Decision Epochs

    Sep 19, 2026Kenny Guo, Valentio Iverson, Sahan Wijetunga +1Markov Decision ProcessesContinuous-Time RL

  19. Graph-Based Stochastic Power-UCT: Monte-Carlo Graph Search with Power Mean Estimation

    Sep 17, 2026Tung Tran, Viet Bao Mai, Hoang Ta +1Markov Decision ProcessesMonte Carlo Tree Search

  20. Strong and Compact Policies for Submodular Markov Decision Processes via LP-Based Submodular Orienteering

    Sep 14, 2026Lars Rohwedder, Rico ZenklusenSubmodular OptimizationMarkov Decision Processes

  21. Windowed A-K-MDP

    Sep 12, 2026Xiangwen Yang, Frankie Cho, Iadine ChadesMarkov Decision ProcessesMarkov Models

  22. Supermartingale Certificates for Parametric MDPs

    Sep 11, 2026Kaushik Mallik, Ðorđe ŽikelićRobust Markov Decision ProcessesMarkov Decision Processes

  23. Power Mean Estimation in Stochastic Continuous Monte Carlo Tree Search

    Sep 6, 2026Tuan DamMonte Carlo Tree SearchMarkov Decision Processes

  24. Categorizer Automata for Discounted-Sum Payoffs

    Aug 27, 2026Nathalie Bertrand, Pranav Ghorpade, Senthil Rajasekaran +2Policy OptimizationMarkov Decision Processes

  25. Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes

    Aug 24, 2026Ege C. Kaya, Aliasghar Pourghani, Mahsa Ghasemi +2Semi-Markov Decision ProcessMarkov Decision Processes