Markov Decision Processes

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 116

All topics

Robust Parameter Learning for Uncertain MDPs
May 2, 2026Yannik Schnitzer, Alessandro Abate, David ParkerRobust Markov Decision Processes Parameter Estimation
University of Oxford, Oxford, United Kingdom
Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
May 2, 2026Ruiquan Huang, Donghao Li, Yingbin Liang +1Markov Decision Processes Reinforcement Learning
University of Kentucky. · University of Virginia. · Ohio State University..
Reinforcement Learning with Markov Risk Measures and Multipattern Risk Approximation
May 1, 2026Andrzej Ruszczynski, Tiangang ZhangMarkov Decision Processes Reinforcement Learning
Department of Management Science and Information Systems, Rutgers University
Model-Based Reinforcement Learning with Double Oracle Efficiency in Policy Optimization and Offline Estimation
May 1, 2026Haichen Hu, Jian Qian, David Simchi-LeviRegret Minimization in RL Markov Decision Processes
Laboratory for Information and Decision Systems · Department of AI and Data Science
Optimal sequential decision-making for error propagation mitigation in digital twins
Apr 24, 2026Annice Najafi, Shokoufeh MirzaeiMarkov Decision Processes Partially Observable Markov Decision Processes
Department of Industrial and Manufacturing Engineering California State Polytechnic University, Pomona Pomona, CA, 91768, USA
Scale-free adaptive planning for deterministic dynamics & discounted rewards
Apr 20, 2026Peter L. Bartlett, Victor Gabillon, Jennifer Healey +1Markov Decision Processes Model-Based Planning
University of California, Berkeley, USA · Noah’s Ark Lab, Huawei Technologies, London, UK · Adobe Research, San Jose, USA +1
Physics-Informed Causal MDPs for Sequential Constraint Repair in Engineering Simulation Pipelines
Apr 20, 2026Chuhan QiaoMarkov Decision Processes Reinforcement Learning
Beijing JiaoTong University
Beyond the Bellman Fixed Point: Geometry and Fast Policy Identification in Value Iteration
Apr 19, 2026Donghwan LeeMarkov Decision Processes Reinforcement Learning
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea
DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees
Apr 17, 2026Argyrios Gerogiannis, Yu-Han Huang, Venugopal V. VeeravalliNon-Stationary RL Regret Minimization in RL
ECE and CSL, The Grainger College of Engineering University of Illinois at Urbana-Champaign
Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model
Apr 17, 2026Jean Tarbouriech, Matteo Pirotta, Michal Valko +1Markov Decision Processes Policy Learning
Facebook AI Research Paris & Inria Lille · Facebook AI Research Paris · DeepMind Paris
Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
Apr 16, 2026Jean-Bastien Grill, Michal Valko, Rémi MunosMarkov Decision Processes Markov Models
SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗
When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs
Apr 11, 2026Jose Efraim Aguilar Escamilla, Haoyang Hong, Jiawei Li +4Markov Decision Processes Adversarial Attacks
Oregon State University, OR, USA · University of Illinois Urbana–Champaign, IL, USA · Princeton University, NJ, USA +1
Value Mirror Descent for Reinforcement Learning
Apr 7, 2026Zhichao Jia, Guanghui LanMarkov Decision Processes Reinforcement Learning
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
Relating Reinforcement Learning to Dynamic Programming-Based Planning
Mar 8, 2026Filip V. Georgiev, Kalle G. Timperi, Başak Sakçak +1Markov Decision Processes Dynamic Programming
Center for Applied Computing, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland · Dept. of Advanced Computing Sciences, Maastricht University, the Netherlands
Calculating Mutual Information between a Reward Maximizer and its Environment
Feb 13, 2026Alfred Harwood, Jose Faustino, Alex AltairMarkov Decision Processes Reinforcement Learning
1Dovetail Research · University of Sao Paulo
Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity
Feb 3, 2026Aneri Muni, Vincent Taboga, Esther Derman +2Markov Decision Processes Q-Learning
Universit´e de Montr´eal, Quebec, Canada · Mila - Quebec AI Institute, Quebec, Canada · GERAD & Department of Decision Sciences, HEC Montr´eal, Quebec, Canada
Data- and Variance-dependent Regret Bounds for Online Tabular MDPs
Feb 2, 2026Mingyi Li, Taira Tsuchiya, Kenji YamanishiRegret Minimization in RL Markov Decision Processes
Department of Mathematical Informatics, The University of Tokyo, Tokyo, Japan · RIKEN AIP, Tokyo, Japan
Poincaré Meets Bellman: Revisable Memory, Operational Quotients, and Evidence-Supported Learning in Changing Environments
Nov 28, 2025Xin LiMarkov Decision Processes Dynamic Programming
Department of Computer Science University at Albany
Beyond Slater's Condition in Online CMDPs with Stochastic and Adversarial Constraints
Sep 24, 2025Francesco Emanuele Stradi, Eleonora Fidelia Chiefari, Matteo Castiglioni +2Markov Decision Processes Constrained RL
Politecnico di Milano
Learning The Minimum Action Distance
Jun 10, 2025Lorenzo Steccanella, Joshua B. Evans, Özgür Şimşek +1Markov Decision Processes State Representation Learning
Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain · Department of Computer Science, University of Bath, Bath, United Kingdom
Adaptive Resolving Methods for Markov Decision Processes with Function Approximations
May 17, 2025Jiashuo Jiang, Yinyu Ye, Yiming ZongMarkov Decision Processes Reinforcement Learning
Department of Industrial Engineering & Decision Analytics, The Hong Kong University of Science and Technology · Department of Management Science & Engineering, Stanford University
Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis
Mar 24, 2025Mohsen Amiri, Sindri MagnússonNon-Stationary RL Markov Decision Processes
Department of Computer and System Science, Stockholm University, SE-164 25 Stockholm, Sweden
Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning
Sep 22, 2024Jia Wan, Sean R. Sinclair, Devavrat Shah +1Regret Minimization in RL Markov Decision Processes
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology · Department of Industrial Engineering and Management Sciences, Northwestern University
Thompson Sampling for Infinite-Horizon Discounted Decision Processes
May 14, 2024Daniel Adelman, Cagla Keceli, Alba V. Olivares-NadalThompson Sampling Regret Minimization in RL
Booth School of Business, The University of Chicago, Chicago, IL 60637, USA · United Airlines, Chicago, IL 60605, USA · UNSW Business School, UNSW, Kensington NSW 2033, Australia
Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration
Oct 11, 2023Zeyang Li, Chuxiong Hu, Yunan Wang +4Markov Decision Processes Reinforcement Learning
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA 02139, United States · Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China · College of Artificial Intelligence and the School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
Optimization-Based Robust Permissive Synthesis for Interval MDPs
Date pendingKhang Vo Huynh, David Parker, Lu FengRobust Markov Decision Processes Markov Decision Processes
Department of Computer Science, University of Virginia, Charlottesville, VA, USA · Department of Computer Science, University of Oxford, Oxford, UK

May 2, 2026, cs.LGJ/K move · Enter open · S save

Robust Parameter Learning for Uncertain MDPs

Yannik Schnitzer, Alessandro Abate, David Parker

University of Oxford, Oxford, United Kingdom

Robust Markov Decision Processes Parameter Estimation Markov Decision Processes Uncertainty Quantification Markov Models

Open paper AskPDF arXiv

Learning-based approaches to verifying unknown Markov decision processes (MDPs) often employ uncertain MDPs. These models use, for example, confidence intervals to capture transition uncertainty and allow synthesis of policies that are robust to this uncertainty. However, this approach typically quantifies uncertainty independently for individual transition probabilities, ignoring dependencies due to shared latent quantities. We propose to learn such models using parametric MDPs (pMDPs), where transition probabilities are expressions over a set of parameters. We project statistical uncertainty from empirical transition frequencies onto the pMDP's parameter space, yielding a probably approximately correct (PAC) uncertainty model for the underlying MDP that respects the algebraic dependencies between transitions. The resulting models are algorithmically challenging to solve, so we propose a hierarchy of sound polytopic outer approximations of the induced confidence set. We implement and evaluate our approach, demonstrating substantially tighter uncertainty estimates than classical interval-based uncertain MDP learning techniques.

Momentum

Related topics

Latest papers 116