Markov Decision Processes

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 116

All topics

Closing the Horizon Gap in Policy Optimization for Adversarial MDPs
Oct 8, 2026Mingyi Li, Taira TsuchiyaRegret Minimization in RL Markov Decision Processes
The University of Tokyo · The University of Tokyo, The University of Osaka, and RIKEN
Toward Optimal Regret in Adversarial MDPs with Stochastic Hard Constraints
Oct 8, 2026Qian Zuo, Francesco Emanuele StradiMarkov Decision Processes Constrained RL
University of Edinburgh · Politecnico di Milano
Policy Synthesis for Finite Populations of MDP Agents under Aggregate Reach-Avoid Chance Constraints
Oct 8, 2026Jie Fu, Anamika DubeyMulti-Agent Control Markov Decision Processes
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA · School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA 99164, USA
Average-Reward Reinforcement Learning for Multichain MDPs: A Hierarchical Decomposition Approach
Oct 7, 2026Huizhen Yu, Isaiah HeidtMarkov Decision Processes Average-Reward RL
Department of Computing Science University of Alberta, Canada
Online Resource Allocation with an Endogenous Markov State: Fewer LP Solves Earn More
Oct 7, 2026Zhaohua ChenOnline Resource Allocation Markov Decision Processes
Taobao & Tmall Group of Alibaba, Renmin University of China
Decision-Focused Learning in MDPs: An Occupancy Measure Approach
Oct 6, 2026Zihao Zhao, Ashwath K. Karunakaram, Ali Eshragh +2Markov Decision Processes Decision-Focused Learning
Georgia Institute of Technology, Atlanta, GA · Johns Hopkins Carey Business School, Washington, DC · International Computer Science Institute, Berkeley, CA
Taylor Representations for Model-Free RL in Networked MDPs
Oct 4, 2026Salah Chikhi, Abdelhaq Chaoui, Asuman Ozdaglar +1Markov Decision Processes Representation Learning for RL
Massachusetts Institute of Technology
Linear Programming Representations and Strongly Polynomial Algorithms for Robust Markov Decision Processes
Oct 1, 2026Han Zhong, Yinyu YeRobust Markov Decision Processes Markov Decision Processes
Shanghai Jiao Tong University · Shanghai Jiao Tong University, SIMIS, and Stanford University
Q-Learning for Reachability in MEC-Free MDPs
Oct 1, 2026Lu-Chin Chang, Suguman BansalMarkov Decision Processes Reinforcement Learning
Georgia Institute of Technology
Towards Optimal Policy Improvement
Oct 1, 2026Yaniv Oren, Viliam Vadocz, Wiktor Zabka +7Markov Decision Processes Policy Optimization
Department of Intelligent Systems, TU Delft · Centrum Wiskunde & Informatica, Amsterdam · Department of Computer Science, ETH Zürich +3
Rate-Optimal Algorithm for Adversarial Linear CMDPs
Oct 1, 2026Kihyun Yu, Honghao Wei, Dabeen LeeRegret Minimization in RL Markov Decision Processes
KAIST · Washington State University · Seoul National University
Policy Iteration Is Not Strongly Polynomial for Deterministic Markov Decision Processes: The Price of Algorithmic Anarchy
Sep 30, 2026Han Zhong, Yinyu YeMarkov Decision Processes Policy Iteration
Shanghai Jiao Tong University. · Shanghai Jiao Tong University, SIMIS, and Stanford University.
Fast Regularized Policy Mirror Descent with One-Step TD Updates
Sep 30, 2026Qipei Chen, Wenye Li, Yule Sun +1Markov Decision Processes Policy Optimization
School of Mathematical Sciences, Fudan University · School of Data Science, Fudan University
Learning Infinite-Horizon Average-Reward CMDPs via State Augmentation
Sep 30, 2026Kihyun Yu, Seoungbin Bae, Dabeen LeeRegret Minimization in RL Markov Decision Processes
KAIST · Seoul National University
Vector Bellman Theory for Multichain Robust Average-Reward Markov Decision Processes
Sep 23, 2026Yue Wang, George AtiaRobust Markov Decision Processes Markov Decision Processes
Department of Electrical and Computer Engineering University of Central Florida
A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing
Sep 22, 2026Xiaoxing Ren, Thomas Parisini, Andreas A. MalikopoulosMarkov Decision Processes Linear Markov Decision Processes
School of Civil and Environmental Engineering, Cornell University, Ithaca, NY, USA · Dept. of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK · Dept. of Electronic Systems, Aalborg University, Denmark +2
Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective
Sep 21, 2026Larry Preuett, Qiuyi Zhang, Muhammad Aurangzeb AhmadDistributional RL Markov Decision Processes
University of Washington, Tacoma, USA · Google DeepMind, California, USA · University of Washington, Bothell, USA
Graph-Based Stochastic Power-UCT: Monte-Carlo Graph Search with Power Mean Estimation
Sep 17, 2026Tung Tran, Viet Bao Mai, Hoang Ta +1Markov Decision Processes Monte Carlo Tree Search
School of ICT Hanoi University of Science and Technology Hanoi, Vietnam
A Geometric Theory of Decision Boundaries in Structured Markov Decision Processes
Sep 16, 2026Fredy PokouMarkov Decision Processes Decision-Making under Uncertainty
MRE, INOCS
Strong and Compact Policies for Submodular Markov Decision Processes via LP-Based Submodular Orienteering
Sep 14, 2026Lars Rohwedder, Rico ZenklusenSubmodular Optimization Markov Decision Processes
University of Southern Denmark, Odense, Denmark · ETH Zurich, Zurich, Switzerland
Windowed A-K-MDP
Sep 12, 2026Xiangwen Yang, Frankie Cho, Iadine ChadesMarkov Decision Processes Markov Models
Monash University Melbourne, Australia
Supermartingale Certificates for Parametric MDPs
Sep 11, 2026Kaushik Mallik, Ðorđe ŽikelićRobust Markov Decision Processes Markov Decision Processes
IMDEA Software Institute, Madrid, Spain
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
Aug 10, 2026Chenhua Fan, Jiahui Zhu, Yuhang Zhang +1Markov Decision Processes Constrained RL
School of EECS, Washington State University, Pullman, WA 99163, USA
Path-dependent Discrete Amortized Inference
Aug 9, 2026Tiago da Silva, Esmeralda S. Whitammer, Salem LahlouMarkov Decision Processes Latent Dynamics Modeling
1MBZUAI · University of Edinburgh · 3CIFAR Fellow
Finite Constant Frontiers and Auditable Regret Certificates for Average-Reward Reinforcement Learning
Aug 7, 2026Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus SwaqeebRegret Minimization in RL Markov Decision Processes
Department of Computer Science, Iowa State University · Department of Computer Science & Engineering, BRAC University
From Optimal Actions to World Models: Identifiability of Transition Kernels in Discounted MDPs
Aug 7, 2026Neal BatraMarkov Decision Processes Markov Models
Dovetail Research · University of Warwick
Sub-Quadratic Bisimulation Metrics via Approximate Nearest Neighbors: Coverage-Augmented Guarantees and Computable Two-Sided Certificates
Aug 7, 2026Ibne Farabi Shihab, Joyanta Jyoti MondalMarkov Decision Processes Approximation Algorithms
Department of Computer Science, Iowa State University, USA · Department of Computer and Information Sciences, University of Delaware, USA
Stochasticity Is Not the Hard Part: Reduction and Complexity in Instructional Sequencing over Prerequisite DAGs
Aug 5, 2026Zonglin Han, Yichen Chen, Jiawen Jiang +2Markov Decision Processes Educational Technology
Department of Computer Science, University of California, Davis · International Digital Economy College, Minjiang University · 2International Digital Economy College, Minjiang University +1
Optimal Constrained sc-LTL Planning in MDPs via Switching Policies
Aug 5, 2026Zetong Xuan, Yu WangMarkov Decision Processes Linear Temporal Logic
Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL 32611 USA
Robust Counterfactual Policy Optimisation via Nondeterministic Causal Models
Aug 3, 2026Jessica Lally, Milad Kazemi, Nicola Paoletti +2Markov Decision Processes Structural Causal Models
1King’s College London · 2Durham University · University College London

Oct 8, 2026, cs.LGJ/K move · Enter open · S save

Closing the Horizon Gap in Policy Optimization for Adversarial MDPs

Mingyi Li, Taira Tsuchiya

The University of Tokyo · The University of Tokyo, The University of Osaka, and RIKEN

Regret Minimization in RL Markov Decision Processes Policy Optimization

Open paper AskPDF arXiv

We consider policy optimization for online episodic tabular Markov decision processes (MDPs) with adversarial losses and bandit feedback. Policy optimization updates the policy locally at each state and avoids optimization over the occupancy-measure polytope, but its existing regret bounds are larger by a factor of the horizon

H

than those of occupancy-measure-based algorithms. We close this gap by using regularized

Q

-functions, which allow us to control the stability of the local updates jointly over all state-action pairs rather than separately at each state. The resulting algorithm attains high-probability regret bounds of

\widetilde O(\sqrt{HS(H+A)T})

for known transitions and

\widetilde O(HS\sqrt{AT})

for unknown transitions, where

S

is the number of states,

A

the number of actions, and

T

the number of episodes. Both bounds improve the horizon dependence of existing policy optimization bounds, and the latter matches the best-known bound. We further extend the algorithm to adversarial linear-mixture MDPs and obtain the same improvement in the horizon dependence.

Momentum

Related topics

Latest papers 116