Stationary Policy

Stationary policies, which employ the same decision-making rule across all time steps, are a central focus in reinforcement learning research, particularly for optimizing risk-averse objectives and improving efficiency in various settings. Current research explores their application in diverse contexts, including federated learning, multi-agent systems, and offline reinforcement learning, often employing algorithms like policy iteration, value iteration, and policy gradient methods with neural network architectures designed to incorporate temporal information or handle high-dimensional data. The ability to leverage stationary policies offers significant advantages in terms of computational tractability, interpretability, and scalability, impacting the design of efficient and robust reinforcement learning algorithms for real-world applications.

Papers

October 19, 2024

Simulation-Based Optimistic Policy Iteration For Multi-Agent MDPs with Kullback-Leibler Control Cost
Khaled Nakhleh, Ceyhun Eksin, Sabit Ekin
Multi Agent Kullback Leibler Divergence Policy Iteration Stationary Policy Greedy Policy

October 11, 2024

Two-person positive shortest path games have Nash equlibria in pure stationary strategies
Endre Boros, Khaled Elbassioni, Vladimir Gurvich, Mikhail Vyalyi
Nash Equilibrium Polynomial Time Two Player Zero Sum Equilibrium State Stationary Policy

October 8, 2024

Model Predictive Control is Almost Optimal for Restless Bandit
Nicolas Gast, Dheeraj Narasimha
Predictive Control Finite Horizon Restless Bandit Bandit Identification Stationary Policy

August 30, 2024

Stationary Policies are Optimal in Risk-averse Total-reward MDPs with EVaR
Xihong Su, Marek Petrik, Julien Grand-Clément
Optimal Policy Dynamic Programming Risk Averse Risk Averse Reinforcement Learning Stationary Policy Entropic Risk

September 25, 2023

Federated Learning Under Restricted User Availability
Periklis Theodoropoulos, Konstantinos E. Nikolakakis, Dionysis Kalogerias
Decentralized Machine Learning Random Access Stationary Policy User Selection

July 17, 2023

Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning
Patrick Emami, Xiangyu Zhang, David Biagioni, Ahmed S. Zamzam
Multi Agent Reinforcement Learning Multi Agent Policy Non Stationary Reinforcement Learning Stationary Policy Multi Timescale

February 20, 2023

Logit-Q Dynamics for Efficient Learning in Stochastic Teams
Ahmed Said Donmez, Onur Unlu, Muhammed O. Sayin
Efficient Learning Stochastic Game Stationary Policy Human Team

February 17, 2023

Post Reinforcement Learning Inference
Vasilis Syrgkanis, Ruohan Zhan
Reinforcement Learning Stationary Policy Counterfactual Path Dynamic Treatment Effect Post Prediction Inference

January 29, 2023

Asymptotic Inference for Multi-Stage Stationary Treatment Policy with High Dimensional Features
Daiqi Gao, Yufeng Liu, Donglin Zeng
High Dimensional Feature Treatment Assignment Stationary Policy Asymptotic Inference

December 13, 2022

Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems
Xin Liu, Honghao Wei, Lei Ying
Multi Agent Reinforcement Learning Policy Gradient Potential Scalability Sample Efficiency Multi Agent Reinforcement Policy Gradient Algorithm Stationary Policy

October 27, 2022

Confident Approximate Policy Iteration for Efficient Local Planning in $q^\pi$-realizable MDPs
Gellért Weisz, András György, Tadashi Kozuno, Csaba Szepesvári
Optimal Policy Approximate Dynamic Programming Stationary Policy Approximate Policy Iteration

June 20, 2022

On the Impossibility of Learning to Cooperate with Adaptive Partner Strategies in Repeated Games
Robert Loftin, Frans A. Oliehoek
LeArning Abstract Long Term Cooperation Learning Algorithm Collaborative Decision Matrix Game Stationary Policy Sequential Game Adaptive Partner

June 14, 2022

Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
Shentao Yang, Yihao Feng, Shujian Zhang, Mingyuan Zhou
Offline Reinforcement Learning Policy Optimization Stationary Policy State Action Distribution

March 24, 2022

Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies
Zihan Zhang, Xiangyang Ji, Simon S. Du
Real Power Markov Chain Polynomial Time Tabular Markov Decision Process Stationary Policy

December 9, 2021

Reinforcement Learning with Almost Sure Constraints
Agustin Castellano, Hancheng Min, Juan Bazerque, Enrique Mallada
Reinforcement Learning Markov Decision Process Constrained Markov Decision Process Near Optimal Policy Bellman Operator Stationary Policy

Stationary Policy

Papers

Simulation-Based Optimistic Policy Iteration For Multi-Agent MDPs with Kullback-Leibler Control Cost

Two-person positive shortest path games have Nash equlibria in pure stationary strategies

Model Predictive Control is Almost Optimal for Restless Bandit

Stationary Policies are Optimal in Risk-averse Total-reward MDPs with EVaR

Federated Learning Under Restricted User Availability

Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning

Logit-Q Dynamics for Efficient Learning in Stochastic Teams

Post Reinforcement Learning Inference

Asymptotic Inference for Multi-Stage Stationary Treatment Policy with High Dimensional Features

Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems

Confident Approximate Policy Iteration for Efficient Local Planning in $q^\pi$-realizable MDPs

On the Impossibility of Learning to Cooperate with Adaptive Partner Strategies in Repeated Games

Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning

Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies

Reinforcement Learning with Almost Sure Constraints