Markov Game

Markov games model strategic interactions among multiple agents in dynamic environments, aiming to find equilibrium solutions like Nash equilibria that represent stable outcomes. Current research focuses on developing efficient algorithms for learning these equilibria, particularly in large-scale settings, often employing techniques like mean-field games, actor-critic methods, and policy gradient approaches, and addressing challenges posed by incomplete information, asymmetry, and robustness to uncertainty. This field is crucial for advancing multi-agent reinforcement learning and has significant implications for diverse applications, including robotics, economics, and energy systems.

Papers

October 3, 2022

Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games
Shicong Cen, Yuejie Chi, Simon S. Du, Lin Xiao
Multi Agent Reinforcement Learning Policy OpTimization Markov Game Last Iterate Convergence Zero Sum Markov Game Single Agent RL

September 26, 2022

$O(T^{-1})$ Convergence of Optimistic-Follow-the-Regularized-Leader in Two-Player Zero-Sum Markov Games
Yuepeng Yang, Cong Ma
Early Stage Convergence Nash Equilibrium Convergence Rate Markov Game Follow the Regularized Leader Zero Sum Markov Game Normal Form Game

August 22, 2022

Minimax-Optimal Multi-Agent RL in Markov Games With a Generative Model
Gen Li, Yuejie Chi, Yuting Wei, Yuxin Chen
Multi Agent Reinforcement Learning Markov Decision Process Generative Modeling Nash Equilibrium Markov Game General Sum Markov Game Minimax Q Learning

August 10, 2022

Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium
Chris Junchi Li, Dongruo Zhou, Quanquan Gu, Michael I. Jordan
Nash Equilibrium Markov Game Nonlinear Function Approximation Correlated Equilibrium Kernel Approximation Linear Mixture Markov Decision Process

August 3, 2022

Efficiently Computing Nash Equilibria in Adversarial Team Markov Games
Fivos Kalogiannis, Ioannis Anagnostides, Ioannis Panageas, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Vaggos Chatziafratis, Stelios Stavroulakis
Nash Equilibrium Markov Game Adversarial Team Game

July 28, 2022

Regret Minimization and Convergence to Equilibria in General-sum Markov Games
Liad Erez, Tal Lancewicki, Uri Sherman, Tomer Koren, Yishay Mansour
Early Stage Convergence Regret Minimization Sublinear Regret Markov Game Efficient Equilibrium General Sum Markov Game

July 25, 2022

Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions
Shuang Qiu, Xiaohan Wei, Jieping Ye, Zhaoran Wang, Zhuoran Yang
Markov Game Single Agent Reinforcement Learning Zero Sum Markov Game State Transition Two Agent Fictitious Play

July 18, 2022

A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in Two-Player Atari Games
Zihan Ding, Dijia Su, Qinghua Liu, Chi Jin
Markov Game Deep Reinforcement Learning Approach Tabular Markov Decision Process ATARI Game Nash Equilibrium Policy Double DQN

June 26, 2022

Generalized Beliefs for Cooperative AI
Darius Muglich, Luisa Zintgraf, Christian Schroeder de Witt, Shimon Whiteson, Jakob Foerster
Optimal Policy Markov Game Belief State Based Policy Ad Hoc Cooperative AI

June 15, 2022

Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games
Dingyang Chen, Qi Zhang, Thinh T. Doan
Policy Gradient Early Stage Convergence Markov Game Curious Price Natural Policy Gradient Stability Guarantee Softmax Policy Markov Potential Game

June 8, 2022

Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games
Yuling Yan, Gen Li, Yuxin Chen, Jianqing Fan
Model Based Reinforcement Learning Nash Equilibrium Markov Game Minimax Optimality Infinite Horizon Approximate Nash Equilibrium

June 6, 2022

Policy Optimization for Markov Games: Unified Framework and Faster Convergence
Runyu Zhang, Qinghua Liu, Huan Wang, Caiming Xiong, Na Li, Yu Bai
Multi Agent Reinforcement Learning Unified Framework Policy OpTimization Markov Game Faster Convergence General Sum Markov Game

June 3, 2022

Decentralized Optimistic Hyperpolicy Mirror Descent: Provably No-Regret Learning in Markov Games
Wenhao Zhan, Jason D. Lee, Zhuoran Yang
Policy Evaluation Markov Game Vector Valued Online Mirror Descent Decentralized Policy

June 1, 2022

Provably Efficient Offline Multi-agent Reinforcement Learning via Strategy-wise Bonus
Qiwen Cui, Simon S. Du
Markov Game Reward Report Offline Multi Agent Reinforcement Learning

May 31, 2022

May 27, 2022

Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games
Sihan Zeng, Thinh T. Doan, Justin Romberg
Gradient Descent Nash Equilibrium Markov Game Zero Sum Markov Game Gradient Descent Ascent Accelerated Convergence

March 14, 2022

Markov Game

Papers

Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games

$O(T^{-1})$ Convergence of Optimistic-Follow-the-Regularized-Leader in Two-Player Zero-Sum Markov Games

Minimax-Optimal Multi-Agent RL in Markov Games With a Generative Model

Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium

Efficiently Computing Nash Equilibria in Adversarial Team Markov Games

Regret Minimization and Convergence to Equilibria in General-sum Markov Games

Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions

A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in Two-Player Atari Games

Generalized Beliefs for Cooperative AI

Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games

Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games

Policy Optimization for Markov Games: Unified Framework and Faster Convergence

Decentralized Optimistic Hyperpolicy Mirror Descent: Provably No-Regret Learning in Markov Games

Provably Efficient Offline Multi-agent Reinforcement Learning via Strategy-wise Bonus

One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning

Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Games

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game

Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games

Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation

Learning Markov Games with Adversarial Opponents: Efficient Algorithms and Fundamental Limits