Fictitious Self Play

Fictitious self-play (FSP) is a multi-agent reinforcement learning technique aiming to find Nash equilibria in competitive games by having agents repeatedly play against past versions of themselves or a diverse population of opponents. Current research focuses on improving FSP's scalability and applicability to complex scenarios, including partially observable games and those with mixed cooperative-competitive elements, often employing model-free algorithms and neural networks within frameworks like counterfactual regret minimization or Monte Carlo tree search. These advancements are significant for addressing challenges in areas such as cybersecurity (e.g., intrusion response), game playing (e.g., poker, air combat), and network security, where finding robust and efficient strategies against adaptive opponents is crucial.

Papers

October 15, 2024

Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
Jiamian Li
Reinforcement Learning Monte Carlo Tree Search Reward Shaping Q Function Imperfect Information Game Value Estimation Double Deep Q Learning Fictitious Self Play

February 29, 2024

Offline Fictitious Self-Play for Competitive Games
Jingxiao Chen, Weiji Xie, Weinan Zhang, Yong yu, Ying Wen
Offline Reinforcement Learning Action Free Offline Offline Multi Agent Reinforcement Learning Fictitious Self Play

October 5, 2023

Fictitious Cross-Play: Learning Global Nash Equilibrium in Mixed Cooperative-Competitive Games
Zelai Xu, Yancheng Liang, Chao Yu, Yu Wang, Yi Wu
Multi Agent Reinforcement Learning Cross Platform Fictitious Self Play

September 6, 2023

Scalable Learning of Intrusion Responses through Recursive Decomposition
Kim Hammar, Rolf Stadler
Reinforcement Learning Intrusion Detection Stochastic Game Scalable Learning Fictitious Self Play

January 11, 2023

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers
Kim Hammar, Rolf Stadler
Game Theoretic Self Play Defense Algorithm Learning Based Intrusion Detection Optimal Stopping Fictitious Self Play

December 5, 2022

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat
Jiajun Chai, Wenzhang Chen, Yuanheng Zhu, Zong-xin Yao, Dongbin Zhao
Reinforcement Learning Action Space REinforcement Learning Unmanned Combat Fictitious Self Play

June 8, 2022

ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret
Stephen McAleer, Gabriele Farina, Marc Lanctot, Tuomas Sandholm
Nash Equilibrium Value Function Video Game Model Free Importance Sampling Digital Computing Approximate Nash Equilibrium Counterfactual Regret Minimization Static Regret Fictitious Self Play

January 17, 2022

NSGZero: Efficiently Learning Non-Exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search
Wanqi Xue, Bo An, Chai Kiat Yeo
Deep Learning Monte Carlo Tree Search Security Game Fictitious Self Play

Fictitious Self Play

Papers

Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search

Offline Fictitious Self-Play for Competitive Games

Fictitious Cross-Play: Learning Global Nash Equilibrium in Mixed Cooperative-Competitive Games

Scalable Learning of Intrusion Responses through Recursive Decomposition

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat

ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret

NSGZero: Efficiently Learning Non-Exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search