Consistent Opponent Modeling in Imperfect-Information Games
Organizations: Ganzfried Research
Abstract
The goal of agents in multi-agent environments is to maximize total reward against the opposing agents that are encountered. Following a game-theoretic solution concept, such as Nash equilibrium, may obtain a strong performance in some settings; however, such approaches fail to capitalize on historical and observed data from repeated interactions against our opponents. Opponent modeling algorithms integrate machine learning techniques to exploit suboptimal opponents utilizing available data; however, the effectiveness of such approaches in imperfect-information games to date is quite limited. We show that existing opponent modeling approaches fail to satisfy a simple desirable property even against static opponents drawn from a known prior distribution; namely, they do not guarantee that the model approaches the opponent's true strategy even in the limit as the number of iterations approaches infinity. We develop a new algorithm that achieves this property given sufficiently informative observations and runs efficiently by solving a convex minimization problem based on the sequence-form game representation.
Figures & tables
| 1 | ||||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 |
| 1 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 | ||||||||||
| -1 | 1 | 1 |
| Leaf node | ||
|---|---|---|
Opponent Modeling
Opponent modeling studies how agents infer others’ goals, preferences, policies, and hidden states to predict and adapt to their actions.
See all 19 papers