cs.AINov 4, 2026

Cooperative Multi-Agent Reinforcement Learning via Epigraph-Form Guided Exploration

Authors: Sungil Son, Hoseong Jung, Dahyun Oh, H. Jin Kim

Organizations: Seoul National University

Abstract

Discovering cooperative behavior in multi-agent reinforcement learning (MARL) is challenging due to the combinatorial complexity of joint state-action spaces, which hinders the emergence of coordinated behaviors from trial-and-error alone. Intrinsic rewards are often used to aid discovery, but naively combining them with team objectives can distort the learning signal, compromising task performance. In this paper, we propose EFXPLORER, a constrained exploration framework that maximizes exploration objectives subject to a constraint that preserves established task performance. We solve the resulting problem using an epigraph reformulation that introduces adaptive exploration budgets. This approach separates intrinsic rewards from task objectives and regulates exploration through task feasibility. To further encourage diverse and temporally extended exploration, we incorporate a successor distance-based intrinsic reward that captures long-horizon dependencies. Empirically, our method outperforms strong baselines and induces novel cooperative strategies across SMAX, VMAS, and MPE benchmark suites.

Figures & tables

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Quality-Aware Exploration Budget Allocation for Cooperative Multi-Agent Reinforcement Learning

    May 3, 2026Dahyun Oh, Minhyuk Yoon, H. Jin KimMulti-Agent CoordinationIntrinsic Reward Methods

  2. Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning

    Jun 3, 2026Zicheng Zhao, Yu Lan, Chengzhengxu Li +2Episodic RLReinforcement Learning

  3. Multi-Agent Reinforcement Learning with Submodular Reward

    Mar 6, 2026Wenjing Chen, Chengyuan Qian, Shuo Xing +2Cooperative MARLMulti-Agent Reinforcement Learning