cs.LGNov 9, 2026

A Modular Perspective for Modeling MARL Systems

Authors: Tiziano Maisonhaute, Jean-Christophe Soulié, Fabien Michel

Organizations: LIRMM, Université de Montpellier, CNRS, Montpellier, France · CIRAD, UPR Recyclage et Risque, F-97743 Saint-Denis, La Réunion, France · Recyclage et Risque, CIRAD, Université de Montpellier, Montpellier, France

Abstract

The growing complexity of Multi-Agent Reinforcement Learning (MARL) has fragmented the literature, obscuring underlying architectural choices behind diverse implementations. This fragmentation makes it difficult to evaluate critical modeling dimensions in isolation, identify how components affect the resulting system dynamics, and compare, reuse, or combine contributions across the literature. This paper proposes a modular perspective for modeling, analyzing, and implementing MARL systems. We decompose them into eight functional modules: agents, environments, learning, reward modeling, training and execution modes, communication, modeling of other agents, and evaluation criteria. For each module, we clarify its role and review representative approaches. We instantiate this decomposition in MARLKIT, a framework whose modules are explicit and interchangeable. MARLKIT demonstrates how the proposed perspective supports transparent experimental assumptions and the controlled variation and combination of MARL mechanisms. Experiments illustrate its ability to prototype, compare, reuse and combine modules while making differences between configurations explicit. Overall, the paper provides a structured perspective on MARL modeling and implementation, offering a common language for making modeling choices explicit, supporting clearer descriptions of MARL approaches, and supporting more controlled experimentation through the systematic isolation and combination of MARL mechanisms.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MARFT: Multi-Agent Reinforcement Fine-Tuning

    Apr 21, 2025Junwei Liao, Muning Wen, Jun Wang +1Multi-Agent LLM SystemsRL Fine-Tuning

  2. FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning

    Feb 10, 2026Zhida Jiang, Zhaolong Xing, Jiawei Lu +13Multi-Agent Reinforcement LearningDistributed RL

  3. Hierarchical Lead Critic based Multi-Agent Reinforcement Learning

    Feb 25, 2026David Eckel, Henri MeeßHierarchical MARLMulti-Agent Reinforcement Learning