cs.AINov 13, 2026

MASTARS: Multi-Agent Sequential Trajectory Augmentation with Return-Conditioned Subgoals

Authors: Jiwon Jeon, Myungsik Cho, Woojun Kim, Seongmin Kim, Woohyeon Byeon, Seungyul Han, Youngchul Sung

Organizations: KAIST · UNIST

Abstract

The performance of offline Multi-Agent Reinforcement Learning (MARL) is limited by the quality of the fixed offline dataset, motivating trajectory augmentation. However, augmenting multi-agent trajectories presents a fundamental challenge of the diversity-coordination trade-off: independent generation improves diversity but lacks coordination, while joint generation induces coordination but lacks diversity, reproducing the training distribution. To address this challenge in multi-agent data augmentation, we introduce MASTARS, a novel diffusion-based framework that generates multi-agent trajectories sequentially across agents while enforcing cross-agent consistent coordination via the method of inpainting. MASTARS produces trajectories that are diverse, coordinated and globally coherent. Experiments on various benchmarks such as MPE, SMAC, and SMAC-v2 demonstrate significant performance gains across multiple offline MARL methods.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning

    Apr 25, 2026Marcel Hedman, Kale-ab Abebe Tessera, Juan Claude Formanek +5Multi-Agent CoordinationOffline RL

  2. G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies

    Sep 25, 2026Guowei Zou, Haitao Wang, Guoxin Wang +4Cooperative MARLTest-Time RL

  3. Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies

    Feb 20, 2026Zhuoran Li, Hai Zhong, Xun Wang +3Multi-Agent Reinforcement LearningMulti-Agent Coordination