cs.AINov 12, 2026

Stack more levels: how to get general and human-like Mario playing

Authors: Shinichi Arimura, Yuchen Li, Julian Togelius

Organizations: New York University, Brooklyn, New York, USA

Abstract

Super Mario Bros. is widely used in game AI and procedural content generation (PCG), yet it remains unclear whether a model-free agent can reliably win unseen levels. To address this question, we train Proximal Policy Optimization (PPO) agents on increasingly large sets of PCG-generated Mario levels and evaluate their generalization on 1000 held-out test levels. We then shape competent PPO policies into three gameplay archetypes---runner, killer, and collector. Finally, we fine-tune these PPO policies using Diffusion-Reward Adversarial Imitation Learning (DRAIL) with human gameplay demonstrations. We find that increasing the number of distinct PCG-generated training levels generally improves unseen-level performance, and that fine-tuning PPO policies with DRAIL increases action-level human-likeness while preserving much of the persona induced by PPO pretraining.

Figures & tables

Explore similar work

CardsList
  1. PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games

    Sep 8, 2026Ryan Truong, Lance Ying, Samuel J. Gershman +1Reinforcement LearningGame-Playing Agents

  2. Policy Improvement with Style-Specific Demonstrations

    Jun 20, 2025Lingfeng Li, Yunlong Lu, Yongyi Wang +1Reinforcement LearningGame-Playing Agents