cs.AINov 18, 2026

3D GameDev Arena: Benchmarking Coding Agents That Plan, Build, and Playtest extreme Long-Horizon Browser Games

Authors: Bizhe Bai, Jiliang Jian, Yuqing Yan, Tao Chen, Ying Wen

Organizations: Fudan University · Shanghai Innovation Institute · Xi’an Jiaotong University · Shanghai Jiao Tong University

Abstract

Coding agents now resolve repository issues and generate compact playable games, but whether they can build a large 3D game from scratch remains open. Such games tightly couple mechanics, scenes, assets, and interaction logic, and their quality rests on player enjoyment, which existing benchmarks rarely assess. Thus, we introduce 3D GameDev Arena, a first extreme long-horizon benchmark of 108 from-scratch game-development tasks across 14 genres. Given only a text specification and UI/UX reference images, a fully automated plan--implement--test--repair pipeline builds each task into a self-contained browser game. Runs average ∼\sim1,000 LLM calls ; the longest uses 14,407 calls, 2.13B tokens, and 608 assets. Evaluation spans (1) objective code-level functional acceptance against the game specification, (2) agent-based QA and playtesting of 3D presentation and playability, and (3) blind pairwise Bradley-Terry human preference in a public arena. The preliminary findings indicate that long-horizon 3D game development remains far from solved, and code-level correctness alone does not dictate player preference. The public arena is hosted online.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. UnityBench-Mini3D: Evaluating LLM Agents for 3D Game Development in Unity

    Oct 26, 2026Yijun Zhou, Minghao Shao, Ke Zhang +20Coding Agent BenchmarksAI Agent Evaluation

  2. GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?

    Aug 22, 2026Kun Chen, Haorong Hong, Peizhong Gao +7AI Coding AgentsAI Agent Benchmarks

  3. From Code to Playable Systems: CEGame-Bench for Long-Horizon Game Creation Agents

    Oct 19, 2026Zixuan Dong, Yanghai Wang, Hanyan Bian +8Automated Game TestingSoftware Engineering Agents