RL Benchmarks

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 62

All topics

Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning
Oct 4, 2026Mickey Beurskens, Tristan Tomilin, Thiago D. SimãoRL Benchmarks Reinforcement Learning
Eindhoven University of Technology
ALER: Adaptive Learnable Experience Rewriting for Reinforcement Learning
Sep 30, 2026Oleg Shchendrigin, Egor Cherepanov, Aleksandr I. Panov +1RL Benchmarks Reinforcement Learning
Innopolis University, Innopolis, Russia · MIRIAI, Moscow, Russia · Cognitive AI Systems Lab, Moscow, Russia
CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL
Sep 30, 2026Shouli Wang, Yanfeng Jia, Zhihao Ou +6RL Benchmarks RL for Code Generation
Tsinghua University · Beihang University · Southern University of Science and Technology +2
Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
Sep 29, 2026Mathias Jackermeier, Jacques Cloete, Alessandro AbateRL Benchmarks Reinforcement Learning
Department of Computer Science, University of Oxford · Oxford Robotics Institute, University of Oxford
PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets
Sep 29, 2026Zhanhua Pan, Xin Qin, Xiao Liu +3RL Benchmarks Multi-Agent Reinforcement Learning
Nanyang Technological University, Singapore. · Cornell University, USA. · University of Bristol, UK.
PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning
Sep 28, 2026Zhanhua Pan, Xiao Liu, Zhilong Cao +2RL Benchmarks Constrained RL
Nanyang Technological University, Singapore. · Cornell University, USA. · University of Bristol, UK.
GlyphBench: A Playground for Language-Model Reinforcement Learning
Sep 28, 2026Roger Creus Castanyer, Marc-Alexandre Côté, Matthew James Sargent +3RL Benchmarks RL for Language Models
Mila - Quebec Artificial Intelligence Institute · Université de Montréal · Vmax
REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff
Sep 15, 2026Riyaaz Shaik, Chandru VenkataramanRL Benchmarks Reinforcement Learning
Apple
Evaluation Metrics for Safe Reinforcement Learning
Sep 14, 2026Lindsay Spoor, Aske Plaat, Thomas MoerlandRL Benchmarks Constrained RL
Leiden Institute of Advanced Computer Science, Leiden University, Leiden, the Netherlands
VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets
Sep 14, 2026Yu Bai, Yukai Miao, Dawei Wang +8RL Benchmarks Reinforcement Learning
Zhongguancun Laboratory · Tsinghua University · Network Management Center, China Mobile
Sim2Signal: Sim-to-Real Benchmarks for Traffic Signal Control
Sep 1, 2026Ferdous Al Rafi, Susrik Mukherjee, Latika Liladhar Dekate +6RL Benchmarks Sim-to-Real Transfer
Arizona State University
Chronocooked: A Benchmark for Interval Timing in Reinforcement Learning Agents
Aug 17, 2026Amrapali Pednekar, Alvaro Garrido-Perez, Yara Khaluf +1RL Benchmarks
Department of Information Technology IDLab, Ghent University - imec · Computer Science Department Vrije Universiteit Amsterdam
DSLE: A Learning Environment for Dark Souls Boss Encounters
Aug 10, 2026Derin Gezgin, Jim O'Connor, Tanner Goodwin +1RL Benchmarks Game-Playing Agents
Autonomous Agent Learning Lab Connecticut College New London, Connecticut, USA
TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning
Jul 23, 2026Chaofan Pan, Lingfei Ren, Xiangyu Jiang +6RL Benchmarks Membership Inference Attacks
1Southwestern University of Finance and Economics · 2Shanxi University
Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning
Jul 17, 2026Vincent Taboga, Justin Veilleux, Doseok Jang +2RL Benchmarks Reinforcement Learning
1Mila - Quebec AI Institute · 2Université de Montréal - Département d’Informatique et de Recherche Opérationnelle
OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning
Jul 14, 2026Emil Mittag, Richard Dazeley, Peter VamplewRL Benchmarks Online Anomaly Detection
Deakin University, Geelong, Victoria, 3220, Australia · Federation University Australia, Ballarat, Victoria, 3350, Australia
Measure the Sim-to-Real Gap: Designing an Affordable Real-World Benchmark Platform for Reinforcement Learning in AIoT Systems
Jul 11, 2026Rongping Zhou, Omid Tavallaie, Shuaijun Chen +1RL Benchmarks Reinforcement Learning
School of Computer Science, The University of Sydney, Camperdown NSW 2050, Australia · Department of Engineering Science, University of Oxford, Wellington Square, Oxford OX1 2JD, United Kingdom · Department of Computer Science, The University of Western Australia, 241, 35 Stirling Hwy, Crawley WA 6009
Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms
Jul 8, 2026Ezgi KorkmazRL Benchmarks Reinforcement Learning
FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games
Jul 7, 2026Chase McDonald, Nathan Tsang, Wesley N. KerrRL Benchmarks Game-Playing Agents
Como Research · Riot Games
EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
Jul 2, 2026Zhilin Wang, Han Song, Runzhe Zhan +13RL Benchmarks AI Agent Benchmarks
University of Science and Technology of China · The Chinese University of Hong Kong · University of Macau +5
RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning
Jun 29, 2026Adithya Mohan, Daniel Kriegl, Torsten SchönRL Benchmarks Reinforcement Learning
AImotion Bavaria · Technische Hochschule Ingolstadt · Ingolstadt, Germany
Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxy
Jun 26, 2026Matthew Vandergrift, Esraa Elelimy, Martha WhiteRL Benchmarks Reinforcement Learning
Department of Computing Science, University of Alberta · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair
SVGym (SciVerseGym): An Environment for Reinforcement Learning and Bayesian Optimization in Crystal Discovery
Jun 21, 2026Bin CaoRL Benchmarks Crystal Structure Generation
The Hong Kong University of Science and Technology (Guangzhou), China · Huawei Noah’s Ark Lab, UK
The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning
Jun 19, 2026Lucas Fagan, Michele Tarquini, Ali Shehper +6RL Benchmarks Reinforcement Learning
Department of Mathematics, California Institute of Technology, Pasadena, CA, USA · Institute of Mathematics, University of Zurich, Zurich, Switzerland · Zero Latency Labs +1
CRAX: Fast Safe Reinforcement Learning Benchmarking
Jun 18, 2026Tristan Tomilin, Mourad Boustani, Mickey Beurskens +2RL Benchmarks Reinforcement Learning
Eindhoven University of Technology
Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning
Jun 17, 2026Thomas Frost, Steve HarrisRL Benchmarks Offline RL
Institute of Health Informatics University College London London, United Kingdom
MineXplore: An Open-Source Reinforcement Learning Exploration Benchmark for GNSS-Denied Underground Environment
Jun 3, 2026Abhishek S, Badrikanath Praharaj, Sreeram MVRL Benchmarks Robot Navigation
BuildMachineLabs
A Unified Benchmark for Dynamic Medical Treatment Reinforcement Learning
May 31, 2026Yuepeng Wang, Ken Kawano, Yoshihiko Fujisawa +12RL Benchmarks Reinforcement Learning
Tokyo University of Agriculture and Technology · Institute of Science Tokyo · National University of Singapore +5
Task diversity produces systematic transfer but inhibits continual reinforcement learning
May 30, 2026Purab Seth, Neil Shah, Ishaan Sinha +4RL Benchmarks Reinforcement Learning
Harvard College · University of Washington · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University +1
Certificate-Guided Evaluation of Reinforcement Learning Generalization
May 30, 2026Vignesh Subramanian, Đorđe Žikelić, Suguman BansalRL Benchmarks Reinforcement Learning
School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA · School of Computing and Information Systems, Singapore Management University, Singapore

Oct 4, 2026, cs.AIJ/K move · Enter open · S save

Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning

Mickey Beurskens, Tristan Tomilin, Thiago D. Simão

Eindhoven University of Technology

RL Benchmarks Reinforcement Learning

Open paper AskPDF arXiv

We introduce Factoriax, a GPU-accelerated factory-building simulator written in JAX. In Factoriax, an agent must collect resources, build machines using those resources, and then automate the collection and crafting process by arranging machines on the map to build production pipelines. This paper discusses the structure of the Factoriax simulator and an initial benchmark called Easy Rocket in which an agent is tasked with building a resource-intensive machine called the Rocket in a limited number of game ticks to escape the planet. We also publish results from a number of PPO-based training runs on Easy Rocket. Factoriax is built to be fast. A 1-billion-step PPO training run, equivalent to 500,000 episodes, runs on Easy Rocket in about 8 minutes on a single NVIDIA A100. A standard laptop GPU can complete the same run in about 84 minutes. Our trained PPO agent learns to gather resources, craft machines from those resources, and place them on the map through a curriculum reward directly tied to a manually designed set of achievements. After training, the agent does not place machines in a functional spatial configuration, failing to fully complete the benchmark, and leaving the challenge open for future attempts.

Momentum

Related topics

Latest papers 62