cs.AINov 13, 2026

AutoRL: A Multi-Agent System towards Autonomous Reinforcement Learning

Authors: Simon Tang, Wenjie Shi, Yongjing Yin, Keheng Wang, Jialin Liu, Zheng Zhang, Linvo Lin, Xiangyuan Xue, 

Organizations: Meituan · The Chinese University of Hong Kong · ShanghaiTech University · Fudan University

Abstract

Recursive self-improvement (RSI) is one of the most promising paths toward an AI singularity for superintelligence. Achieving this vision requires AI systems to undertake the full training pipeline for iterative improvement. Reinforcement learning (RL) plays an essential role in this cycle to improve model capabilities and generalization, yet automating RL remains difficult due to unstable training dynamics and the need for sustained supervision. We introduce AutoRL, a multi-agent system for automating RL training, as a cornerstone for future RSI. Different from conducting research by end-to-end complete experiments, AutoRL monitors the evolving training state and integrates process-level supervision and research into an ongoing RL training, following the practical workflow of an RL researcher. Specifically, AutoRL consists of three RL-specific agentic roles. Perceptors track training dynamics, rollouts, evaluation, and efficiency, triggering deliberation when they detect anomalies, degradation, or stagnation. Three complementary Proposers diagnose problems and refine candidate interventions through shared evidence and checks. The Orchestrator reviews their recommendation and applies approved changes through online hot updates or checkpoint-based resumption. Experiments on Qwen3 and LongCat-Flash-Lite demonstrate the superiority of AutoRL over static RL recipes and a general agent. The research records show that existing agents can handle unattended RL supervision and dynamic adjustment. The difficulty for future RSI lies in developing fundamental methods that are valuable for resource-consuming validation.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Recursive Self-Improvement through Multi-Agent Self-Supervision

    Oct 8, 2026Hyunin Lee, Jinglue Xu, Jeffrey Seely +4Language Model Self-ImprovementLLM Agent Workflow Optimization

  2. RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement

    Sep 28, 2026Yaxin Du, Xiyuan Yang, Zhifan Zhou +10Language Model Post-TrainingSelf-Improving Agents

  3. RSI-Env: An Open-Ended Environment for Recursive Self-Improvement of Agents

    Nov 1, 2026Wanying Qu, Zhiyuan Fan, Jiyao Liu +4LLM Agent Self-ImprovementRecursive Self-Improvement