cs.AISep 29, 2026

AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

Authors: Hongjin Qian, Chaofan Li, Kun Luo, Wenqing Wei, Jianlyu Chen, Shuqi Lu, Yuyang Hu, Hongwang Xiao, +6 more

Organizations: Beijing Academy of Artificial Intelligence (BAAI)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present AREX-2, an effort to advance the self-improving capability of LLM agents, which we define as the ability to iteratively refine a solution at test time. This ability rests on two complementary capabilities: reflection, which produces a solution better than the current one, and long-horizon execution, which keeps the iteration effective over many rounds. We hypothesize that both capabilities are domain-agnostic, and can therefore be learned in scenarios that are well suited for supervision. Accordingly, we synthesize long-horizon improvement trajectories from machine learning and algorithmic programming tasks, two domains that offer verifiable feedback and reward sustained iteration. Trained on this data, our agent, built on Qwen3.8-27B, achieves strong results on MLE-bench Lite (81.8) and Frontier-CS (70.7), transfers to deep research with 84.0 on BrowseComp, 52.6 on HLE, 92.2 on GAIA, and 93.8 on DeepSearchQA, and keeps improving as its budget of rounds grows. These results show that long-horizon reflective data is an effective route toward self-improving agents.

Figures & tables

Figure 1 : Results of AREX-2 on six benchmarks, compared with selected closed and open models.
Figure 2 : Model size versus performance on four benchmarks. AREX-2 (27B) matches or exceeds much larger models.
Figure 3 : Overview of AREX-2. Left: environments are constructed from GitHub repositories and online judges, and are kept only if they pass execution and validation. Middle: an agent improves its solution over many rounds, acquiring operational knowledge on demand, and its trajectories become training data. Right: the model trained on this data improves its solution over multiple rounds at test time.
ModelSizeFrontier-CSMLE-Lite
Closed-Weight Models
GPT-5.6 Sol–76.472.7
Claude Opus 4.8–74.563.6
GPT-5.5–72.168.2
Gemini-3.1-Pro–68.9–
Qwen3.7-Max–61.9–
Table 1 : Comparison on coding and machine learning engineering benchmarks. Model size refers to the total number of parameters. Frontier-CS results are reported on the 188-task Agent Track. ∗ denotes results reproduced by us. MLE-Lite results are reproduced and reported by OpenMLE. MLE-Lite reports Any Medal (mean over three seeds) on MLE-bench Lite. AREX-2 is evaluated on MLE-Lite with skills in its context ( \Cref sec:case-mle).
ModelSizeBrowseCompHLEGAIADeepSearchQA
Frontier Models
GPT-5.6 Sol–90.458.0 *––
GPT-5.6 Terra–87.5–––
GPT-5.6 Luna–83.3–––
Kimi-K32.8T91.256.0 *–95.0
Claude Fable 5–88.064.5 *–94.2
Table 2 : Comparison on general agentic reasoning and deep research benchmarks. ∗ denotes results reported on the full HLE set; unmarked results use the text-only subset.
Figure 6
Figure 6 : Stage-wise ablation on MLE-bench Lite. Stages M0 to M2 use the base model, and M3 and M4 use AREX-2.

Explore similar work

CardsList

  1. Retrospective Progress-Aware Self-Refinement for LLM Agent Training

    Jun 12, 2026Xinbei Ma, Congmin Zheng, Jiyang Qiu +10Self-Improving AgentsLarge Language Model Agents

    1Shanghai Jiao Tong University · 2OPPO Research Institute

  2. AEL: Agent Evolving Learning for Open-Ended Environments

    Apr 23, 2026Wujiang Xu, Jiaojiao Han, Minghao Guo +4Self-Evolving AgentsRecursive Self-Improvement

    Rutgers University · Independent Researcher

  3. SelfSearch: Reward-Free Search for Self-Improving Agents

    Sep 29, 2026Jungwoo Yang, Injin Kong, Yohan JoSelf-Improving AgentsSelf-Evolving Agents

    Graduate School of Data Science, Seoul National University