cs.AIMar 24, 2026

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

Authors: ARC Prize Foundation

Organizations: ARC Prize Foundation

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions. Like its predecessors ARC-AGI-1 and 2, ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge. ARC-AGI-3 environments only leverage Core Knowledge priors and are difficulty-calibrated via extensive testing with human test-takers. Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%. In this paper, we present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.

Figures & tables

Figure 1: Frontier AI performance on ARC-AGI since introduction in 2019.
Figure 2: Screenshot of ARC-AGI-3 environment ls20 .
Figure 3: First level of ls20 in graph form. Notice the three repeating states – an artifact of the three-life mechanic of the level. PwinP_{\text{win}} for this level is exactly 1 in 355.
DatasetPurpose# of environments
Public DemoA public demonstration set that shows the format and basic mechanics of ARC-AGI-3 environments.25
Semi-PrivateA private hold out set that is used to test models behind an external API.55
Fully PrivateA private hold out that is used for the competition. This set is only given to a very limited number of partners.55
Table 1: ARC-AGI-3 dataset composition.
Figure 4: Action progression and RHAE scoring for environment re86 . Each line represents a different human player’s playthrough, with the x-axis showing cumulative actions taken and the y-axis showing level reached. Labels indicate the final RHAE score each playthrough achieves.
ProviderModelScore
AnthropicOpus 4.6 (Max)0.50%
GoogleGemini 3.1 Pro Preview0.40%
OpenAIGPT 5.4 (High)0.20%
xAIGrok-4.20 (Beta 0309 Reasoning)0.10%
Table 2: Semi-private leaderboard scores for frontier models at release.
Figure 5: Participant demographics.
Figure 6: Per-level efficiency distribution relative to the median human baseline across all public environments with human data (n=1614 level completions across 340 sessions). The dashed red line marks 100% (median efficiency).
Figure 7: Time spent on environments by outcome, split between successful runs (“correct”) and unsuccessful runs (“correct”).
Figure 8: Total actions by level for environment ls20 .
More in this topic

ARC-AGI

ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence) benchmarks the ability to infer and apply abstract rules from sparse examples.

See all 22 papers

Explore similar work

CardsList

  1. The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning

    Mar 9, 2026Sahar Vahdati, Andrei Aioanei, Haridhra Suresh +1Abstract ReasoningARC-AGI

    TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology +1

  2. Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3

    May 25, 2026Liew Keong HanGame-Playing AgentsAI Agent Evaluation

    Independent researcher.

  3. Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

    Jun 5, 2026Jiayu Wang, Weijiang Lv, Bowen Fu +8AI-Assisted Scientific ResearchAI Agent Evaluation

    Xi’an Jiaotong University · Xidian University