The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning
Authors: Sahar Vahdati, Andrei Aioanei, Haridhra Suresh, Jens Lehmann
Organizations: TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology · Amazon (work done outside of Amazon)
The Abstraction and Reasoning Corpus (ARC-AGI) has become a key benchmark for fluid intelligence in AI. This survey presents the first cross-generation analysis of 82 approaches across three benchmark versions and the ARC Prize 2024-2025 competitions. Our central finding is that performance degradation across versions is consistent across all paradigms: program synthesis, neuro-symbolic, and neural approaches all exhibit 2-3x drops from ARC-AGI-1 to ARC-AGI-2, indicating fundamental limitations in compositional generalization. While systems now reach 93.0% on ARC-AGI-1 (Opus 4.6), performance falls to 68.8% on ARC-AGI-2 and 13% on ARC-AGI-3, as humans maintain near-perfect accuracy across all versions. Cost fell 390x in one year (o3's 4,500/tasktoGPT−5.2′s12/task), although this largely reflects reduced test-time parallelism. Trillion-scale models vary widely in score and cost, while Kaggle-constrained entries (660M-8B) achieve competitive results, aligning with Chollet's thesis that intelligence is skill-acquisition efficiency. Test-time adaptation and refinement loops emerge as critical success factors, while compositional reasoning and interactive learning remain unsolved. ARC Prize 2025 winners needed hundreds of thousands of synthetic examples to reach 24% on ARC-AGI-2, confirming that reasoning remains knowledge-bound. This first release of the ARC-AGI Living Survey captures the field as of February 2026, with updates at https://nimi-ai.com/arc-survey/
Figures & tables
Figure 1. Example task from each ARC-AGI version and the performance cliff across them. On the left side, we show one example from each ARC-AGI version, illustrating the increased complexity and requirements. On the right side, horizontal stacked bars show best AI performance (darker color) and the gap to human baseline (lighter color) for each benchmark version.
Figure 2. Representative ARC-AGI tasks illustrating two core reasoning categories. Left: Object-centric reasoning (Task f76d97a5, 3 training examples). The transformation extracts the colored checkerboard pattern from a gray background, requiring the system to identify “object” versus “background” without explicit segmentation cues. Right: Geometric transformation (Task c97c0139, 2 training examples). Red line segments define reflection axes around which cyan diamond shapes must be generated symmetrically. Both tasks require inferring abstract rules from minimal demonstrations and generalizing to novel configurations.
Feature
ARC-AGI-1
ARC-AGI-2
ARC-AGI-3
Release date
Nov 2019
Mar 2025
Jul 2025
Format
Static
Static
Interactive
Number of tasks
1,000
600
Variable
Grid size
≤ 30 × 30
≤ 30 × 30
64 × 64
Color palette
10 colors
10 colors
16 colors
Best AI performance
90.5% † / 55.5% ‡
54.2% † / 24.03% ‡
12.58%
Table 1. Comparison of ARC-AGI benchmark versions showing progressive increases in difficulty and the dramatic widening of the human-AI performance gap.
Paradigm
System
Score (%)
Pure Paradigm Approaches
Induction
Neural-guided synthesis ( Ouellette, 2025 )
79.3
Induction
Abductive solver ( Lim et al., 2024 )
66.0
Induction
ConceptSearch ( Singhal and Shroff, 2025 )
50-58
Induction
Greenblatt GPT-4o ( Greenblatt, 2024 )
50.0
Induction
Graph constraints ( Xu et al., 2022 )
35.6
Table 2. Performance comparison across solution paradigms and representative systems on ARC-AGI-1 public evaluation set. Scores represent percentage of tasks solved exactly. Systems evaluated on ≥ 100 tasks provide more reliable estimates than those tested on smaller subsets. The paradigm choice establishes a performance ceiling, but within-paradigm variance often exceeds between-paradigm differences.
20–30 pt advantage over single-representation systems
Compositional primitives
Reusable operations composed for novel tasks
Enables systematic generalization; limited to 1–2 steps in current systems
Table 4. Success factors shared by systems exceeding 70% on ARC-AGI-1.
Anti-pattern
Description
Impact
Pure pattern matching
Learns correlations, not causal/compositional rules
∼ 40% ceiling; collapses on harder compositions
Unguided exhaustive search
Enumerates programs without strong heuristics
Exponential blowup; usually ≤ 30%
Static representation
Locks into one abstraction (objects/regions/etc.)
Brittle failures; ∼ 25-pt deficit
No test-time adaptation
Fixed policy; ignores task demos as learning signal
∼ 30–40% plateau; ∼ 20–30 pts behind adaptive
Monolithic architecture
No modular hybrid of perception + symbolic ops
∼ 20–40 pts worse than hybrids
Frontier inefficiency
High score via expensive compute; hardness cliff persists
Costly per task; far from human efficiency
Table 5. Common anti-patterns and their observed impacts.
Figure 3. ARC-AGI performance across benchmark versions. Public leaderboard (solid bars) allows unconstrained compute and API access; Kaggle competition (hatched bars) is constrained to 50computebudgetwithnointernet.Arrowsshowgaptohumanbaseline.Thehumanbaselineof1001.88 and 3.64/taskvs.11.40 and 38.90/task);(3)PublicscoresexceedKaggleby30–70percentagepoints,reflectingunconstrainedvs.constrainedcomputeregimes;(4)Theperformancecliffpersistsacrossallsystems:eventhebestpublicentrydrops230.20/task (120 pts/)vs.Land’s72.938.90/task (2 pts/),a60\times$ efficiency gap.
Figure 4. Cross-generation performance cliff: seven systems evaluated on both ARC-AGI-1 and ARC-AGI-2. Land’s cross-model ensemble ( Land, 2026 ) achieves the highest scores on both benchmarks but still drops 23%. Notably, Opus 4.6 shows the smallest single-model drop ( − 26%, from 93.0% to 68.8%), compared to its predecessor Opus 4.5 ( − 53%) and GPT-5.2 Pro ( − 40%). Other single frontier models (solid bars) drop 40–63%, while Kaggle competition winners (hatched bars) drop 70–77%. The human baseline of 100% represents task-level solvability (every task solved by at least one person); average individual accuracy is 76% on ARC-AGI-1 and 60% on ARC-AGI-2 ( Chollet et al., 2025a ) .
Figure 5. Temporal evolution of ARC-AGI performance (2019–2025). Blue: ARC-AGI-1; Amber: ARC-AGI-2. The 2024 phase transition shows performance improving more in six months than the previous five years. The consistent 2.5–3 × degradation from ARC-AGI-1 to ARC-AGI-2 across all approaches indicates fundamental compositional limitations.
Figure 6. Yearly progress of ARC-AGI scores from 2020 to 2026. Left: maximum score progression. ARC-AGI-1 scores rose from 78.8% (2020) through a plateau in 2023–2024 before surging to 93.0% in 2026 (Opus 4.6); ARC-AGI-2, introduced in 2024, jumped from 2.5% to 68.8% in under two years. Right: mean scores with ±1σ confidence bands. The widening bands in 2024–2025 (ARC-AGI-1 mean 39–45%, σ≈20 –26) reflect growing methodological diversity as the field expanded from 2–3 papers per year to over 40. The persistent gap between maximum and mean scores indicates that top-performing techniques (large-scale inference, test-time training) have not yet diffused broadly across the research community.
Figure 7. Publication trends by research category. The 2024–2025 surge reflects the ARC Prize competition’s catalytic effect. Hybrid approaches dominate, reflecting recognition that neither pure neural nor symbolic methods suffice. Pure neural approaches declined after demonstrating consistent 20% ceilings.
Figure 8. Score distributions across ARC-AGI benchmark versions and evaluation sets. Public eval scores (self-reported on the public evaluation set) cover 38 systems on ARC-AGI-1 (median 25.6%, mean 34.1%) and 10 on ARC-AGI-2 (median 8.2%, mean 12.6%). Semi-private eval scores (official leaderboard) appear higher because the two groups represent different populations : frontier models (Opus 4.6, GPT-5.2 Pro, Gemini 3) submitted exclusively to the semi-private eval and never published public eval scores, skewing that distribution upward (18 systems on ARC-AGI-1, median 54.5%; 11 on ARC-AGI-2, median 26.0%). Among the 8 systems reporting both scores on ARC-AGI-1, semi-private results are 5–38 percentage points lower than public, consistent with the harder held-out set.
Figure 9. Cost per task versus accuracy on ARC-AGI-1 for the 10 systems (12% of surveyed papers) reporting cost data. Dotted lines are iso-efficiency curves (constant pts/).Costsspanfiveordersofmagnitude(0.0003–44/task)withefficienciesrangingfrom2to90,000pts/. The Pareto frontier (dashed) shows steep diminishing returns above $2/task. The human baseline (star) occupies the high-efficiency region that no AI system yet approaches.
Figure 10. Score vs. cost per task vs. estimated model scale across ARC-AGI-1 and ARC-AGI-2. Bubble area is proportional to parameter count (log-scaled) and black edges denote Kaggle-constrained entries. Parameter counts marked (est.) are community estimates; undisclosed architectural choices make cross-family comparisons approximate. On ARC-AGI-1, trillion-scale models vary widely in both score and cost: Opus 4.6 reaches 93.0% at 1.88/task,GPT−5.2Proscores90.511.64/task, and Grok 4 scores 66.7% at 1.01/task.Kaggle−constrainedentriesclusterat0.15–$0.20/task with 660M–8B parameters, while TRM reaches 6.3% on ARC-AGI-2 with just 7M parameters. This aligns with Chollet’s thesis underlying ARC-AGI ( Chollet, 2019 ) : “solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience.”
Figure 11. ARC breakthroughs filtered for evaluation rigor ( ≥ 100 tasks for ARC-AGI-1, ≥ 50 for ARC-AGI-2; see exclusion criteria in the Supplementary Material, Section 2). Circle markers denote public evaluations; diamond markers denote semi-private evaluations. The 2020–2023 timeline is compressed to emphasize the rapid progress in 2024–2026. On ARC-AGI-1, the rigorous maximum rose from 39.0% (Icecuber, 2020, public) to 93.0% (Opus 4.6, 2026, semi-private). ARC-AGI-2, introduced in late 2024, has seen its best rigorous score reach 68.8% (Opus 4.6, 2026).
Figure 12. Public versus semi-private ARC scores. Only 11 of 82 papers report both (87% transparency gap). On ARC-AGI-1, the mean public-to-semi-private drop is 10.8% (median 6.1%), with gaps ranging from − 5.0% to 37.8%. On ARC-AGI-2, gaps are smaller (mean 2.0%), likely because the semi-private set was introduced recently and fewer systems have been evaluated on both splits. Points above the diagonal indicate higher semi-private than public scores.
System
Score
Key innovation
Refinement mechanism
NVARC ( Sorokin and Puget, 2025 ) (1st)
24.0%
Synthetic data mixing + 3-component ensemble (Qwen3 + TRM)
Cross-model ensemble (GPT-5.2, Gemini 3, Opus 4.5)
Multi-model program synthesis + evaluator-based adjudication
Table 6. ARC Prize 2025 winners (private ARC-AGI-2 test) and public leaderboard SOTA. All top systems implement refinement at different abstraction levels.
Axis
Interpretation question
Efficiency
What computational resources are required per solved task (e.g., sampling budget, number of refinement steps, wall-clock time, monetary cost)? A method that succeeds only through massive test-time computation may approximate intelligent behavior without revealing a compact mechanism.
Adaptability
What changes at test time, and how? Does the system perform test-time adaptation (e.g., puzzle-specific training, search, hypothesis revision, tool use), and is that adaptation principled or primarily brute-force?
Compositional scope
Does the strategy generalize beyond the easiest regime? In particular, does it preserve competence as tasks demand deeper multi-step composition (e.g., ARC-AGI-2) and, eventually, interactive learning (ARC-AGI-3)?
Error signature
When the system fails, does it fail in interpretable, systematic ways that illuminate a missing primitive or decomposition step, or does it collapse unpredictably across task families?
Table 7. Mechanism-relevant axes for interpreting ARC-AGI scores (RQ1).
Dimension
“More with more” (scaling)
“More with less” (compression)
Primary driver
More samples, more compute, more data, larger ensembles
Effective theories: coarse-grained abstractions that screen off irrelevant detail ( Krakauer et al., 2025 )
What transfers
Surface regularities and task-family coverage
Structural principles that generalize across substrates (analogy) ( Hofstadter and the Fluid Analogies Research Group, 2001 )
Typical failure
Cliff-like breakdown under deeper composition or distribution shift
Table 8. Two routes to performance. ARC-AGI progress is currently dominated by scaling, while human-like fluid intelligence is associated with conceptual compression.
Direction
Summary
Recursive latent refinement
Jolicoeur-Martineau’s Tiny Recursive Model (TRM) achieves 45% on ARC-AGI-1 and 8% on ARC-AGI-2 with only 7M parameters, less than 0.01% of leading LLMs ( Jolicoeur-Martineau, 2025 ) . The architecture alternates between “think” steps ( z←f(x,y,z) ) that accumulate latent reasoning and “act” steps ( y←g(y,z) ) that refine answers, unrolled across 16 iterations. This demonstrates that recursive depth compensates for parameter count : computational depth of 42 effective layers through iteration enables generalization that massive parameter counts alone cannot achieve.
Self-improving program synthesis
Pourcel et al.’s SOAR framework fine-tunes LLMs on their own evolutionary search traces, achieving 52% on ARC-AGI-1 without hand-engineered DSLs ( Pourcel et al., 2025 ) . The key insight is that failed solution attempts contain valuable learning signal. By converting search failures into training data, the system discovers that “positive transfer between sampling and refinement tasks” enables progressive improvement across iterations. This suggests that learning how to search may matter more than searching with fixed capabilities.
Compression as intelligence
Liao and Gu’s CompressARC solves 20% of ARC-AGI-1 with 76K parameters and zero pretraining , optimizing description length purely at inference time ( Liao and Gu, 2025a ) . The approach formalizes puzzle-solving as code-golfing: finding the shortest program encoding correct solutions. This operationalizes the Solomonoff–Kolmogorov thesis that compression is intelligence ( Solomonoff, 1964 ) , demonstrating that MDL-based inference can substitute for massive pretraining.
Table 9. Mechanism-oriented directions highlighted by ARC Prize 2025 paper awards.
Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures. We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly. First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs. On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at $0.25 per task, and the orchestrator reaches 67.25% pass@2 at $0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute. Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking. The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.
The Abstraction and Reasoning Corpus (ARC) has become a prominent benchmark for evaluating general abstract reasoning and fluid intelligence in AI models. Yet standard ARC evaluation considers only a single capability: producing the correct output grid for a test input. We argue that this narrow format fails to evaluate the diversity of abilities that genuine abstract skill acquisition should enable. We introduce PotARCin, a benchmark that extends ARC by assessing understanding of a task's underlying abstract rule across five dimensions: Definition, Classification, Constrained Generation, Editing, and Inversion. PotARCin employs programmatic methods to generate new task instances and transform given inputs for a given ARC task, enabling dynamic generative sampling beyond fixed input-output pairs. Across five state-of-the-art models evaluated on the ARC-AGI-1 training set, we observe a 25-52 percentage-point performance gap between standard ARC evaluation and evaluation on PotARCin, and find that multi-dimensional evaluation reorders models that standard accuracy ranks alike. We further investigate effects of generative sampling, difficulty of corruption types, and questions of self-consistency, showing that models frequently contradict their own formalized rule even where they have stated it correctly. We also introduce P-ARC, a held-out hand-crafted test set, on which models achieve 1-8% accuracy across all five dimensions, underscoring the importance of more holistic evaluations of abstract reasoning capabilities.
We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions. Like its predecessors ARC-AGI-1 and 2, ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge. ARC-AGI-3 environments only leverage Core Knowledge priors and are difficulty-calibrated via extensive testing with human test-takers. Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%. In this paper, we present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.