cs.AIMar 9, 2026
SaveThe ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning
Organizations: TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology · Amazon (work done outside of Amazon)
Abstract
The Abstraction and Reasoning Corpus (ARC-AGI) has become a key benchmark for fluid intelligence in AI. This survey presents the first cross-generation analysis of 82 approaches across three benchmark versions and the ARC Prize 2024-2025 competitions. Our central finding is that performance degradation across versions is consistent across all paradigms: program synthesis, neuro-symbolic, and neural approaches all exhibit 2-3x drops from ARC-AGI-1 to ARC-AGI-2, indicating fundamental limitations in compositional generalization. While systems now reach 93.0% on ARC-AGI-1 (Opus 4.6), performance falls to 68.8% on ARC-AGI-2 and 13% on ARC-AGI-3, as humans maintain near-perfect accuracy across all versions. Cost fell 390x in one year (o3's 12/task), although this largely reflects reduced test-time parallelism. Trillion-scale models vary widely in score and cost, while Kaggle-constrained entries (660M-8B) achieve competitive results, aligning with Chollet's thesis that intelligence is skill-acquisition efficiency. Test-time adaptation and refinement loops emerge as critical success factors, while compositional reasoning and interactive learning remain unsolved. ARC Prize 2025 winners needed hundreds of thousands of synthetic examples to reach 24% on ARC-AGI-2, confirming that reasoning remains knowledge-bound. This first release of the ARC-AGI Living Survey captures the field as of February 2026, with updates at https://nimi-ai.com/arc-survey/
Figures & tables
Figure 1. Example task from each ARC-AGI version and the performance cliff across them. On the left side, we show one example from each ARC-AGI version, illustrating the increased complexity and requirements. On the right side, horizontal stacked bars show best AI performance (darker color) and the gap to human baseline (lighter color) for each benchmark version.
Figure 2. Representative ARC-AGI tasks illustrating two core reasoning categories. Left: Object-centric reasoning (Task f76d97a5, 3 training examples). The transformation extracts the colored checkerboard pattern from a gray background, requiring the system to identify “object” versus “background” without explicit segmentation cues. Right: Geometric transformation (Task c97c0139, 2 training examples). Red line segments define reflection axes around which cyan diamond shapes must be generated symmetrically. Both tasks require inferring abstract rules from minimal demonstrations and generalizing to novel configurations.
| Feature | ARC-AGI-1 | ARC-AGI-2 | ARC-AGI-3 |
|---|---|---|---|
| Release date | Nov 2019 | Mar 2025 | Jul 2025 |
| Format | Static | Static | Interactive |
| Number of tasks | 1,000 | 600 | Variable |
| Grid size | 30 30 | 30 30 | 64 64 |
| Color palette | 10 colors | 10 colors | 16 colors |
| Best AI performance | 90.5% † / 55.5% ‡ | 54.2% † / 24.03% ‡ | 12.58% |
Table 1. Comparison of ARC-AGI benchmark versions showing progressive increases in difficulty and the dramatic widening of the human-AI performance gap.
| Paradigm | System | Score (%) |
| Pure Paradigm Approaches | ||
| Induction | Neural-guided synthesis ( Ouellette, 2025 ) | 79.3 |
| Induction | Abductive solver ( Lim et al., 2024 ) | 66.0 |
| Induction | ConceptSearch ( Singhal and Shroff, 2025 ) | 50-58 |
| Induction | Greenblatt GPT-4o ( Greenblatt, 2024 ) | 50.0 |
| Induction | Graph constraints ( Xu et al., 2022 ) | 35.6 |
Table 2. Performance comparison across solution paradigms and representative systems on ARC-AGI-1 public evaluation set. Scores represent percentage of tasks solved exactly. Systems evaluated on 100 tasks provide more reliable estimates than those tested on smaller subsets. The paradigm choice establishes a performance ceiling, but within-paradigm variance often exceeds between-paradigm differences.
| System | Score | $/task | Note |
|---|---|---|---|
| GPT-5.2 Pro | 90.5% | 11.64 | 390 vs. o3; mainly fewer samples ( ARC Prize Foundation, 2025b ) |
| o3 | 87.5% | 4,500 | High-cost frontier baseline ( Chollet, 2024 ) |
| Gemini 3 Flash | 84.7% | 0.17 | Best cost-efficiency ratio ( ARC Prize Foundation, 2025b ) |
| Pang | 77.1% | 3.97 | Library-based program synthesis ( Pang, 2025 ) |
| NVARC | 24.03% | 0.20 | Competition winner (ARC-AGI-2) ( Sorokin and Puget, 2025 ) |
Table 3. Cost-performance points showing non-monotonic efficiency; frontier gains can reflect reduced parallelism, while algorithmic designs reach practical costs.
| Success factor | Description | Evidence |
|---|---|---|
| Guided search | Learned heuristics, neural nets, or LLMs prioritize promising candidates | Unguided search rarely exceeds 30%; guided systems reach 72–80% |
| Multiple hypotheses | Diverse candidates maintained; selected via demo consistency | Franzen’s Product-of-Experts (71.6%); Berman, Pang generate 10 candidates |
| Test-time adaptation | Fine-tuning, dynamic prompts, or adaptive search per task | Present in every system; 20–30 pt gain over frozen baselines ( Akyürek et al., 2025 ) |
| Hybrid reasoning | Neural perception + symbolic verification and composition | Pure transduction caps 40%; pure induction 79%; hybrids reach 80–94% |
| Robust perception | Multiple candidate representations (objects, patterns, regions) | 20–30 pt advantage over single-representation systems |
| Compositional primitives | Reusable operations composed for novel tasks | Enables systematic generalization; limited to 1–2 steps in current systems |
Table 4. Success factors shared by systems exceeding 70% on ARC-AGI-1.
| Anti-pattern | Description | Impact |
|---|---|---|
| Pure pattern matching | Learns correlations, not causal/compositional rules | 40% ceiling; collapses on harder compositions |
| Unguided exhaustive search | Enumerates programs without strong heuristics | Exponential blowup; usually 30% |
| Static representation | Locks into one abstraction (objects/regions/etc.) | Brittle failures; 25-pt deficit |
| No test-time adaptation | Fixed policy; ignores task demos as learning signal | 30–40% plateau; 20–30 pts behind adaptive |
| Monolithic architecture | No modular hybrid of perception + symbolic ops | 20–40 pts worse than hybrids |
| Frontier inefficiency | High score via expensive compute; hardness cliff persists | Costly per task; far from human efficiency |
Table 5. Common anti-patterns and their observed impacts.
Figure 3. ARC-AGI performance across benchmark versions. Public leaderboard (solid bars) allows unconstrained compute and API access; Kaggle competition (hatched bars) is constrained to 1.88 and 11.40 and 0.20/task (120 pts/38.90/task (2 pts/\times$ efficiency gap.
Figure 4. Cross-generation performance cliff: seven systems evaluated on both ARC-AGI-1 and ARC-AGI-2. Land’s cross-model ensemble ( Land, 2026 ) achieves the highest scores on both benchmarks but still drops 23%. Notably, Opus 4.6 shows the smallest single-model drop ( 26%, from 93.0% to 68.8%), compared to its predecessor Opus 4.5 ( 53%) and GPT-5.2 Pro ( 40%). Other single frontier models (solid bars) drop 40–63%, while Kaggle competition winners (hatched bars) drop 70–77%. The human baseline of 100% represents task-level solvability (every task solved by at least one person); average individual accuracy is 76% on ARC-AGI-1 and 60% on ARC-AGI-2 ( Chollet et al., 2025a ) .
Figure 5. Temporal evolution of ARC-AGI performance (2019–2025). Blue: ARC-AGI-1; Amber: ARC-AGI-2. The 2024 phase transition shows performance improving more in six months than the previous five years. The consistent 2.5–3 degradation from ARC-AGI-1 to ARC-AGI-2 across all approaches indicates fundamental compositional limitations.
Figure 6. Yearly progress of ARC-AGI scores from 2020 to 2026. Left: maximum score progression. ARC-AGI-1 scores rose from 78.8% (2020) through a plateau in 2023–2024 before surging to 93.0% in 2026 (Opus 4.6); ARC-AGI-2, introduced in 2024, jumped from 2.5% to 68.8% in under two years. Right: mean scores with confidence bands. The widening bands in 2024–2025 (ARC-AGI-1 mean 39–45%, –26) reflect growing methodological diversity as the field expanded from 2–3 papers per year to over 40. The persistent gap between maximum and mean scores indicates that top-performing techniques (large-scale inference, test-time training) have not yet diffused broadly across the research community.
Figure 7. Publication trends by research category. The 2024–2025 surge reflects the ARC Prize competition’s catalytic effect. Hybrid approaches dominate, reflecting recognition that neither pure neural nor symbolic methods suffice. Pure neural approaches declined after demonstrating consistent 20% ceilings.
Figure 8. Score distributions across ARC-AGI benchmark versions and evaluation sets. Public eval scores (self-reported on the public evaluation set) cover 38 systems on ARC-AGI-1 (median 25.6%, mean 34.1%) and 10 on ARC-AGI-2 (median 8.2%, mean 12.6%). Semi-private eval scores (official leaderboard) appear higher because the two groups represent different populations : frontier models (Opus 4.6, GPT-5.2 Pro, Gemini 3) submitted exclusively to the semi-private eval and never published public eval scores, skewing that distribution upward (18 systems on ARC-AGI-1, median 54.5%; 11 on ARC-AGI-2, median 26.0%). Among the 8 systems reporting both scores on ARC-AGI-1, semi-private results are 5–38 percentage points lower than public, consistent with the harder held-out set.
Figure 9. Cost per task versus accuracy on ARC-AGI-1 for the 10 systems (12% of surveyed papers) reporting cost data. Dotted lines are iso-efficiency curves (constant pts/0.0003–. The Pareto frontier (dashed) shows steep diminishing returns above $2/task. The human baseline (star) occupies the high-efficiency region that no AI system yet approaches.
Figure 10. Score vs. cost per task vs. estimated model scale across ARC-AGI-1 and ARC-AGI-2. Bubble area is proportional to parameter count (log-scaled) and black edges denote Kaggle-constrained entries. Parameter counts marked (est.) are community estimates; undisclosed architectural choices make cross-family comparisons approximate. On ARC-AGI-1, trillion-scale models vary widely in both score and cost: Opus 4.6 reaches 93.0% at 11.64/task, and Grok 4 scores 66.7% at 0.15–$0.20/task with 660M–8B parameters, while TRM reaches 6.3% on ARC-AGI-2 with just 7M parameters. This aligns with Chollet’s thesis underlying ARC-AGI ( Chollet, 2019 ) : “solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience.”
Figure 11. ARC breakthroughs filtered for evaluation rigor ( 100 tasks for ARC-AGI-1, 50 for ARC-AGI-2; see exclusion criteria in the Supplementary Material, Section 2). Circle markers denote public evaluations; diamond markers denote semi-private evaluations. The 2020–2023 timeline is compressed to emphasize the rapid progress in 2024–2026. On ARC-AGI-1, the rigorous maximum rose from 39.0% (Icecuber, 2020, public) to 93.0% (Opus 4.6, 2026, semi-private). ARC-AGI-2, introduced in late 2024, has seen its best rigorous score reach 68.8% (Opus 4.6, 2026).
Figure 12. Public versus semi-private ARC scores. Only 11 of 82 papers report both (87% transparency gap). On ARC-AGI-1, the mean public-to-semi-private drop is 10.8% (median 6.1%), with gaps ranging from 5.0% to 37.8%. On ARC-AGI-2, gaps are smaller (mean 2.0%), likely because the semi-private set was introduced recently and fewer systems have been evaluated on both splits. Points above the diagonal indicate higher semi-private than public scores.
| System | Score | Key innovation | Refinement mechanism |
|---|---|---|---|
| NVARC ( Sorokin and Puget, 2025 ) (1st) | 24.0% | Synthetic data mixing + 3-component ensemble (Qwen3 + TRM) | Data-generation loop: 266K puzzles from concept mixing; TRM hypothesis cycles; multi-component voting |
| ARChitects ( Franzen et al., 2025b ) (2nd) | 16.5% | LLaDA-8B masked diffusion LM with soft masking | 102 recursive refinement steps via selective re-masking of uncertain regions |
| MindsAI ( Cole et al., 2025 ) (3rd) | 12.6% | TTFT + AIRV on Llama 2 70B | Task-specific fine-tuning + augmentation-based ensemble voting |
| Land ( Land, 2026 ) (public SOTA) | 72.9% | Cross-model ensemble (GPT-5.2, Gemini 3, Opus 4.5) | Multi-model program synthesis + evaluator-based adjudication |
Table 6. ARC Prize 2025 winners (private ARC-AGI-2 test) and public leaderboard SOTA. All top systems implement refinement at different abstraction levels.
| Axis | Interpretation question |
|---|---|
| Efficiency | What computational resources are required per solved task (e.g., sampling budget, number of refinement steps, wall-clock time, monetary cost)? A method that succeeds only through massive test-time computation may approximate intelligent behavior without revealing a compact mechanism. |
| Adaptability | What changes at test time, and how? Does the system perform test-time adaptation (e.g., puzzle-specific training, search, hypothesis revision, tool use), and is that adaptation principled or primarily brute-force? |
| Compositional scope | Does the strategy generalize beyond the easiest regime? In particular, does it preserve competence as tasks demand deeper multi-step composition (e.g., ARC-AGI-2) and, eventually, interactive learning (ARC-AGI-3)? |
| Error signature | When the system fails, does it fail in interpretable, systematic ways that illuminate a missing primitive or decomposition step, or does it collapse unpredictably across task families? |
Table 7. Mechanism-relevant axes for interpreting ARC-AGI scores (RQ1).
| Dimension | “More with more” (scaling) | “More with less” (compression) |
|---|---|---|
| Primary driver | More samples, more compute, more data, larger ensembles | Effective theories: coarse-grained abstractions that screen off irrelevant detail ( Krakauer et al., 2025 ) |
| What transfers | Surface regularities and task-family coverage | Structural principles that generalize across substrates (analogy) ( Hofstadter and the Fluid Analogies Research Group, 2001 ) |
| Typical failure | Cliff-like breakdown under deeper composition or distribution shift | Graceful degradation: errors reveal missing primitives/decompositions |
| Interpretation | Capability accumulation (coverage/search) | Emergent intelligence (compression, explanation, reuse) ( Krakauer et al., 2025 ) |
Table 8. Two routes to performance. ARC-AGI progress is currently dominated by scaling, while human-like fluid intelligence is associated with conceptual compression.
| Direction | Summary |
|---|---|
| Recursive latent refinement | Jolicoeur-Martineau’s Tiny Recursive Model (TRM) achieves 45% on ARC-AGI-1 and 8% on ARC-AGI-2 with only 7M parameters, less than 0.01% of leading LLMs ( Jolicoeur-Martineau, 2025 ) . The architecture alternates between “think” steps ( ) that accumulate latent reasoning and “act” steps ( ) that refine answers, unrolled across 16 iterations. This demonstrates that recursive depth compensates for parameter count : computational depth of 42 effective layers through iteration enables generalization that massive parameter counts alone cannot achieve. |
| Self-improving program synthesis | Pourcel et al.’s SOAR framework fine-tunes LLMs on their own evolutionary search traces, achieving 52% on ARC-AGI-1 without hand-engineered DSLs ( Pourcel et al., 2025 ) . The key insight is that failed solution attempts contain valuable learning signal. By converting search failures into training data, the system discovers that “positive transfer between sampling and refinement tasks” enables progressive improvement across iterations. This suggests that learning how to search may matter more than searching with fixed capabilities. |
| Compression as intelligence | Liao and Gu’s CompressARC solves 20% of ARC-AGI-1 with 76K parameters and zero pretraining , optimizing description length purely at inference time ( Liao and Gu, 2025a ) . The approach formalizes puzzle-solving as code-golfing: finding the shortest program encoding correct solutions. This operationalizes the Solomonoff–Kolmogorov thesis that compression is intelligence ( Solomonoff, 1964 ) , demonstrating that MDL-based inference can substitute for massive pretraining. |
Table 9. Mechanism-oriented directions highlighted by ARC Prize 2025 paper awards.