Ceci n'est pas une pipe -- Toward a Symbolic Solver for the Abstraction and Reasoning Corpus Using Evolutionary Computation
Authors: Luca Favole, Giorgio Mongardi, Giovanni Squillero, Alberto Tonda
Organizations: Department of Control and Computer Engineering Politecnico di Torino Corso Duca degli Abruzzi 24, 10129, Torino, Italy · Applied Mathematics and Computer Science Universit´e Paris-Saclay, INRAE 22 place de l’Agronomie, 91120, Palaiseau, France · Complex Systems Institute of Paris CNRS 113 rue Nationale, 75013, Paris, France
The Abstraction and Reasoning Corpus (ARC) is a benchmark designed to assess the capability of Artificial Intelligence (AI) systems to generalize starting from a few samples, a task humans excel at. The ARC-AGI contests held since 2020 have been the focus of intense competition, with participants proposing approaches ranging from traditional symbolic AI to Transformer architectures. Despite a long history of successes, evolutionary algorithms (EAs) were notably absent from the approaches appearing in the main contest. As score-and-search program synthesis approaches, EAs suffer from issues related to choosing the proper representation in the Domain-Specific Language applied to the problem (for example, pixel-level or object-level), and distinguishing between two candidate solutions producing the same outputs. In this work, we present the first fully symbolic, evolutionary-based program synthesis approach submitted to the ARC-AGI-2 main contest. The main contributions are: evolving the grid parser and the output projector together with the program that transforms the grid, so that a task-specific language emerges during program synthesis; grounding the symbols the parser can extract in core knowledge theory from cognitive psychology; and an object-aware fitness that grades partially correct outputs beyond pixel-level discrepancies.
Figures & tables
Figure 1: Example tasks: Each arrow leads from an input grid to its output grid; in each row, the first pairs are the demonstration pairs and the last is the test pair. Sources: github.com/fchollet/ARC-AGI and github.com/arcprize/ARC-AGI-2 .
Figure 2: Different graph representations of the same grid. Parser A uses four-neighbor connectivity and explicitly detects the cavity enclosed by the blue object. Parser B uses eight-neighbor connectivity and omits cavity detection. Consequently, the two green objects merge into one and the cavity node disappears. Only the nodes and relations relevant to this example are shown.
Benchmark
Best run
All runs
ARC-AGI-1 training
197/400
219/400
ARC-AGI-2 evaluation
3/120
3/120
Table 1: Comparison between the best single run and cumulative task coverage across all available runs at pass@2 .
840s
3840s
Benchmark
Tasks
Fit.
@1
@2
Fit.
@1
@2
ARC-AGI-1 training
400
237
182
193
258
183
197
ARC-AGI-1 evaluation
400
146
96
103
164
103
110
ARC-AGI-2 training
1000
405
273
294
455
283
304
ARC-AGI-2 evaluation
120
7
3
3
10
3
3
ARC-AGI-2 tr., new
233
42
13
20
55
19
20
Table 2: Fitted tasks (Fit.) and tasks solved at pass@1 and pass@2 with 840 and 3840s per task: the best of three seeds at 840s , one seed at 3840s . Of the 1000 ARC-AGI-2 training tasks, 767 come from ARC-AGI-1; the last row covers the 233 that do not.
Figure 3: Comparison of the percentage of tasks solved on ARC-AGI with published systems. Our system, FairlySymbolic, was limited to 3840s per task on ARC-AGI-1; on ARC-AGI-2 the figure reports the result of the Kaggle entry. All systems were allowed two attempts per task, except on ARC-AGI-1 training, where icecuber, GPT-4, and DreamCoder used three attempts; and on ARC-AGI-1 evaluation, where GPT-4o used one attempt. It should also be noted that, on ARC-AGI-1 evaluation, the results for o3 refer to a pre-release version using six samples per task, and those for the ARChitects to the 2025 method .
Figure 4: Percentage of tasks solved at pass@1 on the 160-task ARC-AGI-1 subset used by ARGA. Our system used 1 800 s per task.
Rules
0
1
2
3
≥ 4
Fitted
90
296
103
80
62
Solved
100%
95%
77%
64%
47%
Table 3: Fitted ARC-AGI-1 training tasks that are also solved at pass@2 , by the number of rules in the program ( 120s per task, summed over three seeds).
Figure 5: For each of the 400 ARC-AGI-1 training tasks, the number of runs that solve it at pass@2 , over the 55 runs of the parameter study (earlier version of the solver, 120s per task) that allow programs of at least three rules.
Search
Fitted
Solved
Generations
Full fitness
212
184
2600
Coarse fitness
158
153
3900
Random search
98
107
3300
Random, no refinement
74
92
3500
Table 4: Weaker searches on the ARC-AGI-1 training tasks at 120s per task: fitted and solved ( pass@2 ) tasks and median generations per task, best of three seeds (the text compares seed means).
Figure 6: Fitted and solved ARC-AGI-1 training tasks against the number of restarts (earlier version of the solver). Solid: 120s per task, three seeds (dots) for 1 and 5 restarts, one otherwise. Dashed: 480s per task, one seed.
Concept
Before
After
TopBottom2D
70.0
100.0
SameDifferent
66.7
96.7
ExtractObjects
56.7
100.0
⋮
CompleteShape
13.3
96.7
Order
6.7
100.0
Table 5: Percentage of ConceptARC test inputs solved at pass@2 , before and after the extensions.
Figure 7: Percentage of ConceptARC test inputs solved. FairlySymbolic used two attempts with 16 voted searches (600 s per task in total), icecuber used three attempts, and the remaining systems used one.
Figure 8: Qualitative examples from ARC-AGI-1: (a) a successfully solved task; (b) a program fitting all demonstrations but failing on an unseen configuration; (c) a task requiring interpretation of a spatial color legend.
ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence) benchmarks the ability to infer and apply abstract rules from sparse examples.
The Abstraction and Reasoning Corpus (ARC-AGI) has become a key benchmark for fluid intelligence in AI. This survey presents the first cross-generation analysis of 82 approaches across three benchmark versions and the ARC Prize 2024-2025 competitions. Our central finding is that performance degradation across versions is consistent across all paradigms: program synthesis, neuro-symbolic, and neural approaches all exhibit 2-3x drops from ARC-AGI-1 to ARC-AGI-2, indicating fundamental limitations in compositional generalization. While systems now reach 93.0% on ARC-AGI-1 (Opus 4.6), performance falls to 68.8% on ARC-AGI-2 and 13% on ARC-AGI-3, as humans maintain near-perfect accuracy across all versions. Cost fell 390x in one year (o3's 4,500/tasktoGPT−5.2′s12/task), although this largely reflects reduced test-time parallelism. Trillion-scale models vary widely in score and cost, while Kaggle-constrained entries (660M-8B) achieve competitive results, aligning with Chollet's thesis that intelligence is skill-acquisition efficiency. Test-time adaptation and refinement loops emerge as critical success factors, while compositional reasoning and interactive learning remain unsolved. ARC Prize 2025 winners needed hundreds of thousands of synthetic examples to reach 24% on ARC-AGI-2, confirming that reasoning remains knowledge-bound. This first release of the ARC-AGI Living Survey captures the field as of February 2026, with updates at https://nimi-ai.com/arc-survey/
Sahar Vahdati, Andrei Aioanei, Haridhra Suresh +1
TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology +1
We present ARCANA, a collaborative multi agent framework for solving ARC AGI 2 tasks under strict test time and hardware constraints. ARCANA decomposes each task into iterative perception, hypothesis generation, symbolic execution, and reflective refinement. A perceptual grounding agent builds object centric scene graphs from raw grids, a latent program policy proposes diverse DSL programs, a symbolic executor verifies candidates on demonstrations, and a reflective agent synthesizes failure driven feedback for the next turn. These agents communicate through a shared differentiable blackboard and are scheduled by a learned meta controller. The design combines structured program search with adaptive multi turn correction, improving reasoning efficiency and solution quality on challenging abstract transformation tasks.
Kunbo Zhang, Lei Fu, Zeyu Wang +2
Columbia University New York, USA · Independent Researcher San Jose, USA · University of California, Los Angeles Los Angeles, USA +2
We study structured abstraction-based reasoning for the Abstraction and Reasoning Corpus (ARC) and compare its generalization to test-time approaches. Purely neural architectures lack reliable combinatorial generalization, while strictly symbolic systems struggle with perceptual grounding. We therefore propose a neuro-symbolic architecture that extracts object-level structure from grids, uses neural priors to propose candidate transformations from a fixed domain-specific language (DSL) of atomic patterns, and filters hypotheses using cross-example consistency checks. Instantiated as a compositional reasoning framework based on unit patterns inspired by human visual abstraction, the system augments large language models (LLMs) with object representations and transformation proposals. On ARC-AGI-2, it improves over the base solver LLM (Grok-4, 16.0%) to 24.4% on the public evaluation set, and to 30.8% when combined with ARC Lang Solver via a meta-classifier. These results support the claim that in the evaluated settings, allocating test-time compute to object abstraction, neural-guided transformation proposal, and consistency filtering can improve accuracy over unstructured sampling-based scaling without task-specific finetuning of the system. We open-source the code: (https://github.com/CoreThink-AI/arc-agi-2-reasoner).