ARC-AGI

Follow this topic Ask about this topic

ARC-AGI tests whether systems can infer a task’s underlying rule from a few examples, usually by transforming colored grids in ARC-AGI-1 and ARC-AGI-2 or adapting to interactive environments in ARC-AGI-3. Approaches include program synthesis with domain-specific languages, neural transformers with test-time training, search over candidate rules, and agents that build executable world models. Systems can score highly on particular public evaluations, but performance does not reliably transfer across benchmark versions or establish robust generalization to new tasks. A central tension is how to measure genuine abstraction while guarding against shortcuts and accounting for interaction efficiency, compute cost, and the persistent gap from human performance on harder settings.

Latest papers

24 papers7 papers in the last four weeks, up 133% on the four weeks before. 0.0% of all new papers.

How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3
Oct 25, 2026Shuzhi Liu, Archana Warrier, Dat Nguyen +3World Model Learning Human Behavior Modeling
Nanyang Technological University · TU Darmstadt · Basis Research Institute & Harvard University +1
Can LLMs Improve Their Own Harness? A Controlled Study on ARC-AGI-3
Oct 21, 2026Jens Lehmann, Sahar VahdatiLLM Agent Harnesses Agent Harness Optimization
Amazon∗ · Technische Universität Dresden · Universität Hannover +1
Nested Shared-Operator Recursion on ARC-AGI-1: An Empirical Study of HRM, TRM and URM
Oct 19, 2026Sara Neri, Eugenio Gaeta, Andrea Ceschini +2Recursive Reasoning Models Abstract Reasoning
University of Padova Padova, Italy · Universidad Politécnica de Madrid Madrid, Spain · Escuela Técnica Superior de Ingenieros de Telecomunicación Universidad Politécnica de Madrid Av. Complutense, 30 28040 Madrid, Spain +2
When the harness decides success: state that cannot be re-observed
Oct 19, 2026Raimondas LenceviciusLLM Agent Harnesses LLM Context Management
Microsoft
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Oct 8, 2026Haoyu Zhao, Zhengxu Yu, Zhiyuan He +5Continual Learning for LLM Agents Recursive Self-Improvement
University College London · Huawei Noah’s Ark Lab, UK · University of Liverpool
Tracing the Thoughts of a Coding Agent Playing ARC-AGI-3: Lessons for Continual Learning
Oct 8, 2026Chen Wu, Josh Passenger, Yin SongContinual Learning for LLM Agents AI Coding Agents
AWS
Kepler: Auditable World Models for ARC-AGI-3
Sep 30, 2026Wensen WuLong-Horizon Agent Evaluation ARC-AGI
Independent Researcher
Combining LLMs and Genetic Search for ARC-AGI-2
Sep 23, 2026Val DyachenkoLarge Language Model-Guided Optimization Genetic Programming
Implicit Rule Induction with Test-Time Task Embeddings in ARC-like Tasks
Sep 18, 2026Adrien Deliège, Claas Beger, Marc Van Droogenbroeck +1Test-Time Training Visual Reasoning
University of Liège · Santa Fe Institute
Scaling Discovery through Test-Time Communication
Sep 17, 2026Jongho Park, Vasilis Kontonis, Shivam Garg +2Multi-Agent Collaboration Multi-Agent Communication
UC Berkeley · Microsoft Research
Twin: Playing an Unknown Game with a Test-Time Digital Twin
Aug 14, 2026Alexy Skoutnev, Kirill Acharya, Gaston Longhitano +3Game-Playing Agents Code World Models
Yeshiva University · Stanford University · University of Southern California +1
BDH-CQ: In-Context Learning with Recurrent Latent Reasoning
Aug 10, 2026Björn Engdahl, Adrian Kosowski, Jan Chorowski +6Cost-Aware Inference Abstract Reasoning
New York University
Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3
Jul 30, 2026Jens Lehmann, Andrei Aioanei, Sahar VahdatiGame-Playing Agents AI Coding Agents
Dresden University of Technology · Amazon · TIB – Leibniz Information Centre for Science and Technology +1
ARCANA: A Reflective Multi-Agent Program Synthesis Framework for ARC-AGI-2 Reasoning
Jul 10, 2026Kunbo Zhang, Lei Fu, Zeyu Wang +2Program Synthesis Multi-Agent Collaboration
Columbia University New York, USA · Independent Researcher San Jose, USA · University of California, Los Angeles Los Angeles, USA +2
Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2
Jun 30, 2026Johan LandVisual Reasoning Multimodal Reasoning
Independent Researcher
Structural Grid Descriptors Predict Within-Task Solver Success on ARC-AGI
Jun 8, 2026Ayan PendharkarARC-AGI Algorithmic Reasoning
Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3
May 25, 2026Liew Keong HanGame-Playing Agents AI Agent Evaluation
Independent researcher.
Executable World Models for ARC-AGI-3 in the Era of Coding Agents
May 6, 2026Sergey RodionovWorld Models Coding Agents
SingularityNET
Multi-Perspective Transformers in ARC-AGI-2 Challenge
May 1, 2026Caleb Talley, Vedant Tibrewal, Seun Adekunle +3Visual Reasoning Product of Experts
ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
Mar 24, 2026ARC Prize FoundationARC-AGI AI Agent Benchmarks
ARC Prize Foundation
Procedural Refinement by LLM-driven Algorithmic Debugging for ARC-AGI-2
Mar 20, 2026Yu-Ning Qiu, Lin-Feng Zou, Jiong-Da Wang +2ARC-AGI Neuro-Symbolic Reasoning
Nanjing University, Nanjing, China
The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning
Mar 9, 2026Sahar Vahdati, Andrei Aioanei, Haridhra Suresh +1Abstract Reasoning ARC-AGI
TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology +1
ARC-TGI: Human-Validated Task Generators with Reasoning Chain Templates for ARC-AGI
Mar 5, 2026Jens Lehmann, Syeda Khushbakht, Nikoo Salehfard +4ARC-AGI Synthetic Data Generation
Dresden University of Technology Dresden, Germany · Amazon · TIB - Leibniz Information Centre Hanover, Germany +1
Symbol-Equivariant Recurrent Reasoning Models
Mar 2, 2026Richard Freinschlag, Timo Bertram, Erich Kobler +2Permutation-Equivariant Neural Networks Sudoku Solving
ELLIS Unit Linz, LIT AI Lab & Institute for Machine Learning, Johannes Kepler University Linz, Austria · Clinical Research Institute for Medical AI, Johannes Kepler University Linz, Austria

Oct 25, 2026J/K move · Enter open · S save

How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3

Shuzhi Liu, Archana Warrier, Dat Nguyen, Chengqi Zheng+2

Nanyang Technological University · TU Darmstadt · Basis Research Institute & Harvard University+1

World Model Learning Human Behavior Modeling ARC-AGI AI Agent Benchmarks

Open paper AskPDF arXiv

Learning through exploration and using newly acquired knowledge to achieve goals are central capabilities of general intelligence. Understanding these capabilities requires tracing how learners build, use, and revise world models during interaction. We construct the first dataset that records how humans discover and use the mechanisms of novel environments, using ARC-AGI-3 as a testbed: 144 trajectories from 64 participants across 25 public games, each pairing time-synchronized actions with think-aloud audio, plus 250 trajectories from large language model (LLM) agents with and without harnesses on the same games. We annotate mechanism triggers and recognition, action purposes, and errors and recovery, linking changes in understanding to the observations and actions that support them. We analyze characteristics of human cognition and compare them with those of LLM agents. Both groups anticipate mechanisms and use familiar analogies. Qualitative representations are more common in successful human trajectories than in successful LLM trajectories (50.0% versus 14.3%). Humans more often explore unfamiliar situations without a concrete hypothesis, while hypothesis testing is common in both groups. We also trace how learners recover through belief revision, replanning, and execution correction. As LLMs with well-designed harnesses reach saturated performance on public ARC-AGI-3 games, our cognitive analyses motivate next-generation benchmarks that challenge familiar priors, require accumulated evidence and abstraction, and test planning through subgoals, experimentation, and recovery. Our dataset is available at https://arc3-human-think-aloud.github.io/.