Synthesizing symbolic models of human and machine reasoning at scale

Solim LeGris; Philipp Mondorf; Jinran Jin; Akshay K. Jagadish; Brenden M. Lake; Todd M. Gureckis

Figure 1: Reverse-engineering individual rules in a complex cognitive domain using programs . ( A ): Schematic task corpus and associated behavioral data. Each of the four representative ARC tasks

T_{i}

presents an agent with a small set of input–output training examples from which a hidden transformation rule must be inferred. From a test input, the agent must then construct the corresponding output grid (hidden with a grey square and “?” for illustrative purposes). We use H-ARC, a dataset of responses from over 1,700 participants to the 800 publicly available ARC-AGI-1 tasks. On the right of each task, we show 4 of the most frequent incorrect responses, and the correct response at the bottom. Under each grid, we show the associated response frequency (red for incorrect, green for correct). ( B ): Behavior-conditioned symbolic synthesis (BCSS). For each unique incorrect response, a large language model is prompted with a full specification of the domain-specific language, task grids, a reference program

P^{*}_{i}

implementing the target rule for task

T_{i}

, and the observed incorrect response (here, from a human participant). It proposes

k

independent sets

E_{1},\dots,E_{k}

of at most

j

line edits to that program. The schematic edit proposals

E_{l}

show LLM-sampled proposals which are then applied to

P^{*}_{i}

. Each proposed set of line edits is applied programmatically to the reference program to obtain a new program

E_{l}(P^{*}_{i})

. A custom parser rewrites the programs such that only causally relevant code is retained (i.e., unreferenced lines of code are removed). Each new program

E_{l}(P^{*}_{i})

is then executed with the test input as argument. Programs whose output reproduces the observed response are kept (green checkmark); all others are discarded (red cross). Because a response is often reachable by more than one unique program, BCSS returns a collection of programs per response rather than a single model. The individual response shown here is also shown surrounded by a red square to the right of task

T_{1}

in panel A. The constant THREE represents the color green.

Figure 2: Synthesized programs suggest people often infer rules simpler than the target rule for a given task . ( A ): A representative ARC task ( 94133066 ). Three training examples demonstrate the target rule—return the contents of the framed sub-grid under whichever of its eight rotations and reflections is consistent with the isolated colored cells scattered around it—followed by the test input for which participants construct an output grid (hidden with a grey square and “?” for illustrative purposes). ( B ): The reference program

P^{*}

implementing the target rule for this task. Because this task belongs to the evaluation split,

P^{*}

is itself model-synthesized rather than human-written. ( C ): Distribution of relative program length change,

(L(p)-L(P^{*}))/L(P^{*})

, over all synthesized programs, shown separately for the training and evaluation splits of ARC-AGI-1. Treatment programs (blue) are those conditioned on incorrect human responses; control programs (grey) are produced by an identical procedure conditioned instead on the correct output grid. Because edits replace lines in-place and causally irrelevant code is subsequently removed, synthesized programs can either shorten, or remain the same, but never lengthen. Therefore, the control condition controls for the shortening bias attributable to the procedure itself, against which the leftward displacement of the treatment distribution should be read. The dashed line marks the median shortening of modeled responses for the task displayed in this figure. ( D ): The correct response (top row) and the three of the most frequent incorrect responses to that task, with response frequencies below each grid (green, correct; red, incorrect). For each, we show the diff with

P^{*}

after applying BCSS, the length of the resulting program relative to

P^{*}

(blue, retained lines; gray, removed lines; red rule, reference length), which training examples the program reproduces (green, reproduced), and its error category. A single edit can account for a large simplification: for example, replacing the final extract over the eight candidate transforms by one fixed transform renders fourteen lines of the reference program inconsequential, and they are removed. Thus, the simpler resulting program hardcodes a transform instead of exhaustively searching over possible transformations.

Figure 3: Synthesized programs suggest people infer rules that only explain some of the data they observe . ( A ): A representative ARC task ( b230c067 ). Three training examples specify the target rule—recolor every shape blue except the least common one, which is recolored red—followed by the test input for which participants construct an output grid (hidden with a grey square and “?” for illustrative purposes). ( B ): A human-written reference program

P^{*}

implementing the target rule for this task. ( C ): Top, prevalence of each rule type among all modeled incorrect attempts. A program is a partial-match if it reproduces at least one, but not all, of the training outputs when executed on training inputs; an alternative rule if it reproduces all of them yet still fails the test example; and neither otherwise. Bottom, conditional on an attempt being explained by a partial-match program, the positions of the training examples it reproduces: the first-

k

or last-

k

example(s) shown on screen, a contiguous block of at least two examples that doesn’t include the first or last examples, or any other configuration. In both plots, bars give the posterior median attempt-weighted prevalence and error bars the 94% HDI, shown separately for the training (dark) and evaluation (light) splits of ARC-AGI-1; orange dots give the corresponding proportion for the task shown in panel A. ( D ): The correct response (top row) and three of the most frequent incorrect responses to that task, with response frequencies below each grid (green, correct; red, incorrect). For each, we show the diff with

P^{*}

after applying BCSS, the length of the resulting program relative to

P^{*}

(blue, retained lines; gray, removed lines; red rule, reference length), which training examples the program reproduces (green, reproduced), and its error category. The most frequent incorrect response is captured by a program that selects the rightmost object rather than the least common one, a rule that satisfies the first two training examples but not the third.

Category	Definition	Example
Attribute misselection	The correct operation is applied, but with a misselected attribute, parameter, or input value.	inferred color $\to$ hardcoded color
Scope restriction	The correct computation is applied, but only to a subset of the entities or regions where it should apply.	all objects $\to$ subset
Operator confusion	The correct class of operation is identified, but the wrong member within that class is selected.	rotate CW $\to$ rotate CCW
Transformation misselection	The correct targets are identified, but a transformation from an entirely different computational family is applied.	mirror $\to$ recolor
Sequencing	All required operations are identified, but they are composed in the wrong order or bound to the wrong targets.	( $A$ then $B$ ) $\to$ ( $B$ then $A$ )
Partial program	A subset of the required steps are executed correctly; one or more computational blocks are entirely missing.	extract $+$ tile $\to$ extract only

Table 1: Taxonomy of error categories used to classify synthesized programs. Each category identifies a way in which programs that reproduce incorrect responses deviate from the reference program implementing the target rule for the same task. Categories are defined over structural differences between the two programs, but the classifier is also conditioned on the response grids. Categories are defined as mutually exclusive and the “unclassifiable” catch-all makes the set exhaustive. The examples in the third column are schematic illustrations of the contrast between reference and response programs that defines each category. Every synthesized program was classified by an LLM given a reference program, task grids, the synthesized program, and its execution trace on the training and test inputs, with five independent classifications sampled per program (§ 3.4 ).

Figure 4: Synthesized programs reveal a restricted set of recurring deviations from target rules . ( A ): A representative ARC task ( 7fe24cdd ). Three training examples specify the target rule—tile the input into a

2\times 2

arrangement in which each quadrant holds the input rotated by a further 90 ∘ clockwise, starting from the input at the top left—followed by the test input for which participants construct an output grid (hidden with a gray square and “?” for illustrative purposes). ( B ): A human-written reference program

P^{*}

implementing the target rule for this task. ( C ): Prevalence of each error category among modeled incorrect attempts. An LLM classifier assigned every program one of seven cognitive categories, given the reference program, the task grids, the synthesized program and its execution trace (Table 1 for definitions). Bars give the posterior median of the attempt-weighted prevalence derived from a hierarchical Bayesian categorical model, and error bars the 94% HDI; the categories are mutually exclusive. Classification was performed on programs fitting responses to training split tasks only. ( D ): The correct response (top row) and three of the most frequent modeled incorrect responses to the task in panel A, with the response frequency below (green, correct; red, incorrect). For each, we show the diff with

P^{*}

after applying BCSS, the length of the resulting program relative to

P^{*}

(blue, retained lines; gray, removed lines; red rule, reference length), which training examples the program reproduces when executed on their inputs (green, reproduced), and its error category. The programs that reproduce these three incorrect responses diverge from

P^{*}

in different ways: omitting the rotations and tiling the input four times, carrying out reflections instead of rotations, or correctly carrying out the rule on a restricted portion of the input.

Figure 5: Machine errors are systematic and performance-dependent. ( A ): Average accuracy across responses of large reasoning models at different reasoning efforts and humans on the training set of ARC-AGI-1. Note that GPT-OSS-120B uses a mixture-of-experts architecture and thus only has 5.1 billion active parameters during inference. ( B ): Relative shortening of programs synthesized from model errors compared with programs for correct-output controls; all model configurations exhibit a simplicity bias, which is stronger for lower-performing configurations. ( C ): The proportion of error programs that partially match or fully reproduce the support examples increases with model performance. ( D ) Schematic of the classification analysis distinguishing human and machine errors using features derived from synthesized error programs. ( E ) Within-task ROC-AUC of three fitted classifiers provides evidence of varying degrees of discriminability of human and machine errors on held-out tasks, with less performant models better discriminated from people. Classifiers are trained using linearized program-level feature vectors. ROC-AUC is computed among the errors of each of the 56 shared tasks and averaged across tasks, with error bars showing 95% task-level bootstrap CIs. Panels B and C report results on the subset of 56 tasks where both reasoning language models and people generated incorrect responses (see Appendix S5 for similar trends across all errors). All machine responses are independently sampled, and we restrict comparisons to human first-attempt responses.

Figure S1: Responses within the two-cell noise tolerance. ( A – C ) Three representative ARC tasks. Left: the training examples and the test input shown to participants (the test output is hidden with a grey square and “?”). Right: for three response grids per task, the output of the best-matching synthesized program (left column) beside the participant responses it is accepted as reproducing (remaining columns); hatched cells indicate that fewer than two distinct responses share that program. The fraction to the left of each row gives the number of participants producing that response out of all participants who attempted the task (red, incorrect; green, correct). Within each row the grids differ in at most two isolated cells (surrounded by white) against the best-match program output.

Figure S2: The two-cell cutoff separates procedural noise from rule-level divergence. ( A ) Coverage of non-trivial incorrect responses as a function of the tolerance

\tau

, attempt-weighted, shown separately for the training (blue) and evaluation (grey) splits. Circles mark the value

\tau=2

used throughout (

52.5\%

and

46.2\%

respectively). ( B ) Share of mismatched cells that are scattered singletons—no mismatched cell among their four neighbors—as a function of

d

, the mismatch count of the best-scoring candidate program. The steep fall immediately past the cutoff shows that small residuals are isolated misclick-like cells, whereas higher values of

d

indicate structured regions/objects. ( C ) Distribution of cell-level distances between incorrect responses and the correct test output, per split. Grey bars (

\leq 2

mismatched cells) are the near-misses removed by the filter.

By incorrect responses (attempts)
	Training	Evaluation	Combined
Incorrect responses	4,293	5,156	9,449
Filtered (all trivial)	808 (18.8%)	891 (17.3%)	1,699 (18.0%)
Near-miss ( $\approx$ correct)	189 (4.4%)	251 (4.9%)	440 (4.7%)
Test-input copy (identity)	368 (8.6%)	377 (7.3%)	745 (7.9%)
Training-example copy	130 (3.0%)	197 (3.8%)	327 (3.5%)

Table S1: Trivial-error filtering by split (§ 3.1 ). Counts and percentages of incorrect responses removed before modeling, under the noise tolerance criterion: i) shapes must match, ii) grids of ten or more cells tolerate up to two mismatched cells, iii) smaller grids must match exactly and iv) copies of a training example are removed only on an exact match at any size. Percentages are over the full pre-filter pool of responses of each split.

	Training	Evaluation	Combined
Reference program	hand-written	synthesized	—
Proposal model	Gemini 3 Pro	Gemini 3.5 Flash	—
Tasks modeled	362	361	723
Unique non-trivial errors	2,695	3,566	6,261
Incorrect attempts	3,485	3,917	7,402
Program samples collected	40,365	53,490	93,855

Table S2: Modeling coverage and program yield by split. Unless specified, all reported numbers refer to errors modeled under the two-cell tolerance criterion. Attempt-weighted quantities weight each unique error by the number of participants who produced it. Exact-match reproduced is the percentage of errors with at least one program reproducing the grid with zero mismatched cells. Effective edits are counted after the parser removes causally irrelevant lines. Four training-split errors were missing from the synthesis run, so program samples cover 2,691 of the 2,695 training errors.

Rule type (all modeled attempts)
	Training	Evaluation	Combined
n (attempts / errors)	1,827 / 1,274	1,809 / 1,507	3,636 / 2,781
Neither	76.2% [74.9, 77.4]	78.7% [77.8, 79.6]	77.4% [76.6, 78.1]
Partial match	22.1% [20.8, 23.3]	19.7% [18.8, 20.6]	20.9% [20.2, 21.7]
Alternative rule	1.8% [1.3, 2.3]	1.6% [1.2, 2.1]	1.7% [1.4, 2.0]
Consistent with $\geq 1$ example	23.8% [22.6, 25.1]	21.3% [20.4, 22.2]	22.6% [21.9, 23.4]

Table S3: Rule types and order effects by split (attempt-weighted). Posterior median and 94% HDI. A program is a partial match if it reproduces at least one but not all training outputs, an alternative rule if it reproduces all of them yet still fails the test example, and neither otherwise. Order effects are conditional on a partial match and describe which of the training examples the program reproduces, ordered as models or participants saw them.

Figure S3: Relative length change as a function of reference complexity and error category. Partial residuals of relative length change against reference-program line count, with the posterior mean slope (dashed) and its 94% HDI (band). Annotations mark the predicted shortening at

L=7

,

14

and

28

lines.

Category	$\gamma$ [94% HDI]	$P(>0)$	% shorter [94% HDI]
Unclassifiable $\dagger$	$-0.369$ [ $-0.395$ , $-0.344$ ]	0.000	$+46.1$ [ $+44.2$ , $+48.0$ ]
Partial program	$-0.231$ [ $-0.264$ , $-0.198$ ]	0.000	$+32.3$ [ $+29.1$ , $+35.6$ ]
Transformation misselection	$-0.121$ [ $-0.150$ , $-0.091$ ]	0.000	$+21.3$ [ $+18.4$ , $+24.0$ ]
Attribute misselection	$+0.014$ [ $-0.007$ , $+0.034$ ]	0.893	$+7.8$ [ $+6.2$ , $+9.3$ ]
Scope restriction	$+0.045$ [ $-0.006$ , $+0.098$ ]	0.947	$+4.6$ [ $-1.1$ , $+10.4$ ]
Operator confusion	$+0.080$ [ $+0.041$ , $+0.118$ ]	1.000	$+1.2$ [ $-2.8$ , $+5.1$ ]

Table S4: Error-category effects on relative program length ( training split).

\gamma_{m}

is the deviation from the grand mean of the seven cognitive categories, which share a sum-to-zero prior; negative

\gamma

means more relative shortening than the average error type. The unclassifiable catch-all (

\dagger

) is held outside that constraint. “% shorter” is the absolute per-category prediction

\beta_{0}+\gamma_{m}

at mean success rate and mean reference length, summed per posterior draw (grand-mean shortening

\beta_{0}=9.2\%

); a negative value means the error programs are longer than the reference.

n=3{,}690

programs over 1,274 errors.

By incorrect responses (attempts)
	Qwen3.5-27B	GPT-OSS-120B
		high	mid	low
Responses with a valid grid	689	742	2,002	3,059
Filtered (all trivial)	114 (16.5%)	90 (12.1%)	264 (13.2%)	568 (18.6%)
Near-miss ( $\approx$ correct)	73 (10.6%)	54 (7.3%)	84 (4.2%)	61 (2.0%)
Test-input copy (identity)	22 (3.2%)	12 (1.6%)	110 (5.5%)	354 (11.6%)

Table S5: Trivial-error filtering by model configuration (§ 3.1 ). Incorrect model responses with a valid grid removed before modeling, under the same noise tolerance criterion applied to human responses (Supplementary Table S1 ), counted by response (top) and by unique incorrect grid (bottom). Percentages are over each configuration’s pre-filter total in the same block.

	Qwen3.5-27B	GPT-OSS-120B
		high	mid	low
Accuracy (%)	80.63	72.28	50.96	25.02
Tasks with $\geq 1$ error	145	173	284	344
Incorrect responses	806	1,153	2,040	3,119
with a valid grid	689	742	2,002	3,059
Attempts	575	652	1,738	2,491

Table S6: Evaluation and modeling yield by model configuration (across all tasks with errors). Accuracy is the mean over ten seeds on the 400 training tasks of ARC-AGI-1. Attempts correspond to every incorrect response that survived filtering; unique errors deduplicate identical grid outputs. Attempts modeled and errors modeled are the counts with at least one verified program reproducing the error within the two-cell noise tolerance criterion—with coverage as a percentage of attempts and of unique errors, respectively.

Simplicity bias (relative length change vs. correct-output control)
	Qwen3.5-27B	GPT-OSS-120B
		high	mid	low
$\beta$	$-0.147$	$-0.153$	$-0.211$	$-0.245$
94% HDI	[ $-0.185$ , $-0.112$ ]	[ $-0.187$ , $-0.119$ ]	[ $-0.236$ , $-0.185$ ]	[ $-0.268$ , $-0.221$ ]
$P(\beta<0)$	1.00	1.00	1.00	1.00
Rule type (all modeled attempts)

Table S7: Simplicity bias, rule types and order effects by model configuration (across all tasks with errors).

\beta

is the posterior mean relative shortening of error programs against correct-output controls, where negative values indicate shorter programs. Rule types and order effects are attempt-weighted population marginals from the hierarchical categorical model (§ 3.8.2 ); posterior median and 94% HDI, in percent. A program is a partial match if it reproduces at least one but not all training outputs, an alternative rule if it reproduces all of them yet still fails the test example, and neither otherwise. Order effects are conditional on a partial match and describe which training examples the program reproduces, ordered as models or participants saw them.

	Qwen3.5-27B	GPT-OSS-120B
		high	mid	low
n (errors)	197	236	713	989
Attribute misselection	61.0 [58.7,63.5]	58.4 [55.9,60.9]	45.7 [44.3,47.3]	38.0 [36.6,39.4]
Transformation misselection	13.0 [11.4,14.5]	12.6 [11.1,14.3]	11.0 [10.0,12.1]	10.1 [9.3,10.9]
Partial program	7.2 [5.4,8.8]	10.2 [8.2,12.2]	10.8 [9.6,12.0]	9.5 [8.4,10.6]
Scope restriction	3.2 [2.3,4.3]	3.9 [3.0,4.7]	2.7 [2.4,3.1]	3.0 [2.5,3.6]

Table S8: Error categories by model configuration (across all tasks with errors). Attempt-weighted population marginals from the hierarchical categorical model (§ 3.8.3 ); posterior median and 94% HDI, in percent. Error categories are the seven cognitive types of § 1.4 and unclassifiable (

\dagger

) is the catch-all category. n is the number of errors with at least one validly classified program; one medium-effort and two low-effort errors (7 and 3 programs) had no valid classification sample and are excluded, hence 713/989 rather than 714/991 in Supplementary Table S6 .

	Within-task ROC-AUC	Pooled ROC-AUC
L2 logistic regression	0.69 [0.63,0.76]	0.64 [0.59,0.70]
EBM	0.68 [0.61,0.74]	0.68 [0.62,0.74]
Hist. gradient boosting	0.65 [0.57,0.73]	0.68 [0.62,0.74]

Table S9: Human-versus-LLM source prediction (§ 3.7 ). Out-of-fold ROC-AUC on the 811 errors (144 human, 667 LLM) from the 56 matched tasks, for classifiers trained on primitive delta vectors from the errors on all tasks (651 human, 2,138 LLM). LLM errors are the positive class. Within-task ROC-AUC is computed among the errors of each task and averaged with equal task weight. Pooled ROC-AUC is computed over all errors at once. Point estimates with 95% percentile confidence intervals from 10,000 bootstrap resamples of whole tasks, shared across rows.

		LLM errors		Within-task ROC-AUC
	Acc. (%)	Train	Eval.	Logistic L2	EBM	Grad. boosting
Qwen3.5-27B	80.63	197	112	0.61 [0.52,0.69]	0.59 [0.49,0.69]	0.58 [0.48,0.67]
GPT-OSS (high)	72.28	236	128	0.60 [0.51,0.69]	0.57 [0.47,0.66]	0.63 [0.53,0.72]
GPT-OSS (mid)	50.96	714	221	0.68 [0.59,0.76]	0.69 [0.61,0.76]	0.68 [0.60,0.76]
GPT-OSS (low)	25.02	991	206	0.74 [0.67,0.82]	0.77 [0.69,0.84]	0.77 [0.68,0.84]

Table S10: Human-versus-LLM source prediction for each LLM configuration (§ 3.7 ). Each row reports classifiers trained on primitive delta vectors to distinguish human errors from those of a single configuration, with the same features, weighting scheme, task folds and evaluation tasks as Supplementary Table S9 . Train includes the configuration’s modeled errors on all tasks, and Eval includes those on the 56 matched tasks, on which out-of-fold within-task ROC-AUC is computed. Every row shares the same 651 human training errors and 144 human evaluation errors. Acc. is mean single-attempt accuracy (humans: 59.90%). Accuracy, reasoning effort and the number of training errors vary together across configurations, so differences between rows should not be attributed to accuracy alone. Point estimates with 95% percentile confidence intervals from 10,000 bootstrap resamples of whole tasks.

Figure S4: Language models display human-like error category profiles, but order effects are weaker and less consistent than in humans. All panels report values corresponding to the subset of 56 tasks where evaluated LRMs and people both generated at least one incorrect response. Human responses are restricted to first-attempt comparisons. ( A ) Prevalence of each error category among modeled incorrect machine errors. As with human results, an LLM classifier assigned every program one of seven cognitive categories or unclassifiable , conditioned on the reference program, the task grids, the synthesized program and its execution trace (Table 1 for definitions). Bars give the posterior median of the attempt-weighted prevalence derived from a hierarchical Bayesian categorical model, and error bars the 94% HDI. The categories are mutually exclusive. ( B ) Conditional on an attempt being explained by a partial-match program, we show the positions of the training examples it reproduces: the first-k or last-k example(s) shown, a contiguous block of at least two examples that does not include the first or last examples, or any other configuration. In both plots, bars give the posterior median attempt-weighted prevalence and error bars the 94% HDI.

Figure S5: Error-program synthesis prompt (§ 3.3 ). The model is given the full DSL specification, the task grids, the reference program implementing the target rule, and the respondent’s incorrect output grid, and returns a structured set of line edits under a fixed JSON schema.

Figure S6: Reference-program (solver) synthesis prompt (§ 3.2.1 ). Used to generate reference programs for evaluation -split tasks, which have no hand-written solver. The prompt is retrieval-augmented with five solved tasks and their human-written solvers from the training split.

Figure S7: Error-classification prompt (§ 3.4 ). The classifier sees the reference program, the task grids, the synthesized error program, and a side-by-side comparison of their behavior on the training examples, and returns one of the seven cognitive categories or the unclassifiable catch-all.

Figure S8: Evaluation prompt (§ 3.6 ). Used to collect machine responses on ARC tasks for comparison with the human data. The model is shown the training pairs and asked to complete the held-out pair.

Synthesizing symbolic models of human and machine reasoning at scale

Abstract

Figures & tables

Symbolic Reasoning

The Changing Role of Symbolic Methods in Artificial Intelligence

Forethought: Verifiable Reasoning from Neurosymbolic Primitive Programming

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition

Abstract

Symbolic Reasoning

Explore similar work

The Changing Role of Symbolic Methods in Artificial Intelligence

Forethought: Verifiable Reasoning from Neurosymbolic Primitive Programming

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition