Discovering Structurally Plausible and Interpretable Cognitive Models with Large Language Models
Authors: Boyoung Kwon, Jiyoon Seo, Sang Ho Lee
Organizations: Graduate School of Data Science, KAIST · School of Digital Humanities and Computational Social Sciences, KAIST · Department of Brain & Cognitive Sciences, KAIST
Computational cognitive modeling has traditionally relied on manual model construction, which is a time-intensive process that requires substantial domain expertise. Although recent LLM-based approaches have begun to automate model generation, they often prioritize predictive accuracy at the expense of structural plausibility and parameter identifiability. We introduce DisCo, an LLM-guided framework that discovers interpretable symbolic cognitive models through evolutionary search with two structural components: a Validator that filters structurally invalid models, and a Cleaner that promotes parameter identifiability by removing redundant parameters. Across three behavioral tasks, DisCo consistently discovers models that achieve performance competitive with task-specific baselines. Ablation analyses show that the Validator is important for recovering psychologically meaningful structure. Without this component, models tend to violate behavioral constraints and show less consistent associations with external cognitive measures. These results suggest that LLM-driven model discovery, when coupled with appropriate structural constraints, can yield cognitively plausible models that go beyond predictive fit.
Figures & tables
Figure 1: Overview of the behavioral tasks. We utilize large-scale data from (A) the two-step reinforcement learning task, (B) the probabilistic reversal learning task, and (C) the recognition memory task.
Figure 2: Overview of the DisCo pipeline.
Task
Model
BIC Mean ± Std
BIC range
Planning
Model-Free
257.89
–
Hybrid
255.22
–
DisCo (Null)
240.54±8.07
[229.38, 249.36]
DisCo (MF)
221.16±1.75
[218.27, 223.57]
DisCo (Hybrid)
221.32±2.83
[216.20, 224.94]
Learning
TD Learning
76.06
–
Table 1: Comparison of BIC scores across tasks, evaluated on held-out test participants. For DisCo, results are reported across five independent runs (mean ± std and range). For the baseline models, BIC was computed once and is reported as a single score.
Task
Method
Valid runs
BIC Mean ± Std
BIC Range
Planning
DisCo
5/5
221.32±2.83
[216.20, 224.94]
GeCCo
3/5
231.65±16.58
[218.95, 255.06]
Learning
DisCo
5/5
74.49±2.20
[70.27, 76.71]
GeCCo
3/5
80.57±7.18
[73.96, 90.55]
Memory
DisCo
5/5
56.03±1.93
[53.46, 59.28]
GeCCo
3/5
837.91±25.77
[802.73, 863.73]
Table 2: Held-out BIC comparison between DisCo and GeCCo across tasks. Statistics are computed over runs in which the final selected model satisfied the validation criteria.
Baseline
Cleaner
Validator
Non-det.
Indep.
Future
Neg-idx
BIC Mean ± Std
MF
✓
✓
0%
0%
0%
0%
221.16±1.75
✓
14.3%
2.3%
0%
0.1%
199.41±38.05
✓
0%
0%
0%
0%
218.07±3.22
17.8%
2.5%
0%
0%
158.35±65.54
Hybrid
✓
✓
0%
0%
0%
0%
221.32±2.83
✓
6.8%
0.9%
0%
0.1%
195.09±33.26
Table 3: Ablation study on the planning task across Validator and Cleaner module combinations. Results are reported across five independent runs, each with 200 iterations. Violation rates are computed only among accepted candidate models.
Condition
BIC
Significant correlations
w/ Validator
224.94
w↔age−.19 , IQ+.19
β↔IQ+.17
221.83
w↔age−.22 , IQ+.12
β↔IQ+.15
216.20
w↔age−.23 , IQ+.16
β↔IQ+.14
Table 4: External validity of model parameters under ablated conditions, shown for representative runs and associations that illustrate key contrasts.
Prompt Configuration
BIC Mean ± Std
BIC range
Full prompt (all components)
221.16±1.75
[218.27, 223.57]
w/o Context
220.29±3.93
[216.24, 227.46]
w/o Constraints
220.23±1.90
[218.25, 223.36]
w/o Parsimony
224.61±3.50
[218.25, 228.17]
w/o Validation Method
219.77±2.03
[218.25, 223.36]
Table 5: Ablation study on prompt components. Each row removes one component from the full prompt. Lower BIC is better. Results are reported across five independent runs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Overview of the behavioral datasets and data-splitting procedure. Participants were divided into train (80%) and test (20%) sets using random sampling.
Asset
Source
Use in this work
License / Usage information
Gillan et al. (2016) dataset
Public dataset / paper
Planning task evaluation
No explicit license found; used for academic research.
Zorowitz et al. (2023) dataset
Public dataset / paper
Learning task evaluation
MIT license.
Cataldo et al. (2023) dataset
Public dataset / paper
Memory task evaluation
No explicit license found; used for academic research.
Qwen2.5-32B
Model repository
LLM-based model generation
See original model license / terms.
vLLM
Open-source software
LLM serving
See original software license.
Appendix
Table 6: External assets used in this work.
Task
Future
Independence
Negative
Non-
Log-likelihood
peeking
violation
indexing
determinism
mismatch
Planning
56
105
0
33
42
Learning
7
149
0
46
11
Memory
0
69
1
83
51
Appendix
Table 7: Frequency of validation-rule violations among GeCCo-generated candidate models. A single candidate may violate more than one rule.
Task
Method
Valid runs
BIC Mean ± Std
BIC Range
Planning
Baseline (Hybrid)
–
478.05
–
DisCo
5/5
384.91±6.71
[372.55, 393.06]
GeCCo
3/5
384.57±6.00
[377.50, 392.16]
Learning
Baseline (TD Learning)
–
134.08
–
DisCo
5/5
125.02±2.67
[123.29, 130.31]
GeCCo
3/5
125.41±1.76
[123.92, 127.88]
Appendix
Table 8: Held-out BIC comparison between baseline models, DisCo, and GeCCo after fitting model parameters on the full trial sequence. Statistics are reported as mean ± standard deviation over valid runs, excluding runs in which the final selected model failed to satisfy the validation criteria.
Task
Model
Valid.
Clean.
Seed
BIC
FDR-significant correlations
Planning
MF
✓
✓
42
218.27
α : age +.12 ; β : IQ +.16
123
221.24
β : IQ +.16
456
223.57
α : age +.13 ; β : IQ +.16 ; δ : OCI +.12
789
222.09
β : IQ +.13 ; γ : age +.19
1234
220.66
α : age +.12 ; β : IQ +.17
×
✓
42
123.48
n.s.
Appendix
Table 9: Full external validation results across random seeds. ✓ indicates the module was active; × indicates it was disabled.
School of Psychology, Georgia Institute of Technology · Princeton AI Lab, Princeton University · Center of Excellence for Computational Cognition, Georgia Institute of Techonology