Authors: Marco Virgolin, Harry Desmond, Ester Bergantin, Giorgio Morales, John W. Sheppard, Luigi Romano, Alessandro Lucantonio
Organizations: Unlayer AI, Amsterdam, the Netherlands · Institute of Cosmology and Gravitation, University of Portsmouth, Portsmouth, UK · Department of Mathematics, University of Trento, Trento, Italy · Aston Centre for Artificial Intelligence Research and Application, Aston University, Birmingham, UK · Gianforte School of Computing, Montana State University, Bozeman, USA · Department of Electrical Engineering, Linköping University, Linköping, Sweden · Department of Mechanical and Production Engineering, Aarhus University, Aarhus, Denmark
The search for better models, whether expressed as analytical equations or neural architectures, has increasingly shifted from manual design toward data-driven automation. While a multitude of algorithms exist, ranging from exhaustive search to AlphaEvolve, recent advances in general-purpose coding agents such as Claude Code and OpenAI Codex raise the question: could providing such systems with a "model discovery recipe" be enough to discover or improve models? We present automodel, an agent skill for general model discovery. With case studies in astrophysics, cardiovascular dynamics, precision agriculture, traffic flow, and vehicle tire dynamics, we show how commercial agents equipped with automodel can successfully identify novel and scientifically-relevant models. Our findings suggest that encoding model-discovery methodology as a portable agent skill can provide a simple, interactive, powerful and broadly applicable alternative to domain-specific automated discovery pipelines. The skill is available at: https://github.com/unlayer-ai/automodel.
Figures & tables
Figure 1: The four phases of automodel . In phase 3, parallel agents are spawned to test different structural hypotheses through sequential attempts, while a coordinating agent compares results and redirects the next meta-iteration. The score and final check are defined for each application during setup.
dataset 1 (Sérsic)
dataset 2 (cmodel)
Function
C
Δ DL
Δ NLL
Δ DL
Δ NLL
Best ESR
10/9
+196.8
+241.5
+189.7
+235.1
Saunders
21
+169.4
+187.9
+144.6
+164.3
Bernardi
23
0
0
0
0
A2 (this work)
13
−23.5
+9.2
+76.3
+109.1
A5 (this work)
19
−36.5
−21.9
−15.7
−5.7
Table 1: Goodness-of-fit statistics (description length DL and negative log-likelihood NLL) for the stellar mass function, defined relative to the Bernardi function (so negative numbers indicate superiority to Bernardi), for the two independent versions of the SMF dataset. C is the number of operators in the expression (“complexity” as defined by ESR), which is independent of the dataset except for the best ESR, which is different between the two datasets. A2 and A5 are the automodel functions discussed in the text; the best ESR function and the two literature forms are shown for reference.
Figure 2: The best ESR function, the Bernardi function, and the best function found by automodel (A5), overplotted on the binned SMF data. Top: the number density ϕ against stellar mass, both logarithmic, with the measurements as points. Bottom: residuals in units of the Poisson measurement uncertainty. A5 and Bernardi both track the data to within about 2σ , as expected for a high-quality fit, while the complexity-limited ESR function is visibly worse.
Figure 3: Pareto fronts of Δ DL (blue) and Δ NLL (red) against complexity, relative to the best value in each panel, for the two versions of the stellar mass function dataset. Connected points are the best ESR function at each complexity, circled markers are the Saunders and Bernardi literature forms and stars are the functions found by automodel , labelled A1–A7 by increasing complexity. The lower panels zoom in on the Δ DL competition, the dashed horizontal line marking Bernardi’s value. The gold-rimmed A5 function beats Bernardi by description length on both datasets. (Markers can coincide: two of the functions are at complexity 19, and A5’s Δ DL and Δ NLL markers lie exactly on top of one another on dataset 2.)
Figure 4: Performance of the structural model identified by automodel . Panel A : MAPE of the calibrated original CircAdapt formulation and the selected model. P1–P3 denote the development patients and P4 the held-out patient. Panel B : mean development MAPE of the selected model, the calibrated original formulation, and the five finalists retained from the subsequent search of 16 additional structural mechanisms. Lower MAPE indicates better agreement with the patient-specific measurements.
Figure 5: Results obtained for Field A. (a) Delineated management zones and (b) Aligned approximated N-response curves for each MZ (N-rate xNr vs. relative yield ry ) [ 44 ] .
Figure 6: Results obtained for Field B. (a) Delineated management zones and (b) Aligned approximated N-response curves for each MZ (N-rate xNr vs. relative yield ry ) [ 44 ] .
MZ
Method
Skeleton e(x)
rˉ
1
automodel
c1tanh(c2x+c3−c4(x−c5)2+1)+c6
0.0043
SeTGAP
c1tanh(c2x+c3)+c4
0.0113
Quadratic-plateau
c1+c2min(c3,x)2
0.5411
Exponential
c1(1−ec2+c3x)+c4
0.0636
2
automodel
(c1+c2x)((c3+c4x)2+1)−c5+c6
0.0067
SeTGAP
c1tanh(c2x+c3)+c4
0.0185
Table 2: Field A: Symbolic skeletons per MZ and mean error rˉ over all sites of the zone.
MZ
Method
Skeleton e(x)
rˉ
1
automodel
(c1+c2x)((c3x+c4−c5(x−c6)2+1)2+1)−c8+c7
0.0021
SeTGAP
c1x4+c2x3+c3x2+c4x+c5
0.0139
Quadratic-plateau
c1+c2min(c3,x)2
0.4044
Exponential
c1(1−ec2+c3x)+c4
0.1401
2
automodel
(c1+c2x)((c3x+c4−c5(x−c6)2+1)2+1)−c8+c7
0.0016
SeTGAP
c1(c2sin(c3x)+c4)−1+c5
0.0204
Table 3: Field B: Symbolic skeletons per MZ and mean error rˉ over all sites of the zone.
Figure 7: Comparison between the Pacejka Magic Formula baseline (dashed blue lines), the selected natural model (orange lines), and the experimental data.
Model
Macro-parameter
Functional dependence
Secondary parameter
Value
Pacejka baseline
B(Fz)
exp(b0+b1q)
b0 ( − )
2.46
b1 ( kN−1 )
−0.136
C(Fz)
c0+c1q
c0 ( − )
1.39
c1 ( kN−1 )
0.00444
D(Fz)
d0+d1q
d0 ( − )
0.558
d1 ( kN−1 )
−0.0318
Table 4: Identified load-dependent parameters of the Pacejka baseline and the natural asymmetric hyperbolic-tangent model. The normalized load variable is q=(Fz−Fz,ref)/(1000N) , with Fz,ref=2934N . The coefficients multiplying q represent sensitivities per unit load variation expressed in kN .
Dataset
Model
RMSE
median∣e∣
R2
Training, bins 1–3
Pacejka baseline
0.0592
0.0367
0.9869
Natural model
0.0521
0.0317
0.9899
Validation, bin 4
Pacejka baseline
0.0467
0.0331
0.9900
Natural model
0.0384
0.0280
0.9932
Test, bin 5
Pacejka baseline
0.0583
0.0517
0.9835
Natural model
0.0546
0.0463
0.9855
Table 5: Prediction performance of the Pacejka baseline and the natural asymmetric hyperbolic-tangent model. Model parameters are identified jointly using load bins 1–3. Bin 4 is used for validation, whereas bin 5 is retained as the final test set. RMSE and median absolute error are evaluated for the normalized lateral force μy=Fy/Fz .
Figure 8: Measured I80 fields and predictions from the five selected automodel corrections. Density is shown in the top row and velocity in the bottom row; the first column contains the measurements and the remaining columns the model predictions. The outlined regions in the measured density panel distinguish the development and held-out time intervals.
Model
Eρ
Ev
Edata (rank)
Δbase
Greenshields
0.294
0.313
9.230 (10)
–
AM–Greenshields
0.255
0.277
7.092 (3)
+23.17%
IDM
0.258
0.270
6.957 (2)
–
AM–IDM
0.256
0.277
7.117 (4)
−2.30%
Weidmann
0.265
0.288
7.651 (8)
–
AM–Weidmann
0.265
0.281
7.459 (7)
+2.51%
Table 6: Held-out I80 prediction errors. Lower is better for all error columns. Parentheses give rank by full-precision test Edata across the ten models. Δbase is the percentage reduction in test Edata relative to the model’s own classical baseline; positive values indicate improvement. AM denotes automodel .
Frameworks that use large language models for scientific discovery typically rely on a fixed, human-designed algorithm that decides what the model sees at each step, leaving the model only the role of proposer. The model knows nothing of the search beyond what it is shown. As models grow more capable, a question arises: does a search strategy chosen by a human before the run scale better than promoting the model from proposer to planner and letting it own the search? The Bitter Lesson suggests that choosing the strategy in advance is the kind of hand-designed structure that general methods eventually outscale. We introduce AgentDiscover, in which a coding agent plans the search using its context as working memory, runs experiments, and records every attempt in a database of ideas, candidates, and their relations. This database serves as the agent's long-term memory and is structured so that the selection rules of classical algorithms such as MAP-Elites and Monte Carlo tree search each reduce to a single query, which the agent is free to use, combine, or replace. A server maintains the database and steers the agent after every submission, keeping it on course over long runs. In our experiments, AgentDiscover is more cost-efficient than existing frameworks, reaching better scores at lower cost. On tasks in kernel engineering, biology, algorithm design, and mathematics, AgentDiscover outperforms prior discovery frameworks. Its programs would have placed first among human competitors in seven past AtCoder heuristic contests, and on eleven mathematical and systems optimization tasks it matches or exceeds every baseline that uses the same model. Our code is available at https://github.com/mhdfb/AgentDiscover.
Coding agents are increasingly used as general-purpose problem solvers, but their flexibility does not by itself confer the domain expertise needed for specialized tasks. Recent work addresses this through agent skills: reusable workflows, and code, that augment agents with domain-specific capabilities. Most skills today are hand-crafted, and existing evolutionary approaches optimize low-level artifacts (e.g. prompts & code) that are tightly coupled to specific models and tasks. We introduce EvoSkill, a self-evolving framework that automatically discovers and refines agent skills through iterative failure analysis. EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders. A Pareto frontier of agent programs governs selection, retaining only skills that improve held-out validation performance while the underlying model remains frozen. We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S.\ Treasury data, where it improves exact-match accuracy by 7.3% (60.6% → 67.9%); and SealQA, a search-augmented QA benchmark with noisy retrieval, where it yields a 12.1% gain (26.6% → 38.7%). We also investigate the zero-shot transfer capabilties of skills evolved on one task to the other; in particular: skills evolved from SealQA transfers zero-shot to BrowseComp, improving accuracy by 5.3% without modification demonstrating that skill-level optimization produces transferable capabilities beyond the training task.
Large language model coding agents increasingly perform open-ended data modeling and analysis. These agents are stochastic and adaptive, and therefore their autonomous model discovery behavior cannot be adequately characterized by a single benchmark run. In this work, we propose an experimental design and analysis framework for systematically evaluating this discovery process, quantifying its variability, and identifying important factors. The proposed framework treats these agents as stochastic model-discovery operators, which map task-specific discovery data and an optimization target to a fitted model. Specifically, we investigate two such operators, Codex and Claude Code, under controlled experimental factors including agent's reasoning effort, task, optimization metric, and composition of training data. For each agent-task-metric combination, regression models and inference are conducted for multiple responses such as output quality, dollar cost, wall-clock time, and process complexity. Furthermore, we develop a utility-aligned canonical decomposition to characterize the dominant direction of the reasoning-effort effect and to assess whether that direction aligns with a performance-cost utility direction. The proposed framework is demonstrated on a testbed of networked word-forming games with insightful findings on reasoning effort with respect to cost and process complexity.
Hao He, Xueying Liu, Chris J. Kuhlman +1
Department of Statistics, Virginia Tech · Department of Statistical Science, Baylor University · Advanced Research Computing, Virginia Tech +1