OptScientist: Automated Discovery of Optimization Laws
Organizations: Plokerm AI Lab · Xiamen University · University of Tübingen · The University of Hong Kong
Abstract
Many scientific phenomena are best understood as outcomes of underlying optimization problems, yet automated scientific discovery has largely focused on recovering equations that describe observed input-output patterns. We introduce OptScientist, a novel automated framework for discovering compact symbolic optimization laws directly from data. OptScientist comprises (i) a model that infers symbolic optimization problems by first learning to represent data as an optimization problem and then extracting its symbolic form, and (ii) an LLM agentic system embedding the model in a discovery process that integrates theoretical priors from scientific literature, discovers candidate laws, abstracts their underlying mechanisms, and suggests new data to collect. We first evaluate OptScientist on OptFeynman, a novel benchmark of 99 physics-derived optimization problems, where OptScientist recovers ground-truth laws for over 80% of the problems and remains robust to noisy contextual information, while substantially outperforming machine-learning, symbolic, and LLM-based model-discovery baselines. On human risky choice and central-bank monetary policy, OptScientist discovers compact symbolic optimization laws that achieve held-out predictive performance competitive with leading baselines, while using few parameters and substantially fewer tokens than LLM baselines. Overall, OptScientist extends automated scientific discovery to uncovering optimization laws that generate observed phenomena, enabling automated theory building across the physical, biological, behavioral, and social sciences.
Figures & tables
| Decision variable | Effective ground truth | Effective recovered expression |
|---|---|---|
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Discrete | Continuous |
|---|---|---|
| OptLearner architecture | ||
| Number of BL layers | 2 | 2 |
| BL layer widths | ||
| Hidden-layer branches | ||
| Default final-layer branches | ||
| activation | Identity | Identity |
| Decision type | Representation | Candidate set | Clean | Noisy-context |
|---|---|---|---|---|
| Discrete | OptScientist law | |||
| Discrete | OptLearner score | |||
| Continuous | OptScientist law | |||
| Continuous | OptLearner score | |||
| Continuous | OptScientist law | |||
| Continuous | OptLearner score |
| Condition | Representation | Exact order | Median [Q1,Q3] | Pairwise | |
|---|---|---|---|---|---|
| Clean | 10 | OptScientist law | |||
| Clean | 10 | OptLearner score | |||
| Clean | 50 | OptScientist law | |||
| Clean | 50 | OptLearner score | |||
| Clean | 100 | OptScientist law | |||
| Clean | 100 | OptLearner score |
| Condition | Representation | KL | Median [Q1,Q3] |
|---|---|---|---|
| Clean | OptScientist law | ||
| Clean | OptLearner score | ||
| Noisy-context | OptScientist law | ||
| Noisy-context | OptLearner score |
| Condition | Representation | Exact order | Median [Q1,Q3] | Pairwise | Exact |
|---|---|---|---|---|---|
| Clean | OptScientist law | ||||
| Clean | OptLearner score | ||||
| Noisy-context | OptScientist law | ||||
| Noisy-context | OptLearner score |
| Condition | Representation | Prob. MAE | MAE median [Q1,Q3] | KL | KL median [Q1,Q3] |
|---|---|---|---|---|---|
| Clean | OptScientist law | ||||
| Clean | OptLearner score | ||||
| Noisy-context | OptScientist law | ||||
| Noisy-context | OptLearner score |
| Decision variable | Effective ground truth | Effective recovered expression |
|---|---|---|
| Decision type | Condition | Initial | Final | Perfect recovery | |
|---|---|---|---|---|---|
| Continuous | Clean, | ||||
| Continuous | Clean, | ||||
| Continuous | Clean, | ||||
| Continuous | Noisy-context, | ||||
| Continuous | Noisy-context, | ||||
| Continuous | Noisy-context, |
| Condition | Method | Exact order | Exact | Pairwise | KL | |
|---|---|---|---|---|---|---|
| Clean | GBDT | 99 | 0.327 | 0.0% | 0.844 | 0.1679 |
| Clean | LR | 99 | 0.716 | 17.2% | 0.919 | 0.0859 |
| Clean | MLP | 99 | 0.420 | 6.1% | 0.859 | 0.1580 |
| Clean | RF | 99 | 0.206 | 0.0% | 0.794 | 0.5992 |
| Clean | XGBoost | 99 | 0.337 | 0.0% | 0.847 | 0.0833 |
| Clean | SR | 99 | 0.076 | 0.0% | 0.743 | 0.8855 |
| Condition | Method | KL | ||||
|---|---|---|---|---|---|---|
| Clean | GBDT | 99 | 0.6362 | |||
| Clean | Neural EBM | 99 | 0.0098 | |||
| Clean | MLP | 99 | 0.6349 | |||
| Clean | RF | 99 | 0.6408 | |||
| Clean | XGBoost | 99 | 0.6399 | |||
| Clean | SR | 99 | 0.6414 |
| Variant | Parameters | Accuracy | AUC | Prob. MSE | NLL | KL | ECE |
|---|---|---|---|---|---|---|---|
| Direct OptScientist | 5 | ||||||
| AutoF OptScientist | 7 |
| Model | Params. | MSE | MAE | Exact acc. | Direction acc. | |
|---|---|---|---|---|---|---|
| MLP | 5,057 | |||||
| Random forest | 134,000 | |||||
| XGBoost | 6,600 | |||||
| Neural EBM | 5,121 | |||||
| GBDT | 6,600 | |||||
| Symbolic regression | 6 |
| Model | Max tokens | Output tokens | Held-out acc. | Acc./token |
|---|---|---|---|---|
| LLM-only | 8,192 | 17,981 | 83.354% | 4.636 |
| 12,000 | 15,771 | 82.956% | 5.260 | |
| 16,384 | 16,048 | 83.173% | 5.183 | |
| 24,000 | 16,124 | 83.259% | 5.164 | |
| GeCCo ( Rmus et al., 2026 ) | 8,192 | 35,947 | 83.777% | 2.331 |
| 12,000 | 37,317 | 83.487% | 2.237 |
| Model | Max tokens | Output tokens | Held-out acc. | Acc./token |
|---|---|---|---|---|
| LLM-only | 8,192 | 18,381 | 44.318% | 2.411 |
| 12,000 | 16,691 | 49.432% | 2.962 | |
| 16,384 | 17,361 | 47.727% | 2.749 | |
| 24,000 | 18,142 | 40.341% | 2.224 | |
| GeCCo ( Rmus et al., 2026 ) | 8,192 | 23,188 | 64.205% | 2.769 |
| 12,000 | 26,350 | 56.818% | 2.156 |
Autonomous Scientific Discovery
Autonomous scientific discovery develops AI systems that generate, test, and refine scientific hypotheses using data, computation, and experiments.
See all 128 papers