LLM sequential decision making under uncertainty in biochemical domains

Mattias Akke; Soojung Yang; Jurgis Ruža; Sathya Edamadaka; Rafael Gómez-Bombarelli

cs.AISep 27, 2026

LLM sequential decision making under uncertainty in biochemical domains

Authors: Mattias Akke, Soojung Yang, Jurgis Ruža, Sathya Edamadaka, Rafael Gómez-Bombarelli

Organizations: Lund University · Stanford · MIT

Abstract

Large language models (LLMs) are increasingly used to drive scientific discovery. Understanding how LLMs make decisions from new data and memory of the literature is vital before trusting them to design experiments under tight experimental budgets. However, their decision strategies are invisible in the current performance scores used to evaluate research agents. Here, we benchmark five frontier LLMs in a Bayesian Optimization setting against published statistical baselines on seven combinatorial datasets spanning protein engineering, reaction optimization, molecular design, peptide self-assembly, and catalysis. Performance is paired with direct measurements of model beliefs and actions, enabling highly resolved behavior analysis. A prompt ablation that progressively strips context separates memorization from chemical reasoning and from bare categorical optimization. Prior chemical knowledge helps in expectation, but with high variance and occasionally even harms performance. No configuration tested decisively beats a mean statistical baseline across domains. Belief-movement and Martingale diagnostics, corrected here for a measurement-noise bias that mislabels rational agents as irrational, show that models overreact to incoming data rather than entrenching on their priors in the contexts studied here. Interestingly, while LLM actions are exploitative, models sincerely intend to explore and consistently act on that intent. This failure is a competence gap arising from context-stickiness. Removing in-context history restores exploration, indicating that priors and data must be decoupled to achieve effective LLM-driven discovery.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of each experimental setup. (a) The standard Bayesian optimization loop. An initial batch of validated data trains a model to predict fitness over the search space of candidates. An acquisition function then selects a batch of next experiments. Experiments (here simulated using a lookup table) validates selected candidates. (b) The LLM replaces the optimization and selection steps. Three different LLM configurations with increasing capabilities are tested. The no-reasoning prompt includes prior information about the task and a history of validated candidates each turn. It provides the next batch of candidates directly. The reasoning models are asked to strategize with reasoning turned on. The agent has a tool wrapping the statistical model in (a), offering an independent mean and uncertainty estimate about the candidates. Additionally, no history agent probes the behavioral impact of the in-context history (Section 2.5 ). (c) All experiments use a single prompt template. Three different prompt modes with decreasing prior information about the task at hand separate memorization, domain knowledge, and optimization ability. (d) Performance is scored using normalized AUC by two different metrics to enable comparisons between datasets.

Panel of Figure 2 — Figure 2: Prior knowledge helps on average but is unreliable. (a) Change in normalized AUC relative to blind mode, one point per dataset and metric, ordered by increasing prior information in the prompt (blind, alias, default). Filled points mark fitness; open points mark recall@2%. Black line marks the mean per model. (b) Normalized fitness AUC per dataset, model, and capability (agent, reasoning, no reasoning), in each prompt mode. 1 (yellow) ties the best statistical model, 0 ties random, black cells underperform random. (c) Normalized AUC pooled over datasets and models, fitness left and recall right. No configuration reaches the best statistical model in expectation; reasoning and agents perform within each other’s confidence intervals. Error bars are 95% intervals from a nested bootstrap (datasets, then models, the seeds). The 7 datasets set the effective sample size

Panel of Figure 3 — Figure 3: Belief dynamics rule out entrenchment towards priors. (a) Agreement between stated beliefs, read from reasoning traces by an LLM judge, and action beliefs derived from the proposed batches. The pooled Pearson $\rho=0.759$ (CI ${}_{95\%}[+0.754,+0.764]$ , 673,702 beliefs from 13,484 reasoning traces over 1,350 campaigns) holds across every model, prompt mode, and dataset. (b) Excess movement $Z_{s}$ and Martingale score $M_{s}$ by BO step, pooled by prompt mode, computed from the stated beliefs. A rational Bayesian agent would sit at 0 for both martingale score and excess movement. Every mode is positive in $Z_{s}$ and negative in $M_{s}$ at every step: the models over-react to incoming data rather than entrenching around their prior. Shading marks 95% confidence intervals. (c) Per-dataset belief movement $\Delta m$ , at the first step (filled) compared with the mean of steps 2–9 (open). Beliefs swing roughly $3\times$ more on the first update as the model moves away from a purely prior-driven belief, then settle into more calibrated updates thereafter. Error bars are 95% confidence intervals across seeds.

Panel of Figure 4 — Figure 4: LLMs act as local, uncertainty-avoiding optimizers, yet intend to explore. (a) Every selection decomposed into a value feature $V$ and an uncertainty feature $U$ through a shared GP surrogate. High $V$ is value-seeking; high $U$ is uncertainty-seeking. The six statistical policies span the uncertainty axis as expected, placing greedy and directed evolution (DE) at the bottom and UCB and EI at the top. Reasoning LLMs sit below every predictive policy tested; non-reasoning models sit at or below DE. The agent recovers roughly half the distance in the $U$ -axis to the uncertainty-based statistical policies, and the agent with the history removed reaches the $U$ -range of a Thompson sampler (Section 2.5 ). (b) Mean Hamming distance from each proposal to the candidates already in history, per campaign step. Higher is more novel. The reasoning LLM stays closer to its own history than any policy except DE, and allowing reasoning enforces the bias against exploration rather than relieving it. (c) Cycle-corrected correlation between the exploration intent extracted from each reasoning trace and three realized exploration proxies (novelty, coverage gain, rarity), for reasoning LLMs, agents, and agents with the history removed. Each point is one campaign step. 13,550 traces, first step of each campaign omitted. For the reasoning LLM, the correlation is indistinguishable from zero on novelty and rarity; giving it a tool that provides an uncertainty estimate makes all three positive. The intent to explore is honest and enacted. The failure is an intention-ability gap, not an intention-action gap. Coloured lines are binned means with 95% intervals. Grey reference levels mark UCB, greedy, and DE.

Study	Dataset	$\|\mathcal{X}\|$	$N_{dim}$	Batch	Budget	Cycles	Surrogate	New Acquisitions
ALDE	TrpB	159,129	4	10	100	10	ALDE DNN	EI
ALDE	GB1	149,361	4	10	100	10	ALDE DNN	EI
EDBO	direct_arylation	1,728	5	5	50	10	EDBO GP
EDBO	aryl_amination_2b	792	4	5	50	10	EDBO GP
Tripeptide	tripeptides	8,000	3	5	50	10	EDBO GP
Gryffin	NFA	301	3	3	30	10	EDBO GP

Table 1: Per-benchmark configurations. LLM campaigns use

n_{\mathrm{init}}{=}0

; statistical runs draw the first batch at random. New acquisitions are manually implemented over the surrogate posterior.

Model	Model ID	Provider	Temp.	topP	Thinking	Max tokens out
GPT-5.4	gpt-5.4-2026-03-05 [ 36 ]	OpenAI		1.0	effort=medium	25,000
Sonnet 4.6	claude-sonnet-4-6 [ 3 ]	Anthropic	1.0		thinking budget 25k	32,000
Kimi 2.5	moonshotai/kimi-k2.5 [ 34 ]	OpenRouter	1.0	1.0	thinking budget 25k	25,000
Qwen3.5	qwen/qwen3.5-397b-a17b [ 37 ]	OpenRouter	1.0	1.0	thinking budget 25k	25,000
GLM-4.7	glm-4.7 [ 54 ]	Z.AI	1.0	0.95	thinking	25,000

Table 2: LLM configuration (reasoning-enabled). Temperatures and topP are all provider defaults.

Diagnostic	Estimator	Estimate	95% CI	Prediction
Martingale score
	naive $M$	$-0.0352$	$[-0.0406,,-0.0301]$	$-0.0397$ (Eq. 6 )
	$\tilde{M}$	$+0.0021$	$[-0.0029,,+0.0068]$	$0$
	$\tilde{M}$ , edge-heavy	$+0.0003$	$[-0.0011,,+0.0016]$	$0$
Belief movement
	naive $Z$	$+0.0241$	$[+0.0211,,+0.0271]$	$+0.0250$ ( $2\sigma_{\varepsilon}^{2}$ )

Table 3: Rational-agent (null) validation. CI computed from bootstrap across

N=300

artificial campaigns.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Mode	exploit	explore	avoid	hyps/cycle	$N$
gpt-5-4-2026-03-05	default	[0.45, 0.49]	[0.33, 0.37]	[0.17, 0.20]	[6.7, 7.7]	2,648
	alias	[0.44, 0.47]	[0.35, 0.38]	[0.16, 0.19]	[7.1, 7.8]	2,764
	blind	[0.43, 0.46]	[0.36, 0.40]	[0.16, 0.19]	[6.9, 7.5]	2,510
claude-sonnet-4-6	default	[0.46, 0.49]	[0.35, 0.38]	[0.15, 0.18]	[8.0, 8.8]	3,188
	alias	[0.45, 0.49]	[0.38, 0.41]	[0.12, 0.15]	[7.5, 8.2]	2,979
	blind	[0.43, 0.47]	[0.38, 0.42]	[0.14, 0.17]	[6.3, 7.0]	2,399

Appendix

Table SI-1: Fraction of hypotheses of different tags by model and prompt mode (exploit, explore, avoid). hyps/cycle: mean number of hypotheses per cycle. 95% CI only.

N

marks the total number of hypotheses extracted per model-prompt mode combination. We use only hypotheses from the first 5 cycles to highlight early-stage hypothesis dynamics.

Model	Tag	default	alias	blind	$N$
gpt-5-4-2026-03-05	exploit	[0.81, 0.86]	[0.79, 0.84]	[0.82, 0.88]	3,585
	explore	[0.68, 0.74]	[0.69, 0.75]	[0.68, 0.74]	2,901
	avoid	[0.12, 0.18]	[0.15, 0.21]	[0.15, 0.22]	1,436
claude-sonnet-4-6	exploit	[0.81, 0.86]	[0.86, 0.90]	[0.80, 0.87]	3,989
	explore	[0.75, 0.81]	[0.74, 0.80]	[0.66, 0.73]	3,281
	avoid	[0.12, 0.17]	[0.15, 0.23]	[0.10, 0.19]	1,296

Appendix

Table SI-2: Fraction of values backed by hypotheses which are later selected in the batch. The hypotheses are divided into three tags: exploitative, exploratory, and avoiding. Relative differences between the three tags highlight the model’s adherence to its own claims. 95% CI shown, bootstrapped from all extracted hypotheses (

N

). We use only hypotheses from the first 5 cycles to highlight early-stage hypothesis dynamics.

Judge pair	Agreement	value
gpt-5-mini vs claude-haiku-4-5	Spearman $\rho$	0.886
gpt-5-mini vs claude-haiku-4-5	Pearson $r$	0.941
gpt-5-mini vs qwen3-8b	Spearman $\rho$	0.827
gpt-5-mini vs qwen3-8b	Pearson $r$	0.894

Appendix

Table SI-3: Cross-model judge agreement on the verbalized beliefs for GPT-5.4 on direct arylation across prompt modes,

n=7800

points. Each gpt-5-mini-vs-judge pair is reported in two ways: the ranked agreement (Spearman

\rho

) and the overall agreement (Pearson

r

Agent model	AUROC	Brier	ECE	$\rho(\mathrm{Gini},h_{\mathrm{data}})$	$\rho(b,P_{\mathrm{sel}})$
gpt-5	$[0.586,\,0.607]$	$[0.301,\,0.307]$	$[0.284,\,0.292]$	$[0.005,\,0.222]$	$[0.730,\,0.775]$
sonnet-4-6	$[0.602,\,0.623]$	$[0.302,\,0.308]$	$[0.289,\,0.296]$	$[0.009,\,0.214]$	$[0.694,\,0.744]$
kimi-k2	$[0.611,\,0.626]$	$[0.299,\,0.304]$	$[0.284,\,0.291]$	$[-0.013,\,0.194]$	$[0.587,\,0.641]$
qwen3.5-397b	$[0.609,\,0.625]$	$[0.301,\,0.306]$	$[0.285,\,0.292]$	$[-0.008,\,0.201]$	$[0.625,\,0.676]$
glm-4-7	$[0.616,\,0.631]$	$[0.298,\,0.304]$	$[0.286,\,0.293]$	$[0.013,\,0.219]$	$[0.699,\,0.741]$

Appendix

Table SI-4: Belief quality (AUROC/Brier/ECE vs. oracle) and Spearman correlations, per judged agent model. Bootstrap 95% CI shown (N=10,000). AUROC significant if CI excludes

0.5

, correlations if CI excludes

0

Figure SI-1: Martingale score (lower) and belief movement (upper) graph from the main paper, pooled across prompt modes (stated belief). Here, we have also derived the same metrics using beliefs estimated directly from actions (action beliefs). Action beliefs and stated beliefs yield the same measured overreaction.

Judge pair	Spearman $\rho$	Pearson $r$
gpt-5-mini vs claude-haiku-4-5	0.83	0.84
gpt-5-mini vs qwen3-8b	0.71	0.71
claude-haiku-4-5 vs qwen3-8b	0.75	0.72

Appendix

Table SI-5: Cross-judge agreement on explore_fraction , GPT-5.4 on direct arylation dataset, pooled over prompt modes,

N=447

reasoning traces.

Target	Spearman $\rho$ [95% CI]
$n_{\text{explore}}$	$+0.24$ $[+0.14,+0.33]$
$n_{\text{exploit}}$	$-0.19$ $[-0.27,-0.10]$
$n_{\text{total}}$	$-0.04$ $[-0.13,+0.05]$

Appendix

Table SI-6: Exploration judge (gpt-5-mini) against independently extracted hypotheses, which tags each stated hypothesis as explore / exploit / avoid .

N=448

reasoning traces. Greater intent to explore should correlate with the number of exploratory hypotheses, but not exploitative ones.

Panel of Figure 15 — Figure SI-2: Per-dataset normalized AUC across the five models, three prompt modes, Moreover, reasoning ablation (max fitness and recall merged per panel). Performance is normalized per dataset against the best statistical model ( $y=1$ ) and random selection ( $y=0$ ). Error bars are standard deviation, dominated by between-model variance.

Panel of Figure 16 — Figure SI-3: Per-model normalized AUC across 7 datasets. No single model dominates: each is strong on a different subset of datasets and modes.

Figure SI-4: Heatmap with full breakdown of normalized AUC by best fitness for each dataset-model-prompt mode triplet. Yellow means better than the best statistical model; black means worse than random. The table to the right gives statistical models’ relative performance as a baseline. Colors do not mark statistical significance. No model performs significantly better or worse than others. Note: The best statistical model is defined by a combination of fitness and recall performance, whereby non-"best" statistical models can be assigned normalized fitness > 1 when performance is close to a tie on one metric.

Figure SI-5: Heatmap with full breakdown of normalized AUC by recall (number of found high-value candidates) for each dataset-model-prompt mode triplet. Yellow means better than the best statistical model; black means worse than random. The table to the right gives statistical models’ relative performance as a baseline. Colors do not mark statistical significance. No one model is performing significantly better or worse than others. Best statistical model is defined by a combination of fitness and recall performance whereby non-"best" statistical models can be assigned normalized fitness > 1 when performance is close to a tie on one metric.

Model	$V$	$U$	Novelty	Rarity	Coverage
gpt-5.4-2026-03-05	[+2.18, +2.47]	[-3.39, -2.95]	[-0.320, -0.293]	[-0.272, -0.251]	[-0.111, -0.096]
claude-sonnet-4-6	[+2.34, +2.63]	[-3.36, -2.86]	[-0.314, -0.289]	[-0.266, -0.249]	[-0.087, -0.073]
moonshotai/kimi-k2.5	[+2.00, +2.24]	[-2.88, -2.48]	[-0.291, -0.266]	[-0.240, -0.222]	[-0.064, -0.054]
qwen3.5-397b-a17b	[+1.92, +2.15]	[-3.14, -2.69]	[-0.288, -0.263]	[-0.243, -0.225]	[-0.072, -0.061]
glm-4.7	[+1.89, +2.12]	[-2.97, -2.60]	[-0.284, -0.261]	[-0.238, -0.221]	[-0.070, -0.060]
Greedy-GP	[+1.61, +2.44]	[-2.80, -1.93]	[-0.262, -0.242]	[-0.201, -0.189]	[-0.068, -0.063]

Appendix

Table SI-7: Per-model behavioral statistics, pooled across 7 datasets, 3 prompt modes, and all seeds (reasoning). Exploration metrics are measured as excess over uniform sampling. Higher

U

, Novelty, Rarity, or Coverage are more exploratory. Brackets are 95% CIs from a 10,000-resample campaign cluster bootstrap. 14,175 batches over 1,575 campaigns. Greedy-GP and UCB-GP are statistical acquisition functions run on the same datasets for reference. Exploration columns use 9,450 batches over 1,050 campaigns.

V

U

here subsample 10 seeds/dataset.

Panel of Figure 20 — Figure SI-6: Summary statistics on the performance of GPT-5.4, Kimi-2.5, Qwen-3.5, and GLM on the direct arylation task. All models but GLM show some negative correlation between the default prompt mode and performance; the variance is large among models. Error bars are standard deviation. $N=15$ seeds.

Panel of Figure 21 — Figure SI-7: Summary statistics on the performance of GPT-5.4, Kimi-2.5, Qwen-3.5, and GLM on the TrpB task. GPT and Qwen show some negative correlation between default prompt mode and performance. Error bars are the standard error. $N=15$ seeds.

Panel of Figure 22 — Figure SI-8: Summary statistics on the performance of GPT-5.4, Kimi-2.5, Qwen-3.5, and GLM on the OER task.

Panel of Figure 23 — Figure SI-9: Convergence of LLMs and statistical models on the EDBO direct arylation dataset (best-yet fitness). LLM panels use 15 seeds; statistical models use 75 seeds for tighter estimates, since these normalize the AUC across datasets and their variance propagates to all downstream metrics.

Mode	Configuration	$V$	$U\uparrow$	norm. AUC	DE-frac. $\downarrow$	Novelty ${}^{\dagger}\uparrow$	Rarity ${}^{\dagger}\uparrow$	Cov. gain ${}^{\dagger}\uparrow$
Default	Standard agent	$[+2.29,+2.92]$	$[-2.48,-1.86]$	$[0.66,0.86]$	$[0.31,0.54]$	$[-0.36,-0.23]$	$[-0.29,-0.20]$	$[-0.11,-0.06]$
	No-history, no filter	$\mathbf{[-0.59,-0.15]}$	$\mathbf{[+0.38,+1.16]}$	$[-0.06,+0.02]$	$\mathbf{[-0.16,-0.00]}$	$\mathbf{[+0.04,+0.06]}$	$\mathbf{[+0.04,+0.07]}$	$\mathbf{[+0.01,+0.03]}$
	history, no filter	$[-0.42,+0.02]$	$[-0.11,+0.19]$	$[-0.04,+0.02]$	$[-0.01,+0.05]$	$[-0.00,+0.01]$	$\mathbf{[+0.00,+0.02]}$	$[-0.00,+0.01]$
	No-history, full filter	$\mathbf{[-0.76,-0.36]}$	$\mathbf{[+0.19,+0.79]}$	$\mathbf{[-0.09,-0.00]}$	$\mathbf{[-0.18,-0.03]}$	$\mathbf{[+0.03,+0.06]}$	$\mathbf{[+0.04,+0.06]}$	$\mathbf{[+0.00,+0.03]}$
Blind	Standard agent	$[+2.23,+2.98]$	$[-2.09,-1.59]$	$[0.60,0.82]$	$[0.32,0.58]$	$[-0.32,-0.20]$	$[-0.24,-0.16]$	$[-0.07,-0.03]$
	No-history, no filter	$\mathbf{[-0.86,-0.31]}$	$\mathbf{[+0.48,+1.40]}$	$[-0.02,+0.05]$	$\mathbf{[-0.21,-0.03]}$	$\mathbf{[+0.03,+0.06]}$	$\mathbf{[+0.03,+0.06]}$	$\mathbf{[+0.00,+0.02]}$

Appendix

Table SI-8: history

\times

filter

2\times 2

, reported in

(V,U)

coordinates, normalized AUC, DE-fraction, and the exploration triple (novelty, rarity, coverage gain). Standard agent: 95% bootstrap CI of the respective metric. Other configurations:

\Delta

vs. Standard (cell

-

Standard), bootstrap difference on joint dataset and model. Exploration metrics † are excess over the pooled null (

0

= chance). 10,000 resamples; point estimates omitted. Pooled over 7 datasets, 2 models (Sonnet-4.6 and GPT-5.4), and 10 seeds. Bold

\Delta

CI excludes 0. Arrows mark increased exploration .

Figure SI-10: Correlation between relative improvement of including the history in the agent prompt and the relative performance of a local optimizer (DE) over an uncertainty-driven optimizer (UCB). Measured over two metrics: normalized AUC of fitness and recall.

Lengthscale $\ell$
GP hyperparameters	$\rho(U)$	$\rho(V)$
Reference ( $\ell=2$ , noise $=0.1$ , $c=1$ )	$1.00$	$1.00$
$\ell=0.5$	$0.96$ $[0.64,1.00]$	$0.79$ $[0.61,0.96]$
$\ell=1$	$1.00$ $[0.89,1.00]$	$0.86$ $[0.61,0.96]$
$\ell=4$	$1.00$ $[0.89,1.00]$	$1.00$ $[0.86,1.00]$
$\ell=8$	$1.00$ $[0.89,1.00]$	$1.00$ $[0.86,1.00]$

Appendix

Table SI-9:

(V,U)

decomposition sensitivity analysis to the hyperparameters of the GP surrogate. Spearman-

\rho

of the policy ordering on each channel against the fixed GP (

\ell=2

, noise

=0.1

c=1

; RBF), with a paired-bootstrap 95% CI (2,000 resamples)

Mode	Prompt	$V$	$U$	norm. AUC	DE-frac.	Novelty †	Rarity †	Cov. gain †
Default	Baseline	$[+1.94,+2.14]$	$[-3.18,-2.74]$	$[0.66,0.82]$	$[0.43,0.58]$	$[-0.34,-0.26]$	$[-0.28,-0.23]$	$[-0.11,-0.07]$
Default	Nudged	$[-0.06,+0.21]$	$[-0.05,+0.31]$	$[-0.01,+0.05]$	$[-0.02,+0.02]$	$[-0.00,+0.01]$	$[-0.00,+0.01]$	$\mathbf{[+0.00,+0.01]}$
Blind	Baseline	$[+2.11,+2.41]$	$[-2.94,-2.60]$	$[0.63,0.77]$	$[0.44,0.60]$	$[-0.32,-0.24]$	$[-0.26,-0.21]$	$[-0.08,-0.04]$
Blind	Nudged	$[-0.11,+0.26]$	$\mathbf{[+0.04,+0.44]}$	$[-0.01,+0.04]$	$[-0.01,+0.04]$	$[-0.00,+0.01]$	$[-0.00,+0.01]$	$[-0.00,+0.01]$

Appendix

Table SI-10: Capability-nudge ablation (all five reasoning LLMs), on the same battery as Table SI-8 :

(V,U)

coordinates, normalized AUC, DE-fraction, and the exploration triple (novelty, rarity, coverage gain). Nudged:

\Delta

vs. Baseline (nudged

-

baseline), bootstrap difference on shared dataset and model. Exploration metrics † are excess over the pool null (

<0

implies less exploration than random). All CIs are based on 10,000 resamples across 7 datasets, 5 models, and 10 seeds each. Bold

\Delta

CI separated from 0.

Large Language Model-Guided Scientific Discovery

Large language model-guided scientific discovery uses AI to propose, test, and refine hypotheses, models, experiments, and designs.

See all 272 papers

From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?
Oct 21, 2026Linus Filbry, Luckeciano C. Melo, Yarin GalBelief Updating in LLMs Bayesian Inference
OATML, University of Oxford
Bayesian Model Discovery with Language Models: Closing the Loop between Hypotheses and Experiments
Oct 19, 2026JooKyung Yuh, Byungmin Park, Eunseo Choi +6Scientific Discovery Large Language Model-Guided Scientific Discovery
Korea University · KAIST · KETI
Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search
Aug 16, 2026Zhongwei Yu, Yan Song, Xue Yan +9Bayesian Optimization Large Language Model-Guided Optimization
The Hong Kong University of Science and Technology (GZ) · University College London · AI Lab, The Yangtze River Delta +3

Oct 21, 2026J/K move · Enter open · S save

From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?

Linus Filbry, Luckeciano C. Melo, Yarin Gal

OATML, University of Oxford

Belief Updating in LLMs Bayesian Inference Language Model Reasoning Evaluation Scientific Reasoning in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly applied across the scientific discovery pipeline, from synthesizing literature to generating hypotheses to informing experimental design. A unifying principle behind these tasks is the need to continuously update beliefs as evidence accumulates, for which Bayesian inference provides a natural framework. However, existing work assumes that LLMs cannot carry out principled Bayesian belief updating in context, instead relying on simplifications and heuristics. To challenge this assumption, we present a principled evaluation of Bayesian reasoning capabilities in LLMs, analyzing both the model's internal consistency as well as its ability to recover an externally determined posterior. Specifically, we build an evaluation framework spanning qualitative epistemological examples to probe probabilistic coherence, quantitative benchmarks of Bayesian inference capabilities, and analyses of the impact of non-Bayesian updates on downstream task performance. Across different scenarios, we find that LLMs generally violate Bayes' rule when updating their beliefs in context. However, when equipped with modern reasoning abilities and code execution capabilities, their performance drastically improves, enabling even the close approximation of analytically intractable posteriors. Our evaluation shows that current LLMs with access to appropriate tools could already be used to make complex belief updates in scientific discovery, enabling rapid acceleration of the efficiency and scale of AI-powered scientific discovery.

Abstract

Large Language Model-Guided Scientific Discovery

Explore similar work

From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?

Bayesian Model Discovery with Language Models: Closing the Loop between Hypotheses and Experiments

Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search