Sequential Experimental Design

Latest papers

13 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Explore or Confirm? LLM Agents Misallocate Measurements Under Noise
Oct 18, 2026Abdul Karim, Matt KeonAI Agents for Scientific Discovery Exploration-Exploitation Tradeoff
Xstendyl
Active Real-World Factor-Based Evaluation for Generalist Robot Policies
Jul 16, 2026Andrew Liao, Hanchen Cui, Karthik Desingh +1Robotic Policy Evaluation Sequential Experimental Design
University of Minnesota Twin Cities
Adaptive Beam Selection for Efficient Scanning Probe Tomography
Jun 19, 2026San Dinh, Zichao Wendy Di, Matt MenickellyImage Reconstruction Sequential Experimental Design
Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, IL, 60439, USA
CARE: Context-Aware Ranking Evolution with Executable Scoring Programs for Budgeted Reaction Optimization
Jun 12, 2026Guanyu Liu, Weiyi Kong, Chao Tang +5Large Language Model-Guided Optimization Program Synthesis
University of Macau · University of Toronto · UCLA +3
AI-Guided Design and Optimization of Graphite-Based Anodes via Iterative Experimental Feedback
May 29, 2026Qian Du, Mark M. Sullivan, James E. Saal +1Surrogate-Assisted Optimization Materials Science
hte GmbH, Kurpfalzring 104, 69123 Heidelberg, Germany · Citrine Informatics, 2629 Broadway St., Redwood City, CA 94063, USA
Constrained Bayesian Experimental Design via Online Planning
May 26, 2026Yujia Guo, Daolang Huang, Xinyu Zhang +3Bayesian Experimental Design Sequential Experimental Design
ELLIS Institute Finland · Department of Computer Science, Aalto University, Finland · Department of Computer Science, University of Manchester, UK
Robust Sequential Experimental Design for A/B Testing
May 13, 2026Qianglin Wen, Xiangkun Wu, Chengchun Shi +4Causal Effect Estimation Model Misspecification
Yunnan Key Laboratory of Statistical Modeling and Data Analysis, Yunnan University, Kunming, China · School of Mathematical Sciences, Zhejiang University, Hangzhou, China · Department of Statistics, London School of Economics and Political Science, London, United Kingdom +3
Sequential Design of Genetic Circuits Under Uncertainty With Reinforcement Learning
May 7, 2026Michal Kobiela, Diego A. Oyarzún, Michael U. GutmannReinforcement Learning Sequential Experimental Design
School of Informatics, The University of Edinburgh, UK · School of Informatics, School of Biological Sciences, The University of Edinburgh, UK · School of Informatics, The University of Edinburgh,, UK
Optimizing Social Utility in Sequential Experiments
May 7, 2026Ander Artola Velasco, Stratis Tsirtsis, Manuel Gomez-RodriguezMechanism Design Sequential Decision Making
Max Planck Institute for Software Systems, Kaiserslautern, Germany · Hasso Plattner Institute, Potsdam, Germany
Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection
Apr 24, 2026Sijie Li, Shanda Li, Haowei Lin +3Scaling Laws Bayesian Experimental Design
Peking University · Carnegie Mellon University · Datadog
ALMAB-DC: Active Learning, Multi-Armed Bandits, and Distributed Computing for Sequential Experimental Design and Black-Box Optimization
Mar 22, 2026Foo Hui-Mean, Yuan-chin I ChangSequential Experimental Design Bayesian Optimization
Institute of Statistical Science, Academia Sinica, Taipei, Taiwan
Experimental Design for Missing Physics
Mar 21, 2026Arno Strouwen, Sebastián Micluţa-CâmpeanuSequential Experimental Design
JuliaHub, MA 02111-1929 USA · Biosystems Department, KULeuven, Leuven, Belgium · Faculty of Physics, University of Bucharest, Bucharest, Romania
Sequential Bayesian Evaluation of Large Language Model Behavior
Nov 4, 2025Saatvik Kher, Shang Wu, Rachel Longjohn +2LLM Evaluation Language Model Generation Evaluation
Department of Computer Science, University of California, Irvine · Department of Statistics, University of California, Irvine

Oct 18, 2026J/K move · Enter open · S save

Explore or Confirm? LLM Agents Misallocate Measurements Under Noise

Abdul Karim, Matt Keon

Xstendyl

AI Agents for Scientific Discovery Exploration-Exploitation Tradeoff Autonomous Experimentation LLM Agent Evaluation Sequential Experimental Design AI Agent Benchmarks

Open paper AskPDF arXiv

As model reasoning gets cheaper, the physical experiment becomes the slow and costly step in discovery. Each experiment returns a noisy reading: a high value can mean a good candidate or a lucky draw. With a fixed budget, an AI agent must keep choosing between exploring a new candidate and confirming a promising one by re-testing it. We measure that choice in Wellocate, a simulated assay campaign with known true values. It is built on a generated landscape and on two real protein datasets, GB1 and PhoQ, scrambled so no model can answer from memory. We test five language models as minimal agents, with no tools and no memory beyond the history in each prompt. They misallocate in both directions. Three over-confirm, spending most of the budget re-testing. On the generated landscape one under-confirms, trusting single readings, and there every model ends with a worse final pick than a basic scripted strategy. Each failure has its own cause. The over-confirmers run short of new candidates worth trying: a list of untried candidates cuts their re-testing and, on real protein data, makes two of them pick much better. A running summary of their readings or a description of the landscape changes it by at most five percentage points. For the under-confirmer, re-testing its top candidates in a final round, replayed on its own campaigns, recovers only about a quarter of what it loses by picking the wrong candidate. This suggests confirmation has to run through the whole campaign. We release Wellocate as a benchmark and reinforcement-learning environment with frozen splits and scripted baselines. With it, anyone can measure how an agent splits its budget before trusting it with real experiments, and train agents to split it better. Code, data and the environment are available at https://github.com/Xstendyl/wellocate.