Competing with AI Scientists: Agent-Driven Approach to Astrophysics Research

Thomas Borrett; Licong Xu; Andy Nilipour; Boris Bolliet; Sebastien Pierre; Erwan Allys; Celia Lecat; Biwei Dai

cs.AIMar 18, 2026

Competing with AI Scientists: Agent-Driven Approach to Astrophysics Research

Authors: Thomas Borrett, Licong Xu, Andy Nilipour, Boris Bolliet, Sebastien Pierre, Erwan Allys, Celia Lecat, Biwei Dai,

Organizations: Cavendish Astrophysics, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Kavli Institute for Cosmology, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France ·

Abstract

We present an agent-driven approach to the construction of parameter inference pipelines for scientific data analysis. Our method leverages a multi-agent system, Cmbagent (the analysis system of the AI scientist Denario), in which specialized agents collaborate to generate research ideas, write and execute code, evaluate results, and iteratively refine the overall pipeline. As a case study, we apply this approach to the FAIR Universe Weak Lensing Uncertainty Challenge, a competition under time constraints focused on robust cosmological parameter inference with realistic observational uncertainties. While the fully autonomous exploration initially did not reach expert-level performance, the integration of human intervention enabled our agent-driven workflow to achieve a first-place result in the challenge. This demonstrates that semi-autonomous agentic systems can compete with, and in some cases surpass, expert solutions. We describe our workflow in detail, including both the autonomous and semi-autonomous exploration by Cmbagent. Our final inference pipeline utilizes parameter-efficient convolutional neural networks, likelihood calibration over a known parameter grid, and multiple regularization techniques. Our results suggest that agent-driven research workflows can provide a scalable framework to rapidly explore and construct pipelines for inference problems.

Figures & tables

Panel of Figure 1 — Figure 1: Inference pipeline for weak lensing. The objective is to develop a mapping, $f$ , that transforms an input weak lensing map, $x$ , into predicted cosmological parameters, $\hat{\theta}$ , and associated uncertainties, $\hat{\sigma}$ . This process yields a joint posterior distribution (right) for the cosmological parameters $\Omega_{m}$ , the matter density, and $S_{8}$ , the clustering amplitude. The input map shown is a square 176 pixel-wide cutout from the full map, which has pixel dimension $1424\times 176$ .

Panel of Figure 2 — Figure 2: The autonomous research loop we adopt for this task. We craft an initial prompt with example code, context, and scores of various approaches. Cmbagent in Planning & Control mode then attempts a solution according to a plan, with analysis of errors and shortcomings. The output report is injected into the initial prompt, and the loop is restarted for a second attempt.

Panel of Figure 3 — Figure 3: Diagram of our inference pipeline. CNN ensemble predictions (with $D_{4}$ test-time augmentation) are treated as summary statistics, $\hat{\theta}$ ; an empirical Gaussian likelihood is calibrated on validation predictions at each cosmology point $\theta_{g}$ and is used to estimate the weights to produce point estimates and uncertainties.

Score	MSE	Coverage
11.70	0.10	0.70

Table 1: Performance of the inference pipeline on an unseen test set. The MSE is the normalized error, and the coverage is the fraction of parameters within

\pm 1\sigma

Category	First surfaced in autonomous search	First surfaced after human redirection
CNN backbone	ResNet (18/34/50)	BaseCNN, Inception, InceptionSE
Inference method	MCMC, direct moments prediction	Grid likelihood over known cosmology points
Augmentation	Flips, rotations	Full $D_{4}$ symmetries with test-time averaging

Table 2: Exploration of the inference design space under autonomous and human-redirected agentic search. Entries indicate when a method family first appeared in the search trajectory, rather than a controlled attribution of individual contributions. Some explored methods do not appear in the final pipeline.

Agents/LLMs	Model	Inference	Augmentation	Score
Cmbagent - P&C	ResNet-18/34/50	MCMC	Flips/rotations	$10.50$
+ Cmbagent - One-Shot	BaseCNN	MCMC	No	11.02
+ GPT-5	BaseCNN	Grid	No	11.42
+ Gemini-2.5-Pro	Inception + InceptionSE	Grid	Flips/rotations/transpose	11.72

Table 3: Summary of model–generation pathways used during the architecture search, showing the models proposed under different Cmbagent /LLM configurations and their corresponding test-set performance. The “+” prefix indicates the incremental integration of the specified LLM into the search pipeline. In the inference method column, “MCMC” refers to sampling the posterior using a Markov Chain Monte Carlo algorithm. “Grid” correspond to our final inference pipeline, described in Section 3.5 . BaseCNN denotes a light-weight CNN. The score presented here is on the validation dataset.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Inception		InceptionSE
Operation	Output size	Operation	Output size
Conv. ( $7\times 7$ , $s=2$ ) MaxPool ( $2\times 2$ )	$32\times\tfrac{H}{4}\times\tfrac{W}{4}$	Conv. ( $7\times 7$ , $s=2$ ) MaxPool ( $2\times 2$ )	$48\times\tfrac{H}{4}\times\tfrac{W}{4}$
Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) MaxPool ( $2\times 2$ )	$64\times\tfrac{H}{8}\times\tfrac{W}{8}$	Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) SE : $r=8$ MaxPool ( $2\times 2$ )	$96\times\tfrac{H}{8}\times\tfrac{W}{8}$
Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) MaxPool ( $2\times 2$ )	$128\times\tfrac{H}{16}\times\tfrac{W}{16}$	Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) SE : $r=8$ MaxPool ( $2\times 2$ )	$192\times\tfrac{H}{16}\times\tfrac{W}{16}$
Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) MaxPool ( $2\times 2$ )	$256\times\tfrac{H}{16}\times\tfrac{W}{16}$	Inception : Conv. ( $1\times 1$ ) Conv. ( $3\times 3$ ) Conv. ( $5\times 5$ ) MaxPool ( $3\times 3$ ) + Conv. ( $1\times 1$ ) SE : $r=8$ MaxPool ( $2\times 2$ )	$256\times\tfrac{H}{16}\times\tfrac{W}{16}$
AdaptiveAvgPool ( $1\times 1$ )	$\mathbb{R}^{256}$	AdaptiveAvgPool ( $1\times 1$ )	$\mathbb{R}^{256}$

Appendix

Table 4: Layer-by-layer summary of the two backbone architectures. The total number of trainable parameters is 422,754 for the Inception model and 722,610 for the Inception–SE model.

Panel of Figure 4 — Figure 4: Left: The cosmologies seen during training, validation, and testing, and the unseen cosmologies only seen during testing. Right: The mean score of our inference pipeline for each unseen cosmology with the associated standard error on the mean. The poor performance at one unseen cosmology is driven by the scarcity of the training data in that region.

Scientific Workflow Automation

Scientific workflow automation uses software, AI agents, and laboratory systems to plan, execute, validate, and reproduce research tasks.

See all 161 papers

Beyond AI as Assistants: Toward Autonomous Discovery in Cosmology
May 14, 2026Licong Xu, Thomas BorrettMulti-Agent LLM Systems AI Agents for Scientific Discovery
Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Kavli Institute for Cosmology, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Cavendish Astrophysics, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK
Plausible but Wrong: A case study on Agentic Failures in Astrophysical Workflows
Apr 28, 2026Shivam Rawat, Lucie FlekAgent Failure Analysis AI for Science
Bonn-Aachen International Center for Information Technology, University of Bonn, Germany; Lamarr Institute for Machine Learning and Artificial Intelligence, Germany.
First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope
May 27, 2026Gianluca IngugliaAI Agents for Scientific Discovery AI Agent Evaluation
Marietta Blau Institute for Particle Physics, Austrian Academy of Sciences, 1010 Vienna, Austria

May 14, 2026J/K move · Enter open · S save

Beyond AI as Assistants: Toward Autonomous Discovery in Cosmology

Licong Xu, Thomas Borrett

Institute of Astronomy, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Kavli Institute for Cosmology, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK · Cavendish Astrophysics, University of Cambridge, Madingley Road, Cambridge CB3 0HA, UK

Multi-Agent LLM Systems AI Agents for Scientific Discovery Autonomous Scientific Discovery

Open paper AskPDF arXiv

Recent advances in artificial intelligence (AI) agents are pushing AI beyond tools toward autonomous scientific discovery. We discuss two complementary agentic systems for cosmology: CMBEvolve, which targets tasks with explicit quantitative objectives through LLM-guided code evolution and tree search, and CosmoEvolve, which targets open-ended scientific workflows through a virtual multi-agent research laboratory. As preliminary demonstrations, we apply CMBEvolve to out-of-distribution detection in weak-lensing maps, where it iteratively improves the benchmark score through code evolution, and CosmoEvolve to autonomous ACT DR6 data analysis, where it identifies non-trivial pair- and scale-dependent behaviour and produces analysis-grade diagnostics. These examples show how cosmology can provide both controlled benchmark tasks and realistic open-ended research problems for the development of AI scientist systems.

Abstract

Scientific Workflow Automation

Explore similar work

Beyond AI as Assistants: Toward Autonomous Discovery in Cosmology

Plausible but Wrong: A case study on Agentic Failures in Astrophysical Workflows

First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope