AI Agents Can Already Autonomously Perform Experimental High Energy Physics

Eric A. Moreno; Samuel Bright-Thonney; Andrzej Novak; Dolores Garcia; Philip Harris

hep-exMar 20, 2026

AI Agents Can Already Autonomously Perform Experimental High Energy Physics

Authors: Eric A. Moreno, Samuel Bright-Thonney, Andrzej Novak, Dolores Garcia, Philip Harris

Organizations: Department of Physics, Massachusetts Institute of Technology · NSF AI Institute for Artificial Intelligence and Fundamental Interactions · CERN

Abstract

Large language model-based AI agents are now able to autonomously execute substantial portions of a high energy physics (HEP) analysis pipeline with minimal expert-curated input. Given access to a HEP dataset, an execution framework, and a corpus of prior experimental literature, we find that Claude Code succeeds in automating all stages of a typical analysis: event selection, background estimation, uncertainty quantification, statistical inference, and paper drafting. We argue that the experimental HEP community is underestimating the current capabilities of these systems, and that most proposed agentic workflows are too narrowly scoped or scaffolded to specific analysis structures. We present a proof-of-concept framework, Just Furnish Context (JFC), that integrates autonomous analysis agents with literature-based knowledge retrieval and multi-agent review, and show that this is sufficient to plan, execute, and document a credible high energy physics analysis. We demonstrate this by conducting analyses on open data from ALEPH, DELPHI, and CMS to perform electroweak, QCD, and Higgs boson measurements. We present two of those results in a condensed short paper form -- a CMS Run1 Open Data $H\to τ^+τ^-$ to demonstrate performance on a well-established result, and the first Lund plane measurement on LEP data -- a genuinely novel result and, to our knowledge, the first produced autonomously by an AI agent. Rather than replacing physicists, these tools promise to offload the repetitive technical burden of analysis code development, freeing researchers to focus on physics insight, truly novel method development, and rigorous validation. Given these developments, we advocate for new strategies for how the community trains students, organizes analysis efforts, and allocates human expertise.

Figures & tables

Figure 1: Diagram of how an AI-agent workflow can mirror the typical high-energy physics analysis workflow. On the left, we show the conventional analysis pipeline, which usually starts with legacy code that is then modified to perform the analysis. Analyses normally involve 3 or more levels of review, starting with feedback from other postdocs, students, and faculty collaborating on the analysis (office feedback). The next tier sits within an analysis subgroup (colloquially referred to as a level 3 group). The second tier of review involves a pre-approval phase, in which physics group conveners review an analysis, followed by a formal collaboration review leading to a result and a submission for publication. On the right side, an equivalent interactive workflow can be entirely handled by AI agents, from the conception of an idea through a result that would then undergo a similar collaboration review, followed by publication.

Figure 2: The JFC framework. A high-level physics objective is passed to an autonomous analysis agent, which plans and executes the full pipeline while querying a literature retrieval system (SciTreeRAG) for domain knowledge. The resulting analysis undergoes multi-agent review; if any reviewer flags an issue the agent revises and resubmits until all reviewers approve. Eventually the final document (a rich analysis note 4 4 4 An analysis note is the standard HEP analysis record used for internal review akin to a thesis draft. ) is passed to human physicists for evaluation.

Task	Required tool	Explicitly forbidden
ROOT file I/O	uproot [ 39 ] ( $\geq$ 5.0)	PyROOT, ROOT C++ macros
Array operations	awkward-array [ 38 ] , numpy [ 22 ]	pandas (for event data)
Histogramming	hist [ 44 ] , boost-histogram [ 43 ]	ROOT TH1 , numpy.histogram
Plotting	matplotlib [ 26 ] + mplhep [ 33 ]	ROOT TCanvas , plotly
Statistical modeling	pyhf [ 23 ] (binned), zfit [ 16 ] (unbinned)	RooFit, RooStats
4-vectors	vector [ 14 ] , particle [ 41 ]	Manual calculations

Table 1: Mandated software stack. Optional packages include coffea for columnar event processing, xgboost / scikit-learn for MVA, iminuit / cabinetry for fitting, and fastjet for jet clustering, activated as needed per analysis.

Analysis	Initial prompt (abridged)	Data	Repo
$H\to\tau^{+}\tau^{-}$	Measure the Higgs boson signal strength in the $\mu\tau_{h}$ final state at $\sqrt{s}=8$ TeV	CMS	Github
Z lineshape	Measure Z lineshape parameters ( $M_{Z}$ , $\Gamma_{Z}$ , $\sigma^{0}_{\text{had}}$ ) and $N_{\nu}$ from the invisible width	ALEPH	Github
Lund plane	Measure the primary Lund jet plane density in hadronic Z decays	ALEPH	Github
$R_{b}$ , $R_{c}$ , and $A_{\text{FB}}^{b}$	Measure $R_{b}$ , $R_{c}$ , and $A_{\text{FB}}^{b}$ using signed impact parameter b-tagging and jet charge	ALEPH	Github
Energy-energy correlators	Measure the two- and three-point energy-energy correlators (E2C, E3C), their ratio, and the EEC asymmetry (AEEC) in hadronic Z decays	ALEPH	Github
Event shapes + $\alpha_{s}$	Measure six event shape distributions and extract $\alpha_{s}(M_{Z})$ from NNLO+NLL QCD fits	ALEPH	Github

Table 2: Summary of analyses produced with JFC . Each analysis was run autonomously from the initial prompt shown, with no human input prior to the unblinding gate. The per-analysis wall-clock time breakdown is given in Figure 7 . † The DELPHI analyses were earlier runs produced with Claude Opus 4.6; they are listed here for completeness, but their numerical results are not presented in this paper.

Figure 3: Agent-produced scalar measurements versus their published reference values, shown as the deviation in units of the agent’s own total uncertainty,

(\theta-\theta_{\mathrm{ref}})/\sigma_{\mathrm{meas}}

, with

\pm 1\sigma

and

\pm 2\sigma

guides (dashed). For the lineshape and heavy-flavour observables (

M_{Z}

\Gamma_{Z}

\sigma^{0}_{\mathrm{had}}

R_{b}

A_{\mathrm{FB}}^{b}

\sin^{2}\theta_{\mathrm{eff}}

) we compare against the measuring experiment’s own published value rather than the LEP+SLD combination; the derived quantities

N_{\nu}

and

\alpha_{s}(M_{Z})

are compared against the LEP electroweak combination and the PDG world average, respectively. The plotted values, their published references, and the reference sources are listed in Table A.1 ; pure shape/density measurements (Lund plane, EEC) have no scalar reference and are omitted.

Figure 4: Headline CMS

H\to\tau^{+}\tau^{-}

result, reproduced from Appendix A . The

S/(S+B)

-weighted post-fit

D_{NN}

distribution of the CMS

H\to\tau^{+}\tau^{-}

measurement (left), combining the 0-jet, boosted, and VBF categories, with the post-fit signal at

\hat{\mu}=1.20

(red) overlaid on the background stack and the weighted residual in the lower panel. Decomposition of the expected signal-strength uncertainty

\sigma_{\mu}

(right), obtained by sequential Asimov re-fits of the analysis workspace: starting from this analysis (

\sigma_{\mu}=1.20

on the partial

11.5~\mathrm{fb}^{-1}

8 TeV open-data sample), successively bringing the luminosity (full Run-1, 8 TeV

{+}

7 TeV), simulation statistics, and systematic precision to CMS scale lowers the expected uncertainty to

\sigma_{\mu}\approx 0.65

— within a factor

\times 1.6

of the published CMS

\mu\tau_{h}

value of

0.41

, with the residual reflecting analysis method and design rather than inputs; removing systematics entirely leaves a statistical floor of

0.45

Figure 5: Indicative cross-model comparison for the CMS

H\to\tau^{+}\tau^{-}

(

\mu\tau_{h}

) measurement: the observed signal strength

\hat{\mu}

on the

D_{NN}

discriminant from our JFC framework run with three different driving models — Claude Opus 4.8 (used for the analyses throughout this paper), a Codex-based agent (GPT-5.5), and Claude Fable 5 — against the published CMS value (grey band,

1.01\pm 0.41

; dashed line). Best-fit values are shown as observed (the Fable 5 fit prefers a negative

\hat{\mu}

). The three runs share the identical prompt, framework, single

\mu\tau_{h}

channel, and partial 2012 CMS Open Data sample, and differ only in the underlying model. This is a single run per model with no repeat trials — an illustrative hint at model-to-model variation, not a statistically controlled benchmark.

Figure 6: Headline ALEPH Lund jet plane result, reproduced from Appendix A.10 . Unfolded primary Lund jet plane density $\rho(\ln 1/\Delta\theta,\,\ln k_{t})$ at charged-particle level, measured on archived ALEPH LEP1 data — to our knowledge the first measurement of this density in $e^{+}e^{-}$ collisions.

Figure 7: Per-analysis decomposition of the wall-clock times reported in Table 2 , measured from git log of the per-analysis repositories. Dark segments are autonomous agent work before the unblinding gate (real compute); mid segments are the idle gaps (rate-limit waits) within that pre-gate phase; pale segments are the post-gate human-review/iteration time with idle gaps likewise removed, measured up to the point at which paper writing begins (the first commit touching the letter/paper sources): the analysis note is the deliverable, so once the letter is being drafted the analysis is effectively complete and any later edits are errata-grade rather than review. The greyed rows are earlier DELPHI analyses produced with Claude Opus 4.6; they are not among the results presented in this paper and are shown only to illustrate the wall-clock profile of those earlier runs.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	Reference				Agent measurement
Observable	Value	$\sigma_{\mathrm{stat}}$	$\sigma_{\mathrm{syst}}$	$\sigma_{\mathrm{tot}}$	Value	$\sigma_{\mathrm{stat}}$	$\sigma_{\mathrm{syst}}$	$\sigma_{\mathrm{tot}}$	Pull
ALEPH Z lineshape [ 6 , 42 ] [code]
$M_{Z}$ [GeV]	$91.1885$	$0.0024$	$0.0017$	$0.0031$	$91.1638$	$0.003$	$0.040$	$0.0402$	$-0.6$
$\Gamma_{Z}$ [GeV]	$2.4951$	$0.0038$	$0.0016$	$0.0043$	$2.4508$	$0.008$	$0.009$	$0.0124$	$\mathbf{-3.4}$
$\sigma^{0}_{\mathrm{had}}$ [nb]	$41.559$	—	—	$0.058$	$42.47$	$0.065$	$0.68$	$0.69$	$+1.3$
$N_{\nu}$	$2.9840$	—	—	$0.0082$	$2.864$	—	—	$0.171$	$-0.7$

Appendix

Table A.1: Numerical values plotted in Fig. 3 : agent-produced scalar measurements alongside the published reference value used for each comparison, grouped by analysis. Both the Reference and Agent blocks split

\sigma_{\mathrm{stat}}

and

\sigma_{\mathrm{syst}}

where the source separates the two;

\sigma_{\mathrm{tot}}

is the corresponding quadrature sum or the single total when no split was provided. Em-dashes mark missing splits. The compatibility pull is

(\theta_{\mathrm{agent}}-\theta_{\mathrm{ref}})\big/\sqrt{\sigma_{\mathrm{agent,\,tot}}^{2}+\sigma_{\mathrm{ref,\,tot}}^{2}}

, treating reference and agent uncertainties as uncorrelated. Bold pulls flag

|\mathrm{pull}|\geq 2

. Each analysis group links to its public code repository ( [code] ); the CMS measurement is additionally detailed in Appendix A . The numerical content is read directly from the same measurements.yaml and references.yaml that feed Fig. 3 .

Panel of Figure 11 — Figure A.1: Representative leading-order diagrams for the two Higgs-boson production modes scaled by the signal strength in this analysis: gluon–gluon fusion (left), $gg\to H$ , proceeding through a heavy-quark loop here drawn as an effective $ggH$ vertex, and vector-boson fusion (right), $qq\to qqH$ , through $t$ -channel $W$ or $Z$ exchange with two forward-tagging quark jets. In both cases the Higgs boson decays to a tau pair, $H\to\tau^{+}\tau^{-}$ , with one tau decaying leptonically to a muon, $\tau\to\mu\,\nu\bar{\nu}$ , and the other hadronically, $\tau\to\tau_{h}\,\nu$ , defining the $\mu\tau_{h}$ final state.

Figure A.2: Yields in the three b-tagged

t\bar{t}

control regions (0-jet, boosted, VBF) entering the simultaneous fit (left): for each region the pre-fit prediction (

k_{t\bar{t}}=1

, grey) and the post-fit prediction (scaled by the fitted

\hat{k}_{t\bar{t}}=0.653\pm 0.078

, brown) are compared with the observed control-region data (points); the pre-fit prediction overshoots the data in all three regions, and the single shared normalization

k_{t\bar{t}}

brings the post-fit prediction into agreement, providing the in-situ

t\bar{t}

constraint that is propagated into the signal regions. Distribution of the visible di-tau mass

m_{\mathrm{vis}}

, the invariant mass of the muon and the visible hadronic-tau decay products, for the inclusive

\mu\tau_{h}

selection (right): the data are compared with the stack of background processes, the

W

+jets and QCD multijet contributions estimated from data; the spectrum is broad and peaks well below the Higgs-boson mass because the neutrinos from both tau decays escape detection, and the absence of a narrow mass peak motivates the use of a multivariate discriminant rather than a simple mass window to separate signal from background.

Figure A.3: Post-fit

D_{NN}

distributions in the three event categories: the data are overlaid on the post-fit background stack with its uncertainty band and the post-fit signal (red) scaled to

\hat{\mu}

, with the data-to-model ratio shown below each panel. 0-jet category (left), the highest-yield and lowest signal-to-background category. Boosted category (middle,

p_{T}^{\tau\tau}>100

GeV), enriched in gluon-fusion events recoiling against initial-state radiation. VBF category (right,

m_{jj}>500

GeV,

|\Delta\eta_{jj}|>3.5

), the highest signal-to-background but lowest-yield category, whose few tens of events are well described by the post-fit model. The post-fit model reproduces the data in all three categories without localized discrepancies, across the three orders of magnitude in event count that separate them.

Figure A.4:

S/(S+B)

-weighted post-fit

D_{NN}

distribution combining the 0-jet, boosted, and VBF categories (left): each (category, bin) is weighted by its post-fit

S/(S+B)

and the weighted yields are accumulated on the common

D_{NN}

axis. The points show the data; the filled histogram is the post-fit per-process background stack (

Z\to\tau^{+}\tau^{-}

Z\to\ell\ell

, rare,

t\bar{t}

W

+jets, QCD), with the hatched band showing its total post-fit uncertainty propagated from the fit covariance; the solid red line is the post-fit signal-plus-background at

\hat{\mu}=1.20

. The lower panel shows the weighted residual (data minus background) as points, with the post-fit signal

S

overlaid as a red step and the post-fit background uncertainty as a hatched band about zero. Observed and expected 95% confidence-level

\mathrm{CL}_{s}

upper limits on the signal strength

\mu

(right): the observed limit is

\mu<3.72

and the expected median

\mu<2.58

, with the green and yellow bands showing the

\pm 1\sigma

and

\pm 2\sigma

ranges of the expectation and the Standard Model value

\mu=1

indicated; the observed limit lies within the

\pm 1\sigma

band of the expectation. The fitted signal is consistent with both the background-only and the Standard Model hypotheses.

Quantity	pyhf	Combine
$\hat{\mu}$ (profile-scan $68\%$ )	$1.19^{+1.28}_{-1.04}$	$1.17^{+1.27}_{-1.02}$
$k_{t\bar{t}}$	$0.653\pm 0.078$	$0.647\pm 0.080$
$Z$ (observed)	1.152	1.152

Appendix

Table A.2: Independent validation of the

D_{NN}

fit: the pyhf result and its reproduction in the CMS Combine framework on the point-by-point identical statistical model. The best-fit signal strength

\hat{\mu}

, the in-situ

t\bar{t}

normalization

k_{t\bar{t}}

, and the observed discovery significance

Z

are listed; the signal-strength intervals are the like-for-like profiled

68\%

(

-2\Delta\ln\mathcal{L}=1

) intervals from each engine’s profile-likelihood scan, quoted as offsets from the best fit (the pyhf headline

\hat{\mu}=1.200\pm 1.128

is the symmetric Hessian uncertainty). The profiled

-2\Delta\ln\mathcal{L}(\mu)

scans overlay to a maximum difference of

0.055

across the physical range, and the likelihoods agree to

\leq 0.005

at every template knot.

Panel of Figure 16 — Figure A.5: Corrected primary Lund jet plane density $\rho(\ln 1/\Delta\theta,\,\ln k_{t})$ at charged-particle level (left), measured on the 1994 ALEPH peak data, and the matching map of the total relative uncertainty $\sigma_{\rm tot}/\rho$ over the same bins (right). The density displays the canonical triangular structure; the central perturbative region is measured to a few percent, while the larger uncertainties are confined to the kinematic edges. The soft, wide-angle corner (upper left of the populated region) carries no underlying-event, MPI, or pileup contamination and is therefore pure QCD radiation and hadronization.

Figure A.6: Corrected data (left; points, grey total-uncertainty band) versus the standalone PYTHIA 8 Monash, Vincia, and default string tunes and the Sherpa AHADIC cluster prediction, with the density projected onto

\ln k_{t}

over the fiducial

\ln(1/\Delta\theta)

range; the lower panel shows the model/data ratio, the grey band giving the data total uncertainty about unity. The shaded colored bands are the assumed per-bin generator-theory uncertainty, scaling from

\sim 6\%

at high

k_{t}

\sim 20\%

at the soft edge following the next-to-leading-log LSS calculation [ lifson2020ljp ] . All modern tunes lie

9

–

18\%

below the data (occupancy-weighted, like-for-like), the deficit largest for the Sherpa cluster model, and the rise toward lower

k_{t}

reflects the running of

\alpha_{s}(k_{t})

. Post-normalization residual

\rho_{\rm data}/(k_{g}\,\rho_{\rm gen})-1

(right, in percent), projected onto

\ln k_{t}

over the same range after rescaling each generator by its single best-fit normalization

k_{g}

(legend); error bars are the projected data total uncertainty and points are staggered horizontally for clarity, and the grey band is the same per-bin generator-theory uncertainty centered on zero. After the normalization the residual is a shallow U-shape rising toward both

k_{t}

ends, resolved in the data covariance (

\chi^{2}/\mathrm{ndf}=3.4

–

6.7

) but contained within the theory band (

\chi^{2}/\mathrm{ndf}=0.4

–

0.6

after the band,

p\simeq 1

): the offset is predominantly an overall normalization plus a differential shape within current generator-theory uncertainty, within which the PYTHIA 8 tunes are statistically consistent with the data and Sherpa remains the most discrepant.

Explore similar work

Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction
May 13, 2026Darius A. Faroughy, Sofia Palacios Schweitzer, Ian Pang +2AI Agents for Scientific Discovery AI Agent Evaluation
New High Energy Theory Center Department of Physics & Astronomy Rutgers University · Faculty of Computing & Data Sciences Boston University
HepScript: A Dual-Use DSL for Human-AI Collaborative Data Analysis Workflows in High-Energy Physics
May 2, 2026Junkun Jiao, Tong Liu, Ke Li +6Scientific Code Generation High-Energy Physics
Jilin University, Changchun, Jilin, China · Institute of High Energy Physics, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Agentic AI -- Physicist Collaboration in Experimental Particle Physics: A Proof-of-Concept Measurement with LEP Open Data
Mar 5, 2026Anthony Badea, Yi Chen, Marcello Maggi +2AI Agents for Scientific Discovery Data Analysis Agents
University of Chicago, Enrico Fermi Institute, 60637 IL, USA · Vanderbilt University, Nashville, Tennessee, USA · Istituto Nazionale di Fisica Nucleare, Bari Division, BA 70126, Italy +1

May 13, 2026J/K move · Enter open · S save

Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction

Darius A. Faroughy, Sofia Palacios Schweitzer, Ian Pang, Siddharth Mishra-Sharma+1

New High Energy Theory Center Department of Physics & Astronomy Rutgers University · Faculty of Computing & Data Sciences Boston University

AI Agents for Scientific Discovery AI Agent Evaluation AI Agent Benchmarks Scientific Reproducibility

Open paper AskPDF arXiv

Autonomous language-model agents are increasingly evaluated on long-horizon tool-use tasks, but existing benchmarks rarely capture the complexity and nuance of real scientific work. To address this gap, we introduce Collider-Bench, a benchmark for evaluating whether LLM agents can reproduce experimental analyses from the Large Hadron Collider (LHC) using only public papers and open scientific software. Such analyses are often difficult to reproduce because the public toolchain only approximates the software used internally by the experimental collaborations, while the published papers inevitably omit implementation details needed for a faithful reconstruction. Agents must therefore rely on physical reasoning, domain knowledge, and trial-and-error to fill these gaps. Each task requires the agent to turn a published analysis into an executable simulation-and-selection pipeline and submit predicted collision event yields in specified signal regions. These predictions are evaluated with standard histogram metrics that provide continuous fidelity scores without a hand-written rubric. We also report the computational cost incurred by each agent per task. Finally, we evaluate the codebase and full session trace using an LLM judge to catch qualitative failure modes such as fabrications, hallucinations and duplications. We release an initial set of tasks drawn from LHC searches, together with a containerized sandbox and event simulation tools. We evaluate across a capability ladder of general purpose coding agents. Our results show that on average no agent reliably beats the physicist-in-the-loop solution.

Abstract

Explore similar work

Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction

HepScript: A Dual-Use DSL for Human-AI Collaborative Data Analysis Workflows in High-Energy Physics

Agentic AI -- Physicist Collaboration in Experimental Particle Physics: A Proof-of-Concept Measurement with LEP Open Data