HERA: Assessing an Autonomous Agent for Quantum Device Measurement

Zachary W. Gomez; Nathan Bittner; Jordan T. McCourt; Qingrui Cao; Sandesh S. Kalantre; Benjamin H. Alexander; Zhiren Zheng; Kenji Watanabe

Figure 1: HERA. ( a ) HERA is an LLM agent designed to conduct autonomous cryogenic transport measurements on quantum devices. The agent prompt holds the standing instructions HERA follows in every measurement session. A session starts from its inputs: a campaign brief, a device card, and a rig card. The brief states the campaign’s goals. The cards are machine-readable datasheets for the device being measured and the rig measuring it. HERA draws on a corpus of recipes, software tools, and literature references. A recipe outlines an experimental protocol and points to the examples, tools, and related recipes. The corpus was seeded from the lab’s historical data. HERA writes code into two notebooks, one for measurement and one for analysis. Code in the measurement notebook drives the rig and records the raw data. HERA analyzes the raw data and reports its results in the analysis notebook. HERA writes problems it encounters and opportunities for improvement to its session notes. The chat transcript records every action HERA takes and every message it exchanges with the operator. The notebooks, raw data, session notes, and chat transcript are the session’s outputs. Together these form the session archive, which feeds the self-improvement loop. ( b ) After each session, the operator launches a review agent that reads the archive for findings and checks it against the fix log. This check validates previous fixes and escalates small issues that recur. Each finding becomes a proposed revision to the corpus or the agent prompt, or is deferred to the fix log. The operator accepts or rejects each proposed revision, and the fix log records each revision.

Figure 2: HERA ’s measurement sequence for stacking order ( a ) Timeline of HERA’s session on the trilayer device, from the brief to the spot check in ( b ), in active hours. The first break in the axis excludes an 11.4 h wait for the operator’s ruling on a safety concern HERA raised. The second break excludes the 6.4 h between HERA’s conclusion and the operator’s request for the fan in ( c ). ( b ) Background-subtracted three-terminal resistance

\Delta R_{3T}

as a function of inverse magnetic field

1/B

at carrier density

n=-1.74\times 10^{12}

cm -2 and displacement field

D=0

. In this spot check,

\Delta R_{3T}

exhibits Shubnikov–de Haas (SdH) oscillations, which are used to discriminate between stacking orders (Sec. S3 ). ( c ) Three-terminal resistance

R_{3T}

as a function of

n

and

B

at

D=0

. HERA measured this higher-resolution follow-up Landau fan at the operator’s request after its conclusion. Fig. S2 shows the fan HERA concluded from and the expert’s fan. ( d ) Fourier amplitude

|\mathrm{FFT}|

of each constant-density linecut of ( c ), each normalized to its maximum, as a function of

n

and SdH frequency

F

. The lines indicate the frequencies expected for ABC and ABA stacking at the density calibration HERA set in the session.

Figure 3: Live sessions with the full and ablated corpus. ( a ) The six live sessions asked to determine the stacking order of a multilayer graphene device. The two shaded sessions are the blinded prospective trial (Sec. III ). The four unshaded sessions are the ablation study (Sec. IV ). The ablated corpus kept only the software tools of the full corpus. Columns show agreement with the expert’s sealed conclusion, operator interventions before the conclusion, active and wall-clock time, and cost at API list prices (Methods). Bold indicates the best value in each column. The interventions for all sessions are listed in Sec. S5 . ( b ) Expert grades of the five tetralayer sessions. Top, the mean grade of each session’s inference, decision, and verdict claims (Methods). Bottom, how well each session answers the brief and how much the expert would trust the same agent in future sessions. Both grades use the rubric’s four options, from worse to better (Methods). Table S2 shows the claim score and both whole-session grades of every session. Fig. S3 shows the expert’s ruling on each step of each session. For Opus 5 with the full corpus, the grades in ( b ) cover the whole session, including a finer Landau fan the operator requested after its first conclusion. Panel ( a ) shows its first conclusion, ABAB at lower confidence than the expert’s, and its final conclusion after the finer fan was ABAB at the expert’s confidence.

Figure 4: Offline replay of Landau fan design. ( a )

P(\text{call})

for six models, each designing up to twenty fans with the full corpus and up to twenty with the ablated corpus (Methods).

P(\text{call})

estimates whether a fan would discriminate between multilayer graphene stacking orders. A fan planned to run longer than an eight-hour cutoff counts as zero. Bars show each model’s mean with a 95% bootstrap interval. ( b )

P(\text{call})

of all fans as a function of their planned acquisition time. Lines mark the Pareto frontiers of the fans with the full and the ablated corpus. A fan is on the frontier when no other fan in its group scores higher in less time. The dashed line indicates the eight-hour cutoff applied in ( a ). ( c ) Frequency coverage as a function of sampling rate for each model with the full and the ablated corpus. Sampling rate is the number of field points per period of the highest expected SdH frequency on the fan’s highest-density linecut. Frequency coverage is the number of Fourier-transform frequency bins that the lowest expected SdH frequency covers across the linecuts that resolve it. Points give each model’s mean and ellipses one standard deviation, and arrows run from the ablated corpus to the full corpus. The dashed line indicates the three points per period that the corpus’s Landau fan recipe recommends.

Figure 5: HERA’s search for superconductivity. ( a ) Timeline of session 6 on the WSe 2 -proximitized bilayer graphene device, from the grant of autonomy to HERA’s conclusion, in active hours. ( b ) Differential resistance

\mathrm{d}V_{xx}/\mathrm{d}I

as a function of dc current

I_{\mathrm{dc}}

at four temperatures, acquired autonomously by HERA in the session in ( a ). ( c ) Longitudinal resistance

R_{xx}

as a function of

n

and

D/\epsilon_{0}

, measured in the additional session that HERA ran with the operator in the loop (Sec. S7 ). The diagonal black line denotes the boundary of the accessible gate range. Grey regions denote accessible regions of gate space outside the measured map. Dashed lines denote the cuts along which session 2 (orange) and session 3 (red) searched for superconductivity. The red circle denotes session 3’s chosen operating point for subsequent measurements. Session 2’s cuts reached only

D/\epsilon_{0}=-0.60

V/nm, just above where

R_{xx}

begins to vanish, and it concluded that the device does not superconduct. Session 3, running with an additional coverage rule, added the sweep along the gate limit.

Figure S1: Optical micrographs of the four devices. The current path (I) and voltage contacts (V) used by HERA are drawn on each image. ( a ) Device 1, the trilayer graphene device used in the blinded trial. ( b ) Device 2, the tetralayer graphene device used in the blinded trial. ( c ) Device 3, the trilayer graphene device used in the development campaign. ( d ) Device 4, the Bernal bilayer graphene/WSe 2 device used in the search for superconductivity. Scale bars, 5

\mu

m.

Experiment	Model	Model id	Effort	Harness	Corpus	Mode
Blinded trial
Device 1	Fable 5	claude-fable-5	high	CC 2.1.235	d8e1337	E1
Device 2	Fable 5	claude-fable-5	high	CC 2.1.239	13b07e9	E1
Corpus $\times$ model, device 2
Full corpus, repeat	Fable 5	claude-fable-5	high	CC 2.1.241	13b07e9	E1
Full corpus	Opus 5	claude-opus-5	high	CC 2.1.243	13b07e9	E1

Table S1: Models, harness, and corpus of each experiment. One row per live session, and one per model for the offline replay. Model id is the model field of the session transcript, or of the replay case file and every replay transcript. Effort is the reasoning-effort setting the harness recorded; the harness versions of June 2026 record none, though their transcripts contain the model’s thinking. Harness is the Claude Code (CC) or Codex CLI version. Corpus is the commit of the lab repository the session ran on: d8e1337 is the corpus frozen for the trial, 13b07e9 the same corpus with the brief and device card for device 2 added, and 039b0f5 the ablated corpus. Sessions before 17 July 2026 did not record their commit, so for the transfer and development sessions the commit given is the last in the repository before the session’s transcript begins. Mode is the autonomy mode (Methods); n/a marks the offline replay, and a dash a value that was not recorded. The conversation record of transfer session 4 was lost, so its model is not recorded; the setup agent that launched it ran claude-opus-4-8 . Transfer session 5 survives as a terminal export that names Opus 4.8 but not the model id. The reference session survives as a terminal export and the harness’s own record, which name its model. Run 6 of the development campaign, a false start, is omitted (Fig. S7 ).

Figure S2: The Landau fans behind the trilayer conclusions. ( a ) The fan behind HERA’s stacking-order conclusion. Three-terminal resistance

R_{3T}

as a function of

n

and

B

at

D=0

, from 1.6 to 5 T uniform in

1/B

, acquired in 1.7 h at 1 nA with a 100 ms time constant. ( b ) Fourier amplitude

|\mathrm{FFT}|

of each constant-density linecut of ( a ), each normalized to its maximum, as a function of

n

and SdH frequency

F

. The lines indicate the frequencies expected for ABC and ABA stacking at the density calibration HERA set in the session. ( c ) The fan behind the expert’s sealed conclusion. Four-terminal longitudinal resistance

R_{xx}

as a function of

n

and

B

, measured on the same device and cooldown before HERA’s session.

Session
	Blinded trial		Corpus $\times$ model, device 2
Device (layers)	1 (3L)	2 (4L)	2 (4L)	2 (4L)	2 (4L)	2 (4L)
Model	Fable 5	Fable 5	Fable 5	Opus 5	Fable 5	Opus 5
Corpus	full	full	full, repeat	full	ablated	ablated
Start (2026)	Aug 18	Aug 21	Aug 22	Aug 24	Aug 23	Aug 24
Sweeps, before $+$ after conclusion	19 + 2	16 + 0	15 + 0	8 + 1	19 + 0	13 + 0

Table S2: The six live stacking-order sessions. All values for the six main-text sessions that determined the stacking order of a multilayer graphene device (Figs. 2 and 3 ): HERA’s conclusion and its confidence level, agreement with the expert’s sealed conclusion, interventions before and after the conclusion, active and wall-clock time, cost at list price, the claim score, and the two expert letter grades, for whether the work answers the brief and for the supervision a further session would need. Sweeps counts the number of measurements in the session archive before and after HERA’s conclusion; the sweeps after are any measurements the operator requested. Expert agrees is yes when stacking order and confidence level both match the sealed conclusion, order only when the order matches at a lower confidence level, and no when the order differs. Standby is the time HERA spent waiting on the operator or a provider outage, and sweep time after conclusion the duration of the runs the operator requested after it (Sec. S5 ). Claims and items are the counts the expert graded; inference, decision, and verdict are the mean grades by claim type that Fig. 3 b plots (Methods).

Figure S3: Session chains of the five tetralayer sessions. One strip per session in the order of Fig. 3 , every graded notebook unit in reading order with context units omitted, colored by the expert’s whole-session ruling on the unit: sound, shaky, wrong, or not applicable. During the whole-session read, the expert also ruled each notebook unit sound, shaky, or wrong, or marked it not applicable. A heavy outline marks a unit that holds a graded inference or verdict. Labels abbreviate the unit’s step.

Stratum	Items	Exact agreement	Weighted $\kappa$
All	145	0.648	0.383
Inference claims	74	0.568	0.336
Decision claims	51	0.784	0.529
Verdict claims	20	0.600	0.292
Device 1, Fable 5, full corpus	24	0.792	0.524
Fable 5, full corpus	24	0.708	0.604

Table S3: Agreement between the two experts by claim type and by session. Items counts the items that neither expert ruled unverifiable, 145 of the 212 in the subset.

\kappa

is quadratic-weighted Cohen’s

\kappa

on item values 1, 0.5, and 0. At the claim level, with a claim’s value the mean of the items that expert scored, 53 claims have a value from both experts: exact agreement 0.453, weighted

\kappa

0.508.

Session	Claims in subset	Claim score on the subset		Claim score of record
		first expert	second expert	(all claims, first expert)
Device 1, Fable 5, full corpus	11	0.941	0.902	0.905
Fable 5, full corpus	10	0.840	0.848	0.774
Fable 5 repeat, full corpus	10	0.751	0.804	0.853
Opus 5, full corpus	8	0.781	0.824	0.801
Fable 5, ablated corpus	7	0.525	0.802	0.543

Table S4: Claim score of each session on the 56-claim subset, under each expert. Subset scores are computed as in Methods from each expert’s own items over the 53 claims both scored, verdict claims weighted double; the last column is the score of record on the full pool. Under both experts every session with the full corpus outscores both sessions with the ablated corpus. Under the second expert the margin is 0.002 (0.804, Fable 5 repeat, full corpus; 0.802, Fable 5, ablated corpus).

Session	Mode	Hours	Interv.	$n$	$D/\epsilon_{0}$	Call	$I_{c}$ (nA)	$B_{c\perp}$ (mT)	$T_{c}$ (mK)
				Operating point			Reported
1	E0	22.7	10	—	—	no call	—	—	—
2	E0	6.3	1	$-5.17$	$-0.673$	not-SC, retracted; SC	8–11	2.5	100–200
3	E0	6.6	0	$-5.01$	$-0.650$	SC (decisive)	18–24	2.5	234
4	E0	9.1	3	$-5.07$	$-0.643$	SC (strong)	30	2.5–3	300 (onset)
5	E1	4.6	3	$-4.80$	$-0.599$	consistent-with, hedged	12–15	—	—

Table S5: HERA sessions on the WSe 2 -proximitized bilayer graphene device. One row per measurement session, in chronological order. Mode is the autonomy mode (Methods). Hours is the wall-clock time of the session and Interv. the number of operator interventions during it. The operating point (

n

in

10^{11}

cm -2 ,

D/\epsilon_{0}

in V/nm) is the chosen location where the session performed its characterization of the superconductivity, recomputed from its gate voltages with the reference session’s calibration. Reported values are those HERA returned in its end-of-session conclusion, where SC denotes superconducting. Session 1 did not commit to a conclusion; session 5 was ended by the operator prior to the temperature ladder. The reference session was run by HERA in collaboration with the operator and is not scored.

Figure S4: Superconductivity in WSe 2 -proximitized bilayer graphene. ( a ) Longitudinal resistance

R_{xx}

as a function of density

n

and displacement field

D/\epsilon_{0}

at 34 mK and effective zero field. The dashed line denotes

D/\epsilon_{0}=-0.64

V/nm (used in b ) and the cyan cross the operating density

n=-5.0\times 10^{11}

cm -2 used in c and d . ( b )

R_{xx}

as a function of

n

and temperature along the dashed line of ( a ). Extracted Ginzburg–Landau coherence length

\xi_{\mathrm{GL}}(n)

from the 50 % criterion of Fig. S5 is overlaid on the right axis. Error bars correspond to one standard deviation of the fit uncertainty. ( c ) Differential resistance

dV/dI

as a function of d.c. bias and temperature at the operating point indicated in a . ( d )

dV/dI

as a function of bias and perpendicular field at 34 mK; the field axis is relative to the effective zero field at the sample. All data were acquired by HERA while collaborating with the operator.

Figure S5: Ginzburg–Landau analysis of WSe 2 -proximitized bilayer graphene. ( a )

R_{xx}(T)

at

n=-5.0\times 10^{11}

cm -2 and

D/\epsilon_{0}=-0.64

V/nm for perpendicular fields of

0

,

0.75

,

1.00

and

1.25

mT. Solid lines are linear fits to the normal state resistance above 260 mK, dashed lines are half of each fit, and circles mark the transition temperature

T_{c}

where the data cross the dashed line. ( b )

T_{c}

as extracted from when

R_{xx}(T,B)

reaches 50% of the normal state resistance as a function of field with the Ginzburg–Landau fit

T_{c}/T_{c0}=1-2\pi\xi_{\mathrm{GL}}^{2}B/\Phi_{0}

, giving

B_{c2}(0)=3.0

mT and

\xi_{\mathrm{GL}}=329\pm 9

nm. ( c )

\xi_{\mathrm{GL}}

across the superconducting dome extracted from the 50 % criterion of b (blue) and the 90 % onset criterion (grey); thin bars span the 40–60 % and 85–95 % criteria, respectively. Faint bars correspond to one standard deviation of the fit uncertainty in the extracted

\xi_{\mathrm{GL}}

. ( d ) Zero-field

T_{c0}

from the same fits.

Figure S6: Survey for superconductivity in WSe 2 -proximitized bilayer graphene. Each session’s survey for superconductivity is overlaid on the zero-field

R_{xx}(n,D)

map of the reference session in greyscale. Every session opened with a low-resolution survey of the full gate window, which is not drawn. Blue lines denote the line cuts the session ran; orange outlines denote the limits of two-dimensional maps. The dashed black line indicates the limit of the accessible gate range. The red circle denotes the session’s operating point chosen for subsequent characterization. The label in each panel is the session’s conclusion on the presence of superconductivity.

Model	6 h	8 h	12 h	No cutoff
Fable 5	$+0.24$	$+0.10$	$-0.02$	$+0.24$
Opus 5	$+0.26$	$+0.31$	$+0.35$	$+0.35$
Fable 5.1	$+0.56$	$+0.72$	$+0.73$	$+0.73$
Opus 5.5	$+0.54$	$+0.18$	$+0.13$	$+0.13$
GPT-6 Astra	$+0.50$	$+0.46$	$+0.10$	$+0.06$
GPT-6 Sol	$+0.81$	$+0.81$	$+0.84$	$+0.84$

Table S6: Gain in

P(\text{call})

with the full corpus at each pre-registered cutoff. Each entry is the model’s mean

P(\text{call})

over its full-corpus fans minus the mean over its ablated-corpus fans, with a fan planned to run longer than the cutoff counted as zero. The eight-hour cutoff is the primary one, and the others are pre-registered sensitivity analyses (Methods). The pooled gain is the mean over the six models, and each pooled gain has an unadjusted permutation

p=0.0001

. Between the pre-registered cutoffs, Fable 5’s gain is

+0.19

at seven hours and

-0.13

at nine.

Figure S7: The development campaign on device 3. HERA ran nine sessions on device 3, a trilayer, between 10 and 22 June 2026, and four are shown, numbered by run. Run 6, a false start, is not counted. Beneath each session is the problem it exposed, in either SdH analysis (sdh) or fan design (fan), and the two rows below hold the fixes that followed. A diamond marks a revision of the corpus by the review agent (Methods), and the fixes shown are a sample. The onset tool also failed in sessions 8 and 10, and it was rewritten in July, after the campaign. Sessions 1 and 2 ran in E0 (Methods), with Fable 5 for session 1 and Opus 4.8 after. The last three sessions ran the same brief with no revision of the corpus between them. Their conclusions varied between ABA and ABC, because the light ABA band lies below resolution at 1.2 K and 8 T on this device.

HERA: Assessing an Autonomous Agent for Quantum Device Measurement

Abstract

Figures & tables

Autonomous Experimentation

Collaborative AI agents for autonomous experimental research with solid-state quantum spins

Hierarchical Experimentalist Agents

Stress-testing large language model agents in a robotic chemistry laboratory

Abstract

Autonomous Experimentation

Explore similar work

Collaborative AI agents for autonomous experimental research with solid-state quantum spins

Hierarchical Experimentalist Agents

Stress-testing large language model agents in a robotic chemistry laboratory