LabCompiler: A Framework for Verifiable Human-Machine Laboratory Execution from Imperfect Protocols

Jingyuan Li; Dongzhan Zhou; Minting Pan; Yujie Liu; Yuan Gao; Yankai Jiang; Xinyu Gu; Zhe Liu

cs.AIOct 14, 2026

LabCompiler: A Framework for Verifiable Human-Machine Laboratory Execution from Imperfect Protocols

Authors: Jingyuan Li, Dongzhan Zhou, Minting Pan, Yujie Liu, Yuan Gao, Yankai Jiang, Xinyu Gu, Zhe Liu,

Organizations: Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University · Shenzhen Institutes of Advanced Technology ·

Abstract

Closing the loop of AI-driven science requires reliably translating natural-language protocols into executable laboratory workflows despite protocol defects and heterogeneous hardware. Existing approaches often couple scientific interpretation with device-specific code generation, making errors difficult to localize and limiting reuse across laboratory platforms. We introduce LabCompiler, a robust protocol-to-execution framework that transforms imperfect natural-language protocols into verifiable human-machine workflows. Its protocol firewall diagnoses and repairs defects before compilation while preserving unaffected content, guided by troubleshooting knowledge accumulated through perturbation-guided self-evolution. A two-level, device-agnostic graph representation separates experimental semantics from hardware-specific implementation and supports capability-based device assignment with coordinated human execution. We further introduce a protocol recovery benchmark covering four controlled perturbation types and naturally occurring errors collected from protocols written by iGEM participants. Experiments show that LabCompiler outperforms frontier LLMs in protocol error correction while enabling coordinated human-machine execution across heterogeneous, partially automated laboratory settings.

Figures & tables

Figure 1: Overview of LabCompiler. Unlike traditional workflows that rely on manual interpretation and operation, LabCompiler transforms ambiguous natural-language protocols into human-machine workflows through diagnosis and repair, device-agnostic abstraction, and hybrid execution.

Figure 2: Overview of LabCompiler. It first screens and repairs ambiguous natural-language protocols, then converts them into device-agnostic semantic and canonical representations. The resulting actions are grouped into execution units and capability-matched to human or automated execution.

Perturbation	Construction	Illustrative example
Parameter or reagent replacement	Replace a valid experimental parameter or reagent with an incompatible alternative.	Incubate at $37^{\circ}\mathrm{C}$ $\rightarrow$ incubate at $4^{\circ}\mathrm{C}$ .
Critical-step deletion	Remove an operation required to establish the state assumed by a downstream step.	Delete the quenching step before washing labeled cells.
Cross-step inconsistency	Modify a reference, quantity, or material such that two distant steps become inconsistent.	Prepare $N$ samples but later use reagents for $M$ samples.
Operation or parameter ambiguity	Replace an explicit operation or quantitative specification with an expression insufficient to determine the intended procedure.	Add $500\,\mu\mathrm{L}$ of PBS $\rightarrow$ add an appropriate amount of buffer.

Table 1: Taxonomy of controlled protocol perturbations.

Method	Critical-step deletion		Operation or parameter ambiguity		Parameter or reagent replacement		Cross-step inconsistency
Method	Acc. $\uparrow$	Rep. $\uparrow$	Acc. $\uparrow$	Rep. $\uparrow$	Acc. $\uparrow$	Rep. $\uparrow$	Acc. $\uparrow$	Rep. $\uparrow$
MiniMax-M3	68.0	26.0	26.6	15.0	45.0	29.0	38.2	39.9
Qwen3.8-Max	65.2	29.0	32.0	20.0	55.0	35.4	45.0	44.6
DeepSeek-V4-Pro	69.8	37.3	37.0	23.7	49.0	35.0	47.4	47.7
Gemini-3.1-Pro	68.0	36.4	39.8	22.0	52.0	32.0	51.4	45.9
Kimi-K3	74.0	36.0	42.4	24.0	58.2	38.5	44.6	42.2

Table 2: Protocol correction on the controlled perturbation subset across four error categories. We report classification accuracy (Acc.) and repair score (Rep.). LabCompiler* denotes the self-evolved variant. Best and second-best results are shown in bold and underline , respectively.

Figure 3: Average performance of protocol correction on the controlled perturbation subset (a,b) and the real-world error subset (c,d). LabCompiler* denotes the self-evolved variant. LabCompiler consistently improves both error classification and semantic repair over direct LLM baselines.

Figure 4: Fidelity and stability of the device-agnostic representation. Original bars measure fidelity to the source protocol, while the other bars measure stability under five semantics-preserving rewrites, evaluated by (a) operation coverage, (b) detail preservation, and (c) relation preservation.

Figure 5: Execution performance across route accuracy, device executability, and code fidelity. LabCompiler consistently outperforms direct LLM baselines across all three metrics.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Protocol defect repair	Explicit protocol verification	Device-agnostic IR	Hierarchical representation	Device capability grounding	Human hybrid workflow	Auditable intermediate artifacts
XDL ( Mehr et al., 2020 )	Partial	Supported	Supported	Partial	Partial	Not explicitly reported	Supported
LabOP ( Bartley et al., 2023 )	Not explicitly reported	Supported	Supported	Partial	Partial	Supported	Supported
BPL ( Song et al., 2026 )	Partial	Supported	Supported	Partial	Supported	Supported	Partial
BioProAgent ( Liu et al., 2026b )	Partial	Supported	Partial	Partial	Supported	Partial	Partial
ProtoAct ( Liu et al., 2026c )	Partial	Partial	Partial	Partial	Supported	Partial	Partial
ProtoPilot ( Jiang et al., 2026 )	Supported	Supported	Partial	Partial	Supported	Partial	Supported

Appendix

Table A.1: Qualitative summary of explicitly reported capabilities in closely related laboratory automation systems. Hierarchical representation refers to systems that expose multiple abstraction levels or explicit transformation stages between protocol descriptions and execution representations. Human hybrid workflow support refers to explicit coordination between automated and human-performed operations rather than the general possibility of manual intervention.

Category	Check	Failure handling
Report structure	Required screening fields and protocol artifact consistency	The screening report is rejected and returned for revision
Experimental content preservation	Preservation of explicit operations, materials, preparation commitments, and protocol constraints	The candidate protocol is rejected if valid content is removed or altered without justification
Consistency checking	Consistency among steps, materials, parameters, and dependencies	Feedback is returned for another repair attempt
Ambiguity classification	Classification of remaining ambiguities into blocking or deferred cases	Blocking ambiguity leads to unresolved status; deferred ambiguity may be carried forward
Evidence-supported completion	Completion of missing information only when supported by available evidence or justified inference	Unsupported assumptions remain unresolved rather than being inserted

Appendix

Table B.2: Summary of protocol firewall screening criteria.

Field	Type	Description
operation_id	String	Unique identifier of the semantic operation.
operation_type	String	Semantic operation type before canonical action normalization.
source_text	String	Exact excerpt from the approved effective protocol supporting the operation.
objects	Array	Logical materials, samples, containers, or collections participating in the operation, together with their semantic roles.
parameters	Object	Quantitative parameters, experimental conditions, and other operation-level values explicitly represented by the protocol.
dependencies	Array	Identifiers of predecessor operations.

Appendix

Table C.3: Operation-level fields of the Semantic Experiment Graph.

Field	Description
normalized_operation_id	Unique identifier of the normalized operation.
canonical_action	One of the 26 canonical action identifiers listed in Table C.5 .
sequence_index	Position of the operation in the normalized operation sequence.
derived_from_semantic _operation_ids	Semantic operations from which the canonical operation is derived.
source_texts	Source excerpts corresponding to the contributing semantic operations.
objects	Mapping between action-specific logical roles and logical objects.

Appendix

Table C.4: Operation-level fields of the Canonical Action Graph.

Canonical action	Semantics and logical roles	Key semantic restriction
liquid_transfer	Move liquid from one logical source to one logical destination.	Single-source, single-destination liquid movement; it does not represent mixing or liquid disposal.
liquid_distribution	Distribute liquid from one source to multiple logical destinations.	Used when one source supplies a logical destination collection.
liquid_consolidation	Collect liquid from multiple logical sources into one destination.	Distinguished from a single-source liquid transfer.
liquid_removal	Remove liquid from a source to a discard destination; a protected material may remain in the source.	Represents removal rather than productive transfer of the removed liquid.
non_liquid_sample_transfer	Move a discrete non-liquid sample to a logical destination.	Used for non-liquid sample movement rather than liquid transfer.
solid_combination	Combine a solid component and a liquid component into a resulting product.	Material phase must be supported by the interpreted protocol.

Appendix

Table C.5: Complete vocabulary of the 26 canonical actions, including their experimental semantics, logical roles, and key semantic restrictions.

Criterion	Grouping rule
Local experimental objective	Actions that jointly accomplish the same local experimental operation may be grouped when no intervening boundary requires independent execution.
Material continuity	Actions operating on the same logical sample or material state may remain in the same unit when the intermediate state does not need to be exposed to another unit.
Dependency structure	The original prerequisite relations in $G_{\mathrm{can}}$ are preserved. Grouping does not remove a dependency required by a downstream operation.
Control boundary	A waiting period, incubation boundary, conditional operation, experimental endpoint, or measurement whose result is consumed downstream may separate execution units.
Resource continuity	Actions may be grouped only when a compatible realization exists for the resulting unit. A required change of executor or physical carrier can therefore introduce a unit boundary.
Intermediate-material use	When an intermediate material or state is independently consumed by another part of the workflow, its producing and consuming operations remain separated by an explicit dependency boundary.

Appendix

Table D.6: Execution-unit grouping rules.

Mode	Eq. (3) executor	Retained routing information
Automated	$d\in\mathcal{D}$	Canonical-operation lineage and dependencies, compatible-device set, physical-resource bindings, generated backend artifact, and validation result.
Human-coordinated	HUMAN	Source operation and canonical-operation context, dependencies, human-execution instruction, relevant runtime inputs, and optional device-class or device-model guidance.

Appendix

Table D.7: Execution modes and corresponding routing information.

Device	Backend family	Supported canonical actions	Integration	Key constraints	Validation
Opentrons OT-2	Opentrons	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing , orbital_shaking , temperature_control , incubation , thermal_cycling , magnetic_separation	Adapter + compiler	Positive liquid volumes; mixing volume up to 1000 uL; shaking 200–3000 rpm; temperature, thermal-cycling, and magnetic-module constraints are checked when applicable.	Official sim.
Opentrons Flex	Opentrons	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing	Adapter + compiler	Positive liquid volumes and compatible liquid-handling resources.	Official sim.
Hamilton STAR	PyLabRobot	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing	Adapter + compiler	Eight-channel liquid handling; volumes up to 1000 uL; compatible plate, reservoir, and tube-rack resources.	Offline sim.
Hamilton Nimbus	PyLabRobot	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing	Adapter + compiler	Eight-channel liquid handling; volumes up to 1000 uL.	Offline sim.
Hamilton Vantage	PyLabRobot	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing	Adapter + compiler	Eight-channel liquid handling; volumes up to 1000 uL.	Offline sim.
Tecan EVO	PyLabRobot	liquid_transfer , liquid_distribution , liquid_consolidation , liquid_removal , pipette_mixing	Adapter + compiler	Eight-channel liquid handling; volumes up to 1000 uL.	Offline sim.

Appendix

Table D.8: Device capability matrix.

Compatibility component	Requirement
Action compatibility	Every canonical action in the execution unit must have a supported realization on the candidate device.
Object and resource compatibility	Required logical objects, carriers, labware types, and input/output shapes must be compatible with the device interface.
Parameter compatibility	Required parameters must be present where necessary and satisfy the applicable unit, range, and device-capability constraints.
Backend compatibility	An applicable adapter and backend realization must exist for the action–device combination.
Validation compatibility	The realization must admit the validation pathway associated with the backend, including simulator validation or contract validation as applicable.

Appendix

Table D.9: Checks contributing to execution-unit/device compatibility.

Validation level	What is checked	Interpretation
Static and device-constraint validation	Action mapping, required parameters, object/resource compatibility, device-capability constraints, backend contract, and physical-layout consistency when applicable.	Required for an automated realization.
Simulator validation	Generated backend artifact is evaluated using the simulator associated with the selected device backend.	Applied when a simulator is available.
Contract validation	The normalized device command is checked against the registered device-command and parameter contract.	Used for backends without an applicable simulator.

Appendix

Table D.10: Validation levels for automated realizations.

Perturbation	Construction rule	Quality-control constraint
Critical-step deletion	Delete one complete operation required to establish a state used by a downstream step.	All remaining source steps retain their original content and relative order, except for necessary step renumbering. The deletion must create the intended missing prerequisite rather than modifying unrelated content.
Operation or parameter ambiguity	Retain the underlying operation while making one execution-critical operation or quantitative specification underdetermined, for example by removing a numerical value, unit, or required range.	No additional step is inserted or deleted, the step order is unchanged, and the target is not replaced by a different concrete reagent or parameter.
Parameter or reagent replacement	Replace one explicit experimental parameter or reagent identity with an incompatible alternative.	The underlying operation, number of steps, step order, and unrelated parameters remain unchanged.
Cross-step inconsistency	Modify a later reference to a sample, reagent, component, quantity, or state established by an earlier step.	The modified statement remains locally plausible while introducing a cross-step inconsistency. Unrelated steps are preserved.

Appendix

Table E.11: Construction rules for the four controlled perturbation categories. Each controlled instance contains one targeted perturbation.

Dataset / category	Accepted	Unique source protocols	Self-evolution	Dev.	Test	Expert audit
Critical-step deletion	600	600	100	0	500	100
Operation or parameter ambiguity	600	600	100	0	500	100
Parameter or reagent replacement	600	600	100	0	500	100
Cross-step inconsistency	600	600	100	0	500	100
Controlled total	2,400	– a	400	0	2,000	400
iGEM real-world subset	259	259	0	0	259	259

Appendix

Table E.12: Benchmark composition and data splits. Unique source protocols are reported within each perturbation category; the same source protocol may appear in different categories.

Perturbation category	One-pass repair	Iterative repair	$\Delta$
Critical-step deletion	38.1	39.8	+1.7
Operation or parameter ambiguity	27.9	31.2	+3.3
Parameter or reagent replacement	44.3	53.4	+9.1
Cross-step inconsistency	53.2	63.6	+10.4
Overall	40.9	47.0	+6.1

Appendix

Table F.13: Ablation of the iterative screening loop. One hundred test protocols are randomly sampled from each controlled perturbation category (400 protocols in total).

\Delta

denotes the improvement of the iterative loop over one-pass diagnosis and repair, in score points.

Figure F.1: Effect of self-evolved troubleshooting memory on protocol recovery. Classification accuracy (a) and correction score (b) are reported using 0, 50, and 100 self-evolution examples per perturbation category, corresponding to 0, 200, and 400 memory-construction examples in total, respectively. The resulting memories are frozen before evaluation on the same held-out test set. Curves show the overall result and the four controlled perturbation categories.

	Fidelity			Stability
Representation	Operation	Detail	Relation	Operation	Detail	Relation
Direct-to-canonical (w/o $G_{\mathrm{sem}}$ )	88.6	87.6	85.4	89.7	86.5	83.2
Two-level ( $G_{\mathrm{sem}}\rightarrow G_{\mathrm{can}}$ )	96.4	96.2	94.6	95.8	97.3	98.2
$\Delta$	+7.8	+8.6	+9.2	+6.1	+10.8	+15.0

Appendix

Table F.14: Ablation of the two-level intermediate representation. Higher values are better.

\Delta

denotes the improvement of the two-level representation over direct-to-canonical mapping.

Routing granularity	Execution-unit grouping	Route accuracy (%)	Device executability (%)
Action-wise routing	No	61.2	49.5
Execution-unit routing	Yes	72.9	59.6
$\Delta$	–	+11.7	+10.1

Appendix

Table F.15: Ablation of execution-unit grouping on the same 50 execution protocols.

\Delta

denotes the improvement obtained by grouping related actions before capability matching.

Metric	Evaluation unit	Raw scale	Within-protocol denominator	Final aggregation
Classification accuracy	Protocol	0/1	All evaluated controlled protocols	Mean correctness over protocols
Repair score	Protocol	0–2 per judge	All protocols with paired judge scores	Two-judge mean, linearly mapped to 0–100
IR fidelity	Protocol	0–5 per dimension	50 original protocols	Mean across protocols
IR stability	Rewrite	0–5 per dimension	100 rewrites per transformation	Average within source protocol, then across source protocols
Route accuracy	Execution unit	0/1	Reference execution units in each protocol	Protocol-level accuracy, then mean across protocols
Device executability	Automated execution unit	0/1	Reference automated execution units in each protocol	Protocol-level success rate, then mean across protocols

Appendix

Table G.16: Summary of evaluation metrics, within-protocol denominators, and final aggregation procedures.

Task	Score-audit fraction	Independent expert ratings per response
Protocol repair	30%	2
IR fidelity	30%	2
IR stability	30%	2
Code fidelity	30%	2

Appendix

Table G.17: Coverage of independent expert auditing of LLM-judge scores. The reported fractions refer only to score-level validation and are separate from the expert verification of representation-fidelity artifacts described above. Each sampled response was evaluated independently by two blinded domain experts using the same rubric as the corresponding LLM judges.

Evaluation	Source protocols	Evaluation instances	Notes
Controlled protocol recovery	2,000	2,000	500 protocols in each of four perturbation categories
iGEM protocol recovery	259	259	Independent real-world held-out subset
Representation fidelity	50	50	Source protocol $\rightarrow G_{\mathrm{can}}$
Representation stability	50	500 rewrites	10 rewrites per source protocol; 100 instances for each of five transformations
Execution grounding	100	643 execution units	376 Automated and 267 HUMAN reference units

Appendix

Table G.18: Evaluation populations for the main experiments.

Method	Accuracy	Precision	Coverage
Human	0.79	0.87	0.91
Claude 3.5 Sonnet	0.48	0.66	0.73
Claude 3 Opus	0.52	0.62	0.84
Gemini 1.5 Pro	0.47	0.58	0.81
GPT-4o	0.53	0.56	0.95
GPT-4 Turbo	0.37	0.59	0.62

Appendix

Table H.19: Results on the 108-question public ProtocolQA subset of LAB-Bench. These metrics follow the LAB-Bench task definition and should not be directly compared with the open-ended protocol-recovery metrics used in our main benchmark.

Laboratory Automation

Laboratory automation uses robotics, software, and AI to plan, execute, monitor, and analyze experiments with less manual intervention.

See all 62 papers

May 15, 2026

From Prompts to Protocols: An AI Agent for Laboratory Automation

Automating science laboratories enables faster, safer, more accurate, and more reproducible execution of protocols, accelerating the discovery and testing of new materials, drugs, and more. However, setting up and running autonomous labs requires coordinating numerous instruments and robots, forcing scientists to write code, manage configuration files, and navigate complex software infrastructure. We present an AI agent architecture that integrates large language models with laboratory orchestration, enabling scientists to interactively create and monitor automated lab protocols using natural language. Integrated into the Experiment Orchestration System (EOS), the AI agent operates under an agentic loop with automated validation and error correction, and supports the complete experimental lifecycle: creating protocols, running and monitoring both protocols and closed-loop optimization campaigns, and analyzing results. A visual graph editor renders protocols as interactive node-based diagrams synchronized with the AI agent's protocol representation, enabling seamless alternation between AI-assisted and manual protocol construction. Evaluated on three simulated automated labs spanning chemistry, biology, and materials science, the AI agent achieves a 97% first-attempt protocol generation success rate and an order of magnitude reduction in required interface actions.

Angelos Angelopoulos, James F. Cahoon, Ron Alterovitz

Department of Computer Science, University of North Carolina at Chapel Hill, NC 27599, USA · Department of Chemistry, University of North Carolina at Chapel Hill, NC 27599, USA

LLM Agent Orchestration Laboratory Automation Scientific Workflow Automation Autonomous Experimentation

Ask PDF

Jun 18, 2026

Dual-Agent Framework for Cross-Model Verified Translation of Natural-Language Protocols into Robotic Laboratory Platform

Biological experiment protocols are written in natural language, whereas automation systems rely on predefined control commands, creating a semantic gap that limits autonomous execution. Microplate-based automatic experiments are particularly challenging due to the need to simultaneously control well mapping, sample-reagent combinations, replicate placement, and parallel dispensing. This study proposes an agent-based protocol translation framework that converts natural-language microplate-based protocols into executable control commands for a robotic laboratory platform. A Parser Agent formalizes the natural-language protocol into a structured representation, and a rule-based mapping engine deterministically incorporates the operational constraints of the robotic laboratory platform to generate device-level control commands. A heterogeneous LLM Validation Agent verifies completeness, parameter accuracy, and execution order, and triggers a self-correction loop with structured feedback when errors are detected. A sweep involving 7 Parsers and 3 Validators on randomly selected ELISA protocols evaluates how model scale and Validator type affect translation accuracy and pass rates under cross-model verification. The accuracy-latency trade-off is further verified by comparing the rule-based mapping of the proposed framework with LLM end-to-end direct mapping. Finally, Bradford assay-based protein quantification using a microplate was demonstrated on a robotic laboratory platform, validating end-to-end autonomous execution from natural-language protocols to real-world experiments. The proposed framework provides a flexible approach to narrowing the semantic gap between natural-language protocols and microplate-based self-driving laboratories.

Hyeonna Choi, Jung Yup Kim, Hyuneui Lim +1

Department of Bionic Machinery, Research Institute of AI Robot, Korea Institute of Machinery & Materials, 156, Gajeongbuk-ro, Yuseong-gu, Daejeon 34103, South Korea · 156, Gajeongbuk-ro, Yuseong-gu, Daejeon 34103, South Korea · Department of Nano-devices & displays, Nano-convergence Manufacturing Research Division, Korea Institute of Machinery & Materials, 156, Gajeongbuk-ro, Yuseong-gu, Daejeon 34103, South Korea

LLM Agent Orchestration Laboratory Automation Scientific Workflow Automation Autonomous Experimentation LLM Agent Reliability

Ask PDF

Sep 25, 2026

SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents

Embodied agents offer a promising route to automating scientific experimentation, yet their progress is constrained by the lack of reliable and systematic evaluation environments. Existing simulation-based laboratory benchmarks rely heavily on manual task engineering, making it challenging to systematically compile diverse scientific protocols into executable and verifiable embodied tasks at scale. To address this challenge, we introduce SciHorizon-eLab, an agentic protocol-to-task compiler that formulates scientific embodied task construction as a compilation problem. Given a natural-language protocol of scientific experiments, SciHorizon-eLab progressively compiles laboratory protocols into semantic-preserving embodied tasks through semantic grounding, executable task synthesis, and multi-stage simulation-based certification. The system generates semantically grounded environments, executable manipulation programs, and step-level success specifications, while enabling reproducible generation of expert demonstrations and execution traces. Using this pipeline, we further construct \BenchName, a ready-to-use benchmark comprising 300 certified tasks across diverse laboratory operations. It supports HIL task execution, reproducible expert-demonstration generation, and ordered step-level evaluation. Across representative tasks, the strongest policy attains an average success rate of only 49.7%, with further evaluations revealing pronounced weaknesses in human and embodied agent coordination. We publicly release the code, benchmark data, and evaluation toolkit at https://github.com/SciHorizon-elab/SciHorizon-elab.

Maokai Qin, Chuan Qin, Qi Zhang +5

Data Intelligence for Scientific Innovation Lab, Computer Network Information Center, Chinese Academy of Sciences · Beihang University

Embodied AI AI Agent Benchmarks Synthetic Benchmark Generation Scientific Workflow Automation Laboratory Automation

Ask PDF

Abstract

Laboratory Automation

Explore similar work

From Prompts to Protocols: An AI Agent for Laboratory Automation

Dual-Agent Framework for Cross-Model Verified Translation of Natural-Language Protocols into Robotic Laboratory Platform

SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents