Embodied agents offer a promising route to automating scientific experimentation, yet their progress is constrained by the lack of reliable and systematic evaluation environments. Existing simulation-based laboratory benchmarks rely heavily on manual task engineering, making it challenging to systematically compile diverse scientific protocols into executable and verifiable embodied tasks at scale. To address this challenge, we introduce SciHorizon-eLab, an agentic protocol-to-task compiler that formulates scientific embodied task construction as a compilation problem. Given a natural-language protocol of scientific experiments, SciHorizon-eLab progressively compiles laboratory protocols into semantic-preserving embodied tasks through semantic grounding, executable task synthesis, and multi-stage simulation-based certification. The system generates semantically grounded environments, executable manipulation programs, and step-level success specifications, while enabling reproducible generation of expert demonstrations and execution traces. Using this pipeline, we further construct \BenchName, a ready-to-use benchmark comprising 300 certified tasks across diverse laboratory operations. It supports HIL task execution, reproducible expert-demonstration generation, and ordered step-level evaluation. Across representative tasks, the strongest policy attains an average success rate of only 49.7%, with further evaluations revealing pronounced weaknesses in human and embodied agent coordination. We publicly release the code, benchmark data, and evaluation toolkit at https://github.com/SciHorizon-elab/SciHorizon-elab.
Figures & tables
Task
Steps
π0.5
ACT
DP
SR
SSR
SR
SSR
SR
SSR
Apparatus & Workspace Interaction
Place Flask
2
86.7
88.3
83.3
85.0
56.7
60.0
Open Drawer
2
96.7
96.7
93.3
93.3
66.7
71.7
Insert Glass Rod
2
23.3
36.7
20.0
30.0
6.7
11.7
Liquid Handling & Transfer
Table 2: Same-task policy evaluation on ten tasks grouped by operation family. Task names are shortened for space.
Task
Policy
SR ↑
PVR ↓
Latency (s) ↓
Lamp-Triggered Heating
ACT
10.0
0.0
8.1
DP
3.3
0.0
6.8
Post-Addition Transport
ACT
16.7
0.0
8.3
DP
13.3
0.0
7.9
Table 3: Human-agent coordination under randomized and hidden intervention timing. Latency is measured in seconds. Each model–task pair is evaluated over 30 episodes.
Operation
Seen
Held-out
Policy
Seen SR
Held-out SR
Δbind
Lift
Beaker
Flask
π0.5
83.3
80.0
3.3
ACT
80.0
56.7
23.3
DP
63.3
36.7
26.7
Transport
Cylinder
Petri Dish
π0.5
93.3
73.3
20.0
ACT
86.7
33.3
53.3
DP
83.3
10.0
73.3
Table 4: Held-out object-binding generalization. Each policy is trained only on the seen object and evaluated over 30 randomized episodes for each binding. Results are SR (%); lower Δbind is better.
Certification pathway
Count
Cumulative (%)
First-pass certification
58/100
58
Automatic recovery via simulator rerun
15/100
73
Automatic visual-state correction
8/100
81
Human correction
19/100
100
Table 5: Controlled compilation and certification audit on 100 task candidates. Recovery categories are mutually exclusive. The cumulative rate shows the proportion certified after each recovery stage.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Primary Operation Family
Scenarios
Proportion (%)
Liquid Handling and Transfer
15
29.4
Mixing and Agitation
6
11.8
Solid Handling and Weighing
7
13.7
Thermal Control and Incubation
9
17.6
Apparatus and Workspace Interaction
14
27.5
Total
51
100.0
Appendix
Table S1 : Family-wise distribution of the 51 protocol-grounded source scenarios in SciVLABench.
Scenario ID
Source Scenario
Primary Family
S01
Large-Cylinder-to-Beaker Liquid Transfer
LHT
S02
Medium-Volume Quantitative Liquid Addition
LHT
S03
Pipette-to-Tube Dispensing
LHT
S04
Sequential Dispensing Across Multiple Tubes
LHT
S05
Liquid Recovery from an Erlenmeyer Flask to a Graduated Cylinder
LHT
S06
Funnel-Assisted Container Transfer
LHT
Appendix
Table S2 : Primary-family assignments of the 51 protocol-grounded source scenarios in SciHorizon-eLab .
ID
Skill
Functional Description
1
step_trajectory
Executes a general robot trajectory represented as an ordered sequence of target waypoints.
2
moveto_entity
Moves the robot end effector to a task-defined position above a specified grounded entity.
3
moveto
Moves the robot end effector to a specified target position.
4
pick
Grasps and lifts a specified manipulable object.
5
place
Precisely places a manipulated object inside a target container or receptacle.
6
drop
Releases a manipulated object onto the workspace surface.
Appendix
Table S3 : Registered atomic manipulation skills used by SciHorizon-Lab. Skill names are reported using the exact callable identifiers in the simulator execution API.