Human-Aware Robot Behaviour in Self-Driving Labs

Satheeshkumar Veeramani; Anna Kisil; Abigail Bentley; Hatem Fakhruldeen; Gabriella Pizzuto; Andrew I. Cooper

cs.ROMar 9, 2026

Human-Aware Robot Behaviour in Self-Driving Labs

Authors: Satheeshkumar Veeramani, Anna Kisil, Abigail Bentley, Hatem Fakhruldeen, Gabriella Pizzuto, Andrew I. Cooper

Organizations: Department of Chemistry University of Liverpool United Kingdom · School of Computer Science and Informatics University of Liverpool United Kingdom · School of Computer Science and Informatics Department of Chemistry University of Liverpool United Kingdom

Abstract

Self-driving laboratories (SDLs) are rapidly transforming research in chemistry and materials science to accelerate new discoveries. Mobile robot chemists (MRCs) play a pivotal role by autonomously navigating the lab to transport samples, effectively connecting synthesis, analysis, and characterisation equipment. The instruments within an SDL are typically designed or retrofitted to be accessed by both human and robotic chemists, ensuring operational flexibility and integration between manual and automated workflows. In many scenarios, human and robotic chemists may need to use the same equipment simultaneously. Currently, MRCs rely on simple LiDAR-based obstruction detection, which forces the robot to passively wait if a human is present. This lack of situational awareness leads to unnecessary delays and inefficient coordination in time-critical automated workflows in human-robot shared labs. To address this, we present an initial study of an embodied, AI-driven perception method that facilitates proactive human-robot interaction in shared-access scenarios. Our method features a hierarchical human intention prediction model that allows the robot to distinguish between preparatory actions (waiting) and transient interactions (accessing the instrument). Our results demonstrate that the proposed approach enhances efficiency by enabling proactive human-robot interaction, streamlining coordination, and potentially increasing the efficiency of autonomous scientific labs.

Figures & tables

Figure 1. Shared workspace interaction between a human chemist and a collaborative mobile robot near a standard laboratory fumehood.

Figure 2. Proposed method to predict the intention of human chemists in a shared workspace within an SDL. $L$ denotes the set of detected labels. $P_{i}$ and $P_{j}$ represent the 3D position vectors of objects $i$ and $j$ , respectively. $D_{ij}$ is the Euclidean distance between objects $i$ and $j$ .

Scenario	Modality	Model	Test Accuracy (%)
S1	Vision	LLaVA-1.5-7b	29±3
	Vision	LLaVA-1.5-7b*	88±4
	Vision+Depth	LLaVA-1.5-7b*	70±8
S2	Vision	LLaVA-1.5-7b	20±2
	Vision	LLaVA-1.5-7b*	94±2
	Vision+Depth	LLaVA-1.5-7b*	94±2

Table 1. Model Performance

Explore similar work

AgentChemist: A Multi-Agent Experimental Robotic Platform Integrating Chemical Perception and Precise Control
Mar 25, 2026Xiangyi Wei, Fei Wang, Haotian Zhang +6Laboratory Automation Autonomous Experimentation
School of Computer Science and Technology, East China Normal University, Shanghai, China. · School of Chemistry and Molecular Engineering, East China Normal University, Shanghai, China. · School of Data Science and Engineering, East China Normal University, Shanghai, China. +1
LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory
Jul 26, 2026Haobo Wang, Baoli Sun, Anqi Zou +8Robot Failure Detection Robotics
School of Software, Dalian University of Technology, Dalian, China · Shenzhen Loop Area Institute-Hong Kong Science and Technology Innovation Cooperation Zone, Shenzhen, China · School of Computer Science and Engineering, Harbin Institute of Technology, Harbin, China +3
Long-Term Memory for VLA-based Agents in Open-World Task Execution
Apr 17, 2026Xu Huang, Weixin Mao, Yinhao Li +2Agent Memory Long-Horizon Robotic Manipulation
Nanjing University · LimX Dynamics

Mar 25, 2026J/K move · Enter open · S save

AgentChemist: A Multi-Agent Experimental Robotic Platform Integrating Chemical Perception and Precise Control

Xiangyi Wei, Fei Wang, Haotian Zhang, Xin An+5

School of Computer Science and Technology, East China Normal University, Shanghai, China. · School of Chemistry and Molecular Engineering, East China Normal University, Shanghai, China. · School of Data Science and Engineering, East China Normal University, Shanghai, China.+1

Laboratory Automation Autonomous Experimentation Robotic Control Multi-Agent Orchestration

Open paper AskPDF arXiv

Chemical laboratory automation has long been constrained by rigid workflows and poor adaptability to the long-tail distribution of experimental tasks. While most automated platforms perform well on a narrow set of standardized procedures, real laboratories involve diverse, infrequent, and evolving operations that fall outside predefined protocols. This mismatch prevents existing systems from generalizing to novel reaction conditions, uncommon instrument configurations, and unexpected procedural variations. We present a multi-agent robotic platform designed to address this long-tail challenge through collaborative task decomposition, dynamic scheduling, and adaptive control. The system integrates chemical perception for real-time reaction monitoring with feedback-driven execution, enabling it to adjust actions based on evolving experimental states rather than fixed scripts. Validation via acid-base titration demonstrates autonomous progress tracking, adaptive dispensing control, and reliable end-to-end experiment execution. By improving generalization across diverse laboratory scenarios, this platform provides a practical pathway toward intelligent, flexible, and scalable laboratory automation.

Abstract

Explore similar work

AgentChemist: A Multi-Agent Experimental Robotic Platform Integrating Chemical Perception and Precise Control

LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory

Long-Term Memory for VLA-based Agents in Open-World Task Execution