Compact Task-Aligned Imitation Learning for Laboratory Automation
Authors: Kanata Suzuki, Hanon Nakamura, Kana Miyamoto, Tetsuya Ogata
Organizations: Spatial Robotics Research Center, Fujitsu Limited., Kanagawa 211-8588, Japan. · Faculty of Science and Engineering, Waseda University, Tokyo 169-8050, Japan. · National Institute of Advanced Industrial Science and Technology, Tokyo 100-8921, Japan.
Robotic laboratory automation has traditionally relied on carefully engineered motion pipelines and task-specific hardware interfaces, resulting in high design cost and limited flexibility. While recent imitation learning techniques can generate general robot behaviors, their large model sizes often require high-performance computational resources, limiting applicability in practical laboratory environments. In this study, we propose a compact imitation learning framework for laboratory automation using small foundation models. The proposed method, TVF-DiT, aligns a self-supervised vision foundation model with a vision-language model through a compact adapter, and integrates them with a Diffusion Transformer-based action expert. The entire model consists of fewer than 500M parameters, enabling inference on low-VRAM GPUs. Experiments on three real-world laboratory tasks - test tube cleaning, test tube arrangement, and powder transfer - demonstrate an average success rate of 86.6%, significantly outperforming alternative lightweight baselines. Furthermore, detailed task prompts improve vision-language alignment and task performance. These results indicate that small foundation models, when properly aligned and integrated with diffusion-based policy learning, can effectively support practical laboratory automation with limited computational resources.
Figures & tables
Fig. 1: Overview of this study for a laboratory automation framework with a compact imitation learning model. We focus on robotic test tube manipulation under limited computational resources.
Fig. 2: Architecture of the proposed TVF-DiT framework. DINOv3 and SigLIP2 extract geometric and language-aligned representations, which are fused via a lightweight Adapter. The resulting task-conditioned tokens are used as cross-attention keys and values in a Diffusion Transformer that predicts action chunks through conditional flow matching.
TABLE I: Structure of Proposed Method
Model Type/Layers
Parameter
VL Encoder
DINOv3 [ 32 ]
vit_small, patch16
21 M
SigLIP2 [ 33 ]
vit_base, patch16
375 M
Adapter
Projection × 2
33 M
GatedRMS [ 34 ] × 2
Trans. Dec. Block × 8
Table 3
Fig. 3: Task 1: Test tube cleaning requiring precise insertion and continuous scrubbing motion. Task 2: Test tube arrangement requiring collision avoidance and bimanual coordination. Task 3: Powder transfer requiring sequential scooping and pouring manipulation. These tasks evaluate geometric precision, object selection, and continuous action generation.
TABLE I: Success rates in test tube manipulations
Task 1
Task 2
Task 3
Ave.
Playback
0/10
0/10
0/10
0.0%
(i) w/ VLM
1/10
2/10
3/10
20.0%
(SmolVLM2)
(ii) w/ VF & LLM
2/10
4/10
5/10
36.6%
(DINOv3 & SmolLM2)
Table 5
Fig. 4: Representative execution sequences generated by the proposed method. Successful trials demonstrate continuous and coordinated manipulation across all tasks. Failure cases (highlighted) mainly occur during fine alignment or placement, indicating sensitivity to small geometric errors.
Fig. 5: Joint trajectories and 3D end-effector (EE) paths during inference. Consistent EE paths across episodes demonstrate robustness of the learned policy.
Fig. 6: Recovery behavior under external disturbance during test tube cleaning. When the tube orientation is manually perturbed, the model re-generates actions based on updated visual observations and resumes the task. This demonstrates the adaptive capability of the diffusion-based policy.
TABLE II: Success rates in trained prompts with TVF-DiT
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China · School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China
Department of Computer Science and Engineering, POSTECH, Pohang, South Korea · Graduate School of Artificial Intelligence, POSTECH, Pohang, South Korea