MicroHookACT: Monocular Microscopic Vision Guided Visuomotor Policy for Flexible Microelectrode Hooking
Organizations: Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China. · State Key Laboratory of Brain Cognition and Brain-Inspired Intelligence Technology, Shanghai 200031, China. · Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China.
Abstract
Automated needle-loop hooking is a critical step in flexible microelectrode (FME) implantation. This paper presents MicroHookACT, an imitation learning-based visuomotor policy for automated 3D hooking under monocular microscopic vision. First, a unidirectional hooking strategy exploits defocus cues and optical-axis guidance to enable palpation-free precise alignment and contact-rich threading. Second, an action-supervised object attention module built on a frozen ViT backbone learns to focus on the micro-needle tip and micro-loop directly from human demonstrations, without requiring manual visual annotations for training. Third, attention-centered global coarse and local fine features are dynamically weighted according to predicted action progress, enabling a single ACT policy to adapt to changing defocus blur and visual requirements throughout the operation. In the experiments, visuomotor policies were trained on 60 human demonstrations and evaluated under five setups with varying difficulties. Our MicroHookACT framework achieved the highest overall success rate of 96.7% with an average execution time of 11.5 s. These results demonstrate the potential of visuomotor policy learning for micron-level control under varying operating conditions.
Figures & tables
| Method | Standard | Hard Position | New Background | New Angle | New FME | Overall | Average Time(s) |
|---|---|---|---|---|---|---|---|
| MicroACT-3D [ 16 ] | 8/10 | 1/5 | 0/5 | 2/5 | 4/5 | 15/30 | 16.1 |
| HLF [ 14 ] | 9/10 | 2/5 | 5/5 | 4/5 | 5/5 | 25/30 | 16.4 |
| DINOv3ACT | 5/10 | 0/5 | 3/5 | 4/5 | 3/5 | 15/30 | 11.7 |
| Ours w/o Adapt. | 8/10 | 1/5 | 5/5 | 3/5 | 3/5 | 20/30 | 12.8 |
| Ours | 10/10 | 5/5 | 5/5 | 5/5 | 4/5 | 29/30 | 11.5 |
| Backbone | Attention Recall | Attention Focality |
|---|---|---|
| Resnet-18 (learnable) | 0.99 | 0.21 |
| Resnet-18 (frozen) | 0.95 | 0.21 |
| DINOv3-ViT (learnable) | 0.99 | 0.02 |
| DINOv3-ViT (frozen) | 0.97 | 0.35 |