SurgiCLAW: An Interpretable Atomic-Skill Agent for Precision Robotic Micromanipulation
Organizations: Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China
Abstract
Recent multimodal large language models (MLLMs) and vision--language--action models have provided robots with new capabilities for long-horizon task understanding and planning. In surgical robotics, however, end-to-end prediction of low-level actions remains limited by insufficient precision, opaque decisions, and scarce data. This paper presents SurgiCLAW, a hierarchical skill-based framework for precision surgical manipulation. SurgiCLAW confines an MLLM to intraoperative visual understanding, structured explanation, and atomic surgical-skill selection, while delegating physical execution to verifiable deterministic control programs. For brain--machine-interface implantation, a representative high-precision surgical task, we construct a state-to-skill invocation dataset and expand it through visual and state perturbations. The model first learns structured skill invocation through supervised fine-tuning and is then optimized using three rewards for action correctness, operation prerequisites, and format validity, improving action selection, operational safety, and output parseability. We evaluate subtask success, offline model comparison, real-robot performance, and ablations. On a fixed test set, SurgiCLAW achieves 94.6% skill-selection accuracy and 93.0% macro-averaged F1 (Macro-F1). Across 50 closed-loop real-robot implantation trials, it achieves an 88.0% task success rate.
Figures & tables
| Function | Meaning | Primary prerequisite |
| ADJUST_EXPOSURE | Adjust microscopic exposure | Camera online; exposure outside the calibrated range |
| ADJUST_FOCUS | Adjust microscopic focus | Camera online; image out of focus |
| E_STOP | Trigger a safety stop | Singular configuration, system fault, or other high-risk anomaly |
| SPATIAL_REGISTRATION | Register image and robot coordinates | Usable image; system self-check passed |
| RESTART_CAMERA | Restart a failed or disconnected camera | Abnormal camera connection; robot held in a safe state |
| ALIGN_NEEDLE | Deterministic visual-servo needle alignment | Registration complete; image sharp; no singular configuration |
| Method | Accuracy (%) | Macro-P (%) | Macro-R (%) | Macro-F1 (%) | Safety violation (%) |
| General VLMs (non-SFT) | |||||
| MiniCPM-o-2.6 [ 23 ] | 14.2 | 2.0 | 14.3 | 3.6 | 18.6 |
| DeepSeek-VL2 [ 24 ] | 20.6 | 22.4 | 20.9 | 14.8 | 14.2 |
| Llama-4-Maverick-17B-128E-Instruct [ 25 ] | 24.7 | 28.9 | 25.1 | 17.2 | 11.6 |
| InternVL3.5-241B-A28B [ 26 ] | 31.6 | 37.8 | 31.2 | 24.4 | 7.4 |
| Qwen2.5-VL-3B-Instruct [ 27 ] | 15.4 | 27.8 | 15.5 | 5.9 | 15.8 |
| Method | Success (%) | Stages | Takeover (%) | Unsafe proposal (%) |
| Surgical-specific VLMs | ||||
| SurgVLM-7B | 46.0 (23/50) | 38.0 (19/50) | 5.4 (11/204) | |
| SurgVidLM | 50.0 (25/50) | 34.0 (17/50) | 4.9 (10/205) | |
| General VLM + SFT | ||||
| Qwen3-VL-4B-Instruct + SFT | 66.0 (33/50) | 20.0 (10/50) | 3.4 (7/207) | |
| Qwen3-VL-32B-Instruct + SFT | 76.0 (38/50) | 10.0 (5/50) | 1.9 (4/206) | |
| Setting | Accuracy (%) | Macro-P (%) | Macro-R (%) | Macro-F1 (%) | Safety violation (%) |
| Full model | 94.6 | 94.0 | 92.0 | 93.0 | 0.5 |
| w/o structured rationale | 88.2 | 87.4 | 85.9 | 86.4 | 2.3 |
| w/o GDPO | 83.0 | 81.4 | 81.0 | 81.1 | 3.2 |
| w/o action-correctness reward | 90.4 | 88.1 | 88.7 | 88.2 | 1.6 |
| w/o prerequisite reward | 91.2 | 90.8 | 88.9 | 89.7 | 3.2 |
| w/o format reward | 92.0 | 92.8 | 88.6 | 90.4 | 1.1 |
| Subtask | Success criterion | Success (%) | Mean error | Time (s) |
| Spatial registration | TRE mm and coordinate transform completed | 95.0 (38/40) | mm (TRE) | |
| Target localization (internal) | Stable target detection available for alignment | 97.5 (39/40) | mm (position) | |
| Needle alignment | Orientation and image-plane position within thresholds | 95.0 (38/40) | / mm | |
| Insertion execution | Direction and 3-D landing position within thresholds | 90.0 (36/40) | / mm | |
| Image-quality adjustment | Image restored to a parseable quality range | 97.5 (39/40) | – | |
| Safety stop | Timely refusal in a high-risk state | 100.0 (40/40) | – |
Robotic Manipulation
Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.
See all 1,685 papers