cs.ROAug 18, 2026

Constructing Predictive Surgical Path for AI-based Capsulorhexis Skill Transfer

Authors: Mohammad Javad Ahmadi, Hamid D. Taghirad

Organizations: Applied Robotics and AI Solutions (ARAS), Faculties of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Automated training of surgeons is one of the most crucial factors that significantly minimize surgical training risks and expenses. With recent advances in artificial intelligence (AI) knowledge and available data from various surgeries, AI's involvement in surgical training is becoming very promising. It is recommended that at the early stages of AI development, it interferes in the surgery as a third agent alongside the trainer. As trust in AI increases, this process will lead to an AI agent acting as a trainer in the future. The first phase in which AI can intervene in the training process is to suggest an improved surgical path to the trainer. A platform must be constructed in the first step, to accomplish this task and to enhance the movement path of trainee surgeons. This paper introduces this platform along with an annotated capsulorhexis surgery dataset called the ARAS-Farabi dataset. In this research, a deep convolutional neural network is pre-trained with JIGSAWS and ARAS-Farabi surgical datasets that can extract surgical skill characteristics from surgery tool tip motion data. The proposed platform develops a reference model from the feature space of an expert surgeon's movement trajectory and proposes an improved path to enhance the skill of a novice surgeon. An optimization with two loss functions is utilized to create a path that raises the skill level of the novice surgeon's path while simultaneously predicting and preserving his/her intent. The results of this study reveal that, with the assistance of an AI agent, the trainee surgeon's movement path can be enhanced by at least 20 percent while maintaining his intentional objective. In addition to the recommended deep network, various tangible indicators have also been developed in this research to verify the level of trainee improvement.

Figures & tables

Figure 1: The proposed structure for skill transfer.
Figure 2: Three general surgical tasks in JIGSAWS dataset.
Figure 3: Output information of Aras-Farabi software.( Video ).
Figure 4: Removing light reflections and defining initial bounding boxes ( Video ).
Figure 5: Comparison of information extracted from ARAS-Farabi software with sensory data.
Figure 6: The architecture of the proposed network.
Figure 7: Comparison of the base path (novice) and the predicted-constructed path (expert).
Indicators and Ratios of Coefficientsba=1.5\frac{\boldsymbol{b}}{\boldsymbol{a}}=\mathbf{1.5}ba=5\frac{\boldsymbol{b}}{\boldsymbol{a}}=\mathbf{5}
Total Skill Level Improvement6.976.9720.3320.33
Total Predictability Improvement14.3414.3418.8718.87
Enhancement in Tremor Reduction0.230.233.443.44
Enhancement in Noise Cancellation0.230.232.402.40
Table I: Skill enhancement of the novice surgeon using the predicted-constructed path.
Figure 8: Idealized performance of the proposed structure on the velocity dimension.
Figure 9: Comparison of the base path (novice) and the predicted-constructed path (expert).
Figure 10: Comparison of the base path (novice) and the predicted-constructed path (expert).
Figure 11: Comparison of the base path (novice) and the predicted-constructed path (expert) for four different surgical procedures performed by an expert surgeon.
Indicators and expert pathsID1 (4.35/5)ID17 (5/5)
Total Skill Level Improvement13.2013.2020.3320.33
Total Predictability Improvement16.3416.3418.8718.87
Enhancement in Tremor Reduction2.332.333.443.44
Enhancement in Noise Cancelation1.321.322.402.40
Table II: Skill enhancement of the novice surgeon using two different expert paths.
Figure 12: Comparison of the base path (novice) and the predicted-constructed path (expert).
Indicators and Ratios of Coefficientsba=1.5\frac{\boldsymbol{b}}{\boldsymbol{a}}=\mathbf{1.5}ba=5\frac{\boldsymbol{b}}{\boldsymbol{a}}=\mathbf{5}
Total Skill Level Improvement5.675.6713.0113.01
Total Predictability Improvement8.958.955.965.96
Enhancement in Tremor Reduction5.915.9118.4418.44
Enhancement in Noise Cancelation0.390.393.403.40
Table III: Skill enhancement of the novice surgeon using the predicted-constructed path.
More in this topic

Trajectory Optimization

Trajectory optimization computes motion or control sequences that minimize costs while satisfying system dynamics, safety limits, and task constraints.

See all 207 papers

Explore similar work

CardsList
Aug 18, 2026

Explainable AI-Powered Framework for Video-Based Skill Assessment in Cataract Surgery

Persistent shortages in the surgical workforce and inherent limitations of traditional training methods highlight the necessity of automated, data-driven approaches in surgical education. This study addresses these challenges by introducing a novel, explainable AI-powered framework for automated skill assessment, specifically focusing on cataract surgery. We present the world's largest dataset of cataract surgery videos, comprising 2,000 recordings. Additionally, we propose an AI-powered analytical framework that employs advanced computer vision and signal-processing techniques to automatically evaluate surgical videos to derive objective, quantitative performance indicators that complement or potentially replace subjective scoring methods. A significant advantage of our framework over previous methods lies precisely in its explainability of outputs, elevating it beyond merely an opaque skill classification tool. Through experimental analysis of 83 cataract surgery videos, we demonstrate that the automatically computed metrics exhibit strong correlations with expert-based subjective evaluations, achieving up to 87% accuracy in surgical skill assessment. Each metric was individually examined, and expert surgeons provided subjective ratings using the newly introduced Capsulorhexis Skill Assessment System (CSAS). These subjective assessments were compared with ten objective motion-based metrics extracted through our framework. The results indicated a robust correlation between subjective ratings and automated indicators, underscoring the framework's capacity to accurately model surgical expertise.

Mohammad Javad Ahmadi, Hamid D. Taghirad

Applied Robotics and AI Solutions (ARAS), Faculties of Electrical and Computer Engineering, K.N. Toosi University of Technology, Tehran, Iran.

Automated EvaluationSurgical Video UnderstandingExplainable Artificial Intelligence
AskPDF
Oct 18, 2026

SurGPT: A Multi-Task Foundation Model for Robotic Surgical Skills Assessment and Gesture Recognition

The assessment of surgical skills in robotic surgery training has traditionally relied on subjective evaluations by human experts, creating bottlenecks in surgical education and limiting scalability. We present a novel foundation model that automatically analyzes robotic surgical videos to provide comprehensive skill assessment and gesture understanding. Combining a visual encoder with a transformer architecture, the model performs three key tasks simultaneously: 1) surgical task classification (knot tying, needle passing, suturing), 2) skill level assessment on a standardized scale, and 3) natural language generation of surgical gesture descriptions. The model processes video sequences from the da Vinci surgical robot and generates human-readable text descriptions of surgical actions, similar to how language models generate text from prompts, using a multi-task learning framework with adaptive uncertainty weighting to optimize performance across all objectives. Our transformer-based architecture incorporates positional encoding for temporal understanding and attention mechanisms for spatiotemporal feature integration. Evaluated on the da Vinci Skills Simulator dataset, the proposed framework demonstrates state-of-the-art performance in surgical task recognition and provides interpretable skill assessments that correlate with expert evaluations, with the model's ability to generate natural language descriptions of surgical gestures opening new possibilities for automated surgical education, real-time feedback systems, and objective skill assessment in surgical training and certification programs.

Parham M. Kebria, Saeid Nahavandi, Mehrdad Saif +1

Electrical and Computer Engineering Department, North Carolina A&T State University, Greensboro, NC, USA · Swinburne University of Technology, Hawthorn, VIC, Australia · Harvard Paulson School of Engineering and Applied Sciences, Harvard University, Allston, MA, USA +3

Surgical Video UnderstandingSurgical RoboticsRobot Foundation ModelsGesture Recognition
AskPDF
Mar 27, 2026

SHANDS: A Multi-View Dataset and Benchmark for Surgical Hand-Gesture and Error Recognition Toward Medical Training

In surgical training for medical students, proficiency development relies on expert-led skill assessment, which is costly, time-limited, difficult to scale, and its expertise remains confined to institutions with available specialists. Automated AI-based assessment offers a viable alternative, but progress is constrained by the lack of datasets containing realistic trainee errors and the multi-view variability needed to train robust computer vision approaches. To address this gap, we present Surgical-Hands (SHands), a large-scale multi-view video dataset for surgical hand-gesture and error recognition for medical training. SHands captures linear incision and suturing using five RGB cameras from complementary viewpoints, performed by 52 participants (20 experts and 32 trainees), each completing three standardized trials per procedure. The videos are annotated at the frame level with 15 gesture primitives and include a validated taxonomy of 8 trainee error types, enabling both gesture recognition and error detection. We further define standardized evaluation protocols for single-view, multi-view, and cross-view generalization, and benchmark state-of-the-art deep learning models on the dataset. SHands is publicly released to support the development of robust and scalable AI systems for surgical training grounded in clinically curated domain knowledge.

Le Ma, Thiago Freitas dos Santos, Nadia Magnenat-Thalmann +1

1MIRALab · 2Quality of Life Technologies Lab (QoL Lab), University of Geneva

Surgical Video UnderstandingGesture RecognitionAnomaly DetectionMulti-View Learning
AskPDF