Automated Assessment of Kidney Ureteroscopy Exploration for Training

Fangjie Li; Nicholas Kavoussi; Charan Mohan; Matthieu Chabanas; Jie Ying Wu

eess.IVFeb 17, 2026

Automated Assessment of Kidney Ureteroscopy Exploration for Training

Authors: Fangjie Li, Nicholas Kavoussi, Charan Mohan, Matthieu Chabanas, Jie Ying Wu

Organizations: Department of Computer Science, Vanderbilt University, 2301 Vanderbilt Place, Nashville, 37235, TN, USA. · Department of Urology, Vanderbilt University Medical Center, 1211 Medical Center Drive, Nashville, 37232, TN, USA.

Abstract

Purpose: Kidney ureteroscopic navigation is challenging with a steep learning curve. However, current clinical training has major deficiencies, as it requires one-on-one feedback from experts and occurs in the operating room (OR). Therefore, there is a need for a phantom training system with automated feedback to greatly \revision{expand} training opportunities. Methods: We propose a novel, purely ureteroscope video-based scope localization framework that automatically identifies calyces missed by the trainee in a phantom kidney exploration. We use a slow, thorough, prior exploration video of the kidney to generate a reference reconstruction. Then, this reference reconstruction can be used to localize any exploration video of the same phantom. Results: In 15 exploration videos, a total of 69 out of 74 calyces were correctly classified. We achieve < 4mm camera pose localization error. Given the reference reconstruction, the system takes 10 minutes to generate the results for a typical exploration (1-2 minute long). Conclusion: We demonstrate a novel camera localization framework that can provide accurate and automatic feedback for kidney phantom explorations. We show its ability as a valid tool that enables out-of-OR training without requiring supervision from an expert.

Figures & tables

Figure 1: The overall workflow of the framework. In stage one, a reference reconstruction is generated and registered to the CT segmentation. This reconstruction is then reused for all query exploration videos, which are localized in stage two. For each video, calyces are marked as visited/not visited.

Figure 2: The accuracy of the SfM reconstruction point cloud compared to the CT volume. The violin plot is plotted with the 99 percentile data, as the outliers, though numerically extreme, has almost no impact on the CT registration and the subsequent pose localization.

Phantom ID	Mean Euclidean Distance (mm)	99% Hausdorff Distance (mm)	Coverage (%)	Projection Error (px)
1	1.0 $\pm$ 1.8	6.1	58.19	1.19
2	1.7 $\pm$ 1.8	4.8	43.17	1.14
3	1.1 $\pm$ 1.9	6.2	63.76	1.32
4	1.4 $\pm$ 1.6	5.3	66.30	1.26

Table 1: Reference Model Point Cloud CT Registration Error

Phantom ID	$Video_{\text{ref}\_em}$ Position Error (mm)	$Video_{\text{query}\_em}$ Position Error (mm)
1	2.7 $\pm$ 2.0	2.6 $\pm$ 1.6
2	2.9 $\pm$ 1.7	3.5 $\pm$ 1.9
3	3.3 $\pm$ 1.7	3.8 $\pm$ 1.7
4	3.0 $\pm$ 1.7	3.2 $\pm$ 2.00

Table 2: The Camera Pose Localization Accuracy of Expert Recordings

Figure 3: A random selection of rendered vs real ureteroscope image Pairs. (Note: the kidney stones are not in the CT view)

Figure 4: Five example cases where the framework accurately identifies visited/missed calyces

Figure 5: Example cases where the framework misidentified one calyx in each case. A: Phantom 2. B: Phantom 3. C: Phantom 4.

Surgical Video Understanding

Surgical video understanding uses computer vision and multimodal AI to interpret procedures, instruments, actions, and events over time.

See all 118 papers

RAUL: Reference-Assisted Ureteroscopy Localization for Skill Assessment
Sep 16, 2026Fangjie Li, Mai Bui, Charan Mohan +43D Reconstruction Surgical Video Understanding
Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37240 USA · Department of Computer Science, Vanderbilt University, Nashville, TN 37240 USA · Department of Urology, Vanderbilt University Medical Center, Nashville, TN 37232 USA
Towards Autonomous Robotic Kidney Ultrasound: Spatial-Efficient Volumetric Imaging via Template Guided Optimal Pivoting
Feb 18, 2026Xihan Ma, Haichong ZhangRobotic Ultrasound Ultrasound Imaging
Seeing Globally, Refining Locally: Global Visual Guidance and Local Ultrasound Cues for Robust Freehand 3-D Ultrasound Reconstruction
Jul 14, 2026Yameng Zhang, Zhongyu Chen, Dianye Huang +3Object Pose Estimation Ultrasound Imaging
Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR, China · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China

Sep 16, 2026J/K move · Enter open · S save

RAUL: Reference-Assisted Ureteroscopy Localization for Skill Assessment

Fangjie Li, Mai Bui, Charan Mohan, Michael Miga+3

Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37240 USA · Department of Computer Science, Vanderbilt University, Nashville, TN 37240 USA · Department of Urology, Vanderbilt University Medical Center, Nashville, TN 37232 USA

3D Reconstruction Surgical Video Understanding Trajectory Inference Surgical Instrument Tracking

Open paper AskPDF arXiv

Objective: Incomplete navigation of anatomy during ureteroscopic kidney stone surgeries can contribute to repeat interventions. While skilled surgeons have lower reintervention rates, there are no objective metrics to quantify scope-navigation performance to evaluate when a trainee becomes skilled. This work aims to recover ureteroscope trajectories from endoscopic video and derive navigation metrics to quantify differences in skill. Methods: We propose RAUL, a reference-assisted reconstruction framework for recovering ureteroscope trajectories from ureteroscope videos only in phantoms. For each phantom, we use a slow, high-quality reference exploration video to generate a reference reconstruction. We localize subsequent exploration videos against this reference. We evaluate localization accuracy against electromagnetically tracked scope pose. We compute navigation metrics from phantom exploration trajectories to compare surgical residents across experience levels. Results: The proposed reference-assisted framework achieves a mean translation root mean square error of

0.5 \pm 0.1

mm across 9 phantoms. Compared to standard Structure-from-Motion (SfM), the proposed pipeline increases frame-wise localization coverage from

50.5 \pm 14.9\%

86.1 \pm 7.2\%

of all video frames. The reconstructed trajectories revealed significant differences between high- and low-experience trainees in established navigation metrics. Conclusion: RAUL enables substantially more complete recovery of ureteroscope trajectories from videos compared to standard SfM pipelines, enabling trajectory-based skill assessment without additional tracking equipment. Significance: To the best of our knowledge, this is the first use of video-only recovery of ureteroscope trajectories without external tracking sensors for skill assessment, supporting scalable automated assessment of ureteroscopy navigation skill.

Abstract

Surgical Video Understanding

Explore similar work

RAUL: Reference-Assisted Ureteroscopy Localization for Skill Assessment

Towards Autonomous Robotic Kidney Ultrasound: Spatial-Efficient Volumetric Imaging via Template Guided Optimal Pivoting

Seeing Globally, Refining Locally: Global Visual Guidance and Local Ultrasound Cues for Robust Freehand 3-D Ultrasound Reconstruction