Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling

Anuska Sarkar; Partha Acharya; Vishnu Nandury; Kaustav Basu; Subhamoy Mandal

eess.IVOct 13, 2026

Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling

Authors: Anuska Sarkar, Partha Acharya, Vishnu Nandury, Kaustav Basu, Subhamoy Mandal

Organizations: School of Medical Science and Technology, Indian Institute of Technology Kharagpur, Kharagpur 721302 India · Computer Science Department, Arizona State University, Tempe, Arizona 85287 USA · Narayana Hospital, Kolkata 700107 India

Abstract

We present a real-time, hybrid framework for occlusion handling and depth-aware tracking in minimally invasive surgeries (MIS). Occlusions caused by surgical instruments often obscure the region of interest (ROI), affecting surgical precision. To address this, our approach integrates enhanced You Only Look Once (YOLO) based detection, Particle Filtering (PF), and monocular depth estimation using Multiple Depth Estimation Accuracy with Single Network (MiDaS). Occlusion is detected using a distance-based threshold between instruments and ROI. Experimental results demonstrate robust instrument detection with a confidence score of 0.85, effective occlusion event handling, and accurate depth prediction with an MAE of 0.34 cm and RMSE of 0.53 cm. Independent phantom-based validation against stereo depth measurements yielded an MAE of 0.65 cm and RMSE of 0.66 cm. The framework further achieved an occlusion detection accuracy of 81.95% while maintaining real-time performance at 38.7 FPS. Preliminary surgeon feedback indicates that the proposed visualization framework is clinically relevant and holds promise for future intraoperative image-guided navigation and decision support in MIS.

Figures & tables

Fig. 1: The workflow of the proposed system: (a) Surgical video frames for pre-processing. (b) Integration of particle filtering for surgical tool and ROI detection. (c) Monocular depth estimation using the MiDaS model. (d) Real-time visualization of the surgical scene.

Fig. 2: Representative frames of different surgical actions acquired from robot-assisted cholecystectomies (a–d) with corresponding relative depth maps (e–h).

Fig. 3: Representative surgical frames illustrating real-time instrument tracking and occlusion handling at (a) t, (b) t+3s with a text-based alert, (c) t+5s with a color-coded bounding box, and (d) t+9s with depth-aware color-coded cues, where the bounding box changes from green (safe) to yellow (partial occlusion) and red (complete occlusion).

Class	Precision	Recall	mAP 50	mAP 50-95
Overall	0.793	0.867	0.894	0.713
ROI	0.505	0.319	0.356	0.102
AtraumaticGrasper_Head	0.894	0.913	0.959	0.760
AtraumaticGrasper_Body	0.858	0.900	0.949	0.757
Electrichook	0.925	0.971	0.984	0.862
CurvedAtraumaticGrasper_Head	0.808	0.898	0.908	0.718

TABLE I: Class-wise detection performance of the proposed YOLOv10 detector on the extended hSDB dataset.

Training Dataset	mAP 50 (%)	mAP 50-95 (%)
Original (17 classes)	$75.47\pm 3.16$	$57.31\pm 2.12$
Extended (17 classes + ROI)	$71.01\pm 2.62$	$53.30\pm 2.20$

TABLE II: Five-fold cross-validation performance of the YOLOv10 detector on the hSDB dataset.

Fig. 4: Depth mapping and validation during robotic surgery: (a) Reference surgical scene with paper scale, (b) Corresponding depth map with marked coordinates, (c) The linear regression plot of real and predicted depth values.

Method	Backbone	Configuration	mAP 50-95 (%)
Faster R-CNN	ResNet50	FPN	20.2
	ResNet50	FPN + OHEM	20.1
	ResNet50	FPN + Double Head	22.9
	ResNet101	FPN	22.8
	ResNeXt101-32 $\times$ 4d	FPN	24.1
	ResNeXt101-64 $\times$ 4d	FPN	24.9

TABLE III: Comparison of object detection performance with SOTA methods on the hSDB dataset.

Method	Motion Model	Appearance	Monocular	Depth-aware	Multi-object	Real-time
		Association	Depth	Occlusion Reasoning	Tracking	Suitable
SORT [ 18 ]	Kalman Filter	$\times$	$\times$	$\times$	✓	✓
DeepSORT [ 19 ]	Kalman + Deep Re-ID	✓	$\times$	$\times$	✓	Limited
ByteTrack [ 20 ]	IoU-based Association	$\times$	$\times$	$\times$	✓	✓
PF + MiDaS (Proposed)	Particle Filter	$\times$	✓	✓	✓	✓

TABLE IV: Qualitative comparison of representative tracking and occlusion handling approaches.

Configuration	tIoU	Mean Confidence	Latency (ms)	FPS
YOLOv10	0.4079	0.6680	18.3	54.6
YOLOv10 + PF	0.6134	0.8339	18.6	53.8

TABLE V: Ablation Study evaluating the contribution of Particle Filtering.

Fig. 5: Independent phantom-based validation of monocular depth estimation using the OAK-D Lite camera (N = 15): (a) Experimental setup showing the phantom and imaging configuration, (b) Acquired depth map overlaid on the corresponding RGB image, with the automatically detected ArUco marker (51.6 cm), (c) Corresponding MiDaS-predicted depth map from the captured RGB image, yielding an estimated depth of 50.96 cm (absolute error = 0.64 cm), and (d) Bland–Altman agreement analysis between MiDaS-predicted depths and OAK-D Lite reference measurements across all validation samples.

Fig. 6: Runtime distribution and throughput comparison across pipeline operating modes: (a) Individual module latency breakdown for full sequential processing ( $70.0$ ms/frame); (b) Latency and frame rate comparison across Sequential ( $14.3$ FPS), Interleaved Depth Sampling ( $38.7$ FPS), and Tracking-Only ( $56.1$ FPS) operational modes against the $30$ FPS real-time target ( $33.3$ ms) and $100$ ms clinical threshold.

Fig. 7: Violin plots of (a) Occlusion frequency and years of surgical experience, (b) Surgeons’ preference of occlusion alerts.

Explore similar work

Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality
Mar 9, 2026Jiaming Zhang, Mingxu Liu, Hongchao Shu +6Visual Object Tracking Multimodal Sensor Fusion
Johns Hopkins University · Johns Hopkins University, School of Medicine · University of Arkansas, Johns Hopkins University
Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery
Mar 3, 2026Muhammad Asad, Emanuele Colleoni, Pritesh Mehta +7Depth Estimation Monocular Depth Estimation
Medtronic Surgical, London, UK. · UCL Hawkes Institute, University College London, London,UK.
Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization
Mar 12, 2026Hanyang Hu, Zekai Liang, Florian Richter +1Visual Object Tracking Covariance Matrix Adaptation Evolution Strategy
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093 USA.

Mar 9, 2026J/K move · Enter open · S save

Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality

Jiaming Zhang, Mingxu Liu, Hongchao Shu, Ruixing Liang+5

Johns Hopkins University · Johns Hopkins University, School of Medicine · University of Arkansas, Johns Hopkins University

Visual Object Tracking Multimodal Sensor Fusion Surgical Robotics

Open paper AskPDF arXiv

Surgical navigation provides real-time guidance by estimating the pose of patient anatomy and surgical instruments to visualize relevant intraoperative information. In conventional systems, instruments are typically tracked using fiducial markers and stationary optical tracking systems (OTS). Augmented reality (AR) has further enabled intuitive visualization and motivated tracking using sensors embedded in head-mounted displays (HMDs). However, most existing approaches rely on a clear line of sight, which is difficult to maintain in dynamic operating room environments due to frequent occlusions caused by equipment, surgical tools, and personnel. This work introduces a framework for tracking surgical instruments under occlusion by fusing multiple sensing modalities within a dynamic scene graph representation. The proposed approach integrates tracking systems with different accuracy levels and motion characteristics while estimating tracking reliability in real time. Experimental results demonstrate improved robustness and enhanced consistency of AR visualization in the presence of occlusions.

Abstract

Explore similar work

Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality

Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery

Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization