Investigating the trackability of landmarks in NBI and WLI
Authors: Jonas Below, Piermarco Gerli, Anna Jötten, Dennis Eggert, Axelle Felicio-Briegel, Veronika Volgger, Christian Betz, Alexander Schlaefer
Organizations: Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany · Department of Otorhinolaryngology, Head and Neck Surgery and Oncology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg, Germany · LMU Klinikum, Marchioninistr. 15, 81377 München, Germany
Reliable visual tracking of anatomical landmarks in endoscopic video is a key prerequisite for computer-assisted interventions in laryngeal surgery. While Narrow Band Imaging (NBI) is clinically established for enhancing vascular contrast, its effect on automated tracking relative to conventional White Light Imaging (WLI) remains unexplored. We present a quantitative comparison of landmark trackability in NBI and WLI using a forward-backward error framework applied to 61 endoscopic videos from 39 patients. Two regions of interest are evaluated: the vocal folds (VF) and mucosa with a prominent vascular pattern (MPVP). We compare a classical feature-based tracker (CSRT) against learning-based point trackers (CoTracker3-Online and TrackOnR) across both modalities. NBI consistently reduced the forward-backward error in both regions, reaching statistical significance for the vocal folds, while failure rates stayed low. These findings indicate that NBI can improve tracking robustness in laryngeal endoscopy, with implications for motion analysis, navigation, and robotic assistance.
Figures & tables
Fig. 1: Example placement of n~=100 tracking candidates within manually annotated VF (red) and MPVP ROIs (green). For point-based trackers, the centroid of each box was used as the tracking point.
Tab. 1: mFBE in pixels ( ↓ ) for MPVP and VF, including the p-value of the modality comparison.
Point tracking follows selected or dense image points through video, enabling motion analysis, 3D reconstruction, and precise spatiotemporal understanding.
White-light imaging (WLI) and narrow-band imaging (NBI) provide complementary cues for endoscopic lesion segmentation, yet their sequential acquisition often results in spatially inconsistent observations. The central challenge is not merely to establish cross-modal correspondence, but to determine how complementary information should be incorporated when such correspondence is uncertain. To address this issue, we propose a correspondence-conditioned, modality-asymmetric complex spectral fusion framework for paired-but-unregistered WLI/NBI segmentation. The framework first establishes bidirectional feature correspondence through topology-regularized registration and derives local reliability from post-warping feature discrepancies. Rather than simply reweighting fused features, the estimated reliability directly controls the construction of modality-asymmetric polar representations, where WLI-derived cues primarily parameterize appearance-related moduli and NBI-derived cues parameterize structure-sensitive angular components. The resulting complex representations are further refined through Fourier-domain recalibration with reliability-biased residual reconstruction. Experiments on FAHSYSU and PICCOLO demonstrate competitive improvements over existing segmentation methods. Module ablations, modality-role reversal, and reliability analyses further support the effectiveness of correspondence-conditioned polar representation learning.
Pengyu Jie, Wanquan Liu, Rui He +5
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen, 518107, Guangdong, P.R. China · Department of Otolaryngology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, 510000, P.R. China · School of Computer Science and Technology, Hainan University, Haikou, 570228, China +2
Reliable vocal-cord localization in emergency video laryngoscopy remains visually challenging due to rapid tissue deformation, specular highlights, and instrument occlusions. We propose Local Point Tracking (LPT), a robust framework that combines a single-frame detector, local point propagation, and stateless mask refinement. LPT follows a localize first, snap second pipeline. First, a single-frame semantic detector generates sparse anchors, within which LPT initializes local query points (SAM-supported by default, with a regular grid as fallback). A point-propagation module tracks these points through detector-missed frames and recovers temporal bounding boxes from their geometry. Second, because point-derived boxes can be loose around the visible boundary, a box-snap step refines the selected box to the target boundary. This decomposition keeps temporal state in local point coordinates rather than a stateful mask or whole-region representation, while the spatial refinement module handles single-frame boundary precision without feeding mask errors back into temporal memory. Extensive evaluations on two distinct datasets demonstrate that our proposed LPT method achieves strong overall performance against existing baselines, with the best mAP_50, success AUC, and missing rate on both datasets. Furthermore, ablation studies confirm that point tracking primarily enhances temporal tracking robustness, while box-snap refinement effectively improves boundary tightness.
Huayu Wang, Gyungmin Ko, Jung Heon Kim +9
University of Washington · Ajou University · National Cheng Kung University +3
Early and timely screening of laryngeal cancer is crucial for improving clinical outcomes. In recent years, NBI endoscopy has become a standard diagnostic tool for the detection of laryngeal lesions. However, its effective use requires well-trained clinicians and the procedure is time-consuming and subject to interobserver variability. In this context, the application of artificial intelligence (AI) offers a promising solution to support clinical decision-making. In this work, we proposed applying transformer and attention mechanism for analyzing the narrow band imaging and distinguish benign and malignant lesions. Results show it has good classification performance with F1 (82.72%), accuracy(82.33%). In addition, the result of laryngeal cancer screening is explainable for clinicians. The explainability is utilizing the state of art segmentation method (MedSAM) to provide the useful pathological information area for clinicians. The proposed methodology fusing classification and segmentation provides a translating on laryngeal cancer screening.
Haiyang Wang, Luca Mainardi
Department of Electrical, Information and Bioengineering Polytechnic University of Milan Milan, Italy · Department of of Electrical, Information and Bioengineering Polytechnic University of Milan Milan, Italy