Beyond Region Memory: Local Point Tracking for Vocal Cord Localization in Video Laryngoscopy
Authors: Huayu Wang, Gyungmin Ko, Jung Heon Kim, Jia-Xian Jian, Dae Kon Kim, Dong Keon Lee, Hsiang-Wei Huang, Pau-Choo Chung, Linda Shapiro, Richard B. Utarnachitt, Jenq-Neng Hwang, Nathan J. White
Organizations: University of Washington · Ajou University · National Cheng Kung University · Gangneung Asan Hospital · Seoul National University Bundang Hospital · Airlift Northwest
Reliable vocal-cord localization in emergency video laryngoscopy remains visually challenging due to rapid tissue deformation, specular highlights, and instrument occlusions. We propose Local Point Tracking (LPT), a robust framework that combines a single-frame detector, local point propagation, and stateless mask refinement. LPT follows a localize first, snap second pipeline. First, a single-frame semantic detector generates sparse anchors, within which LPT initializes local query points (SAM-supported by default, with a regular grid as fallback). A point-propagation module tracks these points through detector-missed frames and recovers temporal bounding boxes from their geometry. Second, because point-derived boxes can be loose around the visible boundary, a box-snap step refines the selected box to the target boundary. This decomposition keeps temporal state in local point coordinates rather than a stateful mask or whole-region representation, while the spatial refinement module handles single-frame boundary precision without feeding mask errors back into temporal memory. Extensive evaluations on two distinct datasets demonstrate that our proposed LPT method achieves strong overall performance against existing baselines, with the best mAP_50, success AUC, and missing rate on both datasets. Furthermore, ablation studies confirm that point tracking primarily enhances temporal tracking robustness, while box-snap refinement effectively improves boundary tightness.
Figures & tables
Figure 1: Motivation of local point tracking. Region-level trackers propagate whole-box or mask features and may drift under deformation, occlusion, specular glare, or similar surrounding tissue. LPT instead tracks target-supported local points, recovers a box from their motion, and applies mask-snap refinement for tighter boundaries.
Figure 2: Overview of LPT. The pipeline follows a localize first, snap second cascade. A YOLO anchor initializes target-supported points, CoTracker propagates them, point-to-box recovery and motion-guided scoring select a candidate box, and a stateless SAM2 box-snap refiner produces the final tight box. Point initialization, propagation, and box recovery are described in Sec. 3.1 ; candidate scoring and box-snap refinement are described in Sec. 3.2 . Blue, orange, and green blocks denote learned models, deterministic modules, and intermediate or output boxes.
Harborview
SNUBH (23 seq.)
Method
mAP 50↑
mAP 50:95↑
AUC ↑
MR ↓
mAP 50↑
mAP 50:95↑
AUC ↑
MR ↓
YOLOv12
81.65
54.68
74.84
10.58
54.20
29.27
47.27
41.88
ByteTrack
81.16
50.52
73.10
10.92
53.28
26.61
46.06
42.59
BoT-SORT
81.84
54.08
74.57
10.39
54.24
28.65
46.91
41.85
CL-MC
84.32
50.95
76.52
6.85
56.50
32.67
55.53
35.25
SAM-PT
68.82
48.80
67.27
23.35
40.07
29.43
41.96
54.62
Table 1: Main detection results. Harborview follows the CL-MC benchmark protocol; SNUBH reports all 23 sequences. CL-MC is tested with the same YOLO weights used by detector-dependent methods. LPT (online) is the streaming variant described in Sec. 4.1 . Bold = best, underline = second.
Class
LPT
YOLO-only
ByteTrack
CL-MC(base)
SAM-PT
SAMURAI
liver
81.58
70.10
34.79
75.45
66.88
77.22
abdominal wall
74.71
68.03
53.96
75.93
48.39
69.34
grasper
72.64
52.74
37.16
75.23
46.53
43.33
gallbladder
54.97
24.48
14.49
48.59
46.43
21.87
fat
48.77
43.15
21.18
49.43
31.81
23.47
l-hook
29.20
27.82
16.29
28.54
12.68
26.10
Table 2: External public-dataset stress test on CholecSeg8k. We report class-wise AP 50 in percent for six evaluated classes after converting semantic masks to box targets under a fixed protocol. LPT denotes the default method with SAM2 box-snap refinement. This experiment is not used for tuning. Bold = best.
Figure 3: Qualitative comparison under detector failure. Rows show sampled frames from one sequence (time increases downward; gaps are non-uniform), and columns show each method’s predicted box: YOLO (white), BoT-SORT (red), ByteTrack (orange), CL-MC (purple), LPT (cyan), and the ground truth (green). After an initially correct detection, YOLO becomes unreliable under glare, deformation, and rapid motion, causing detection-dependent trackers to drift or drop the target. LPT maintains localization with local point trajectories and then snaps the final box to the visible boundary.
Dataset
Gap len.
#gaps
CL-MC
LPT (w/o box snap)
LPT (default)
Mean IoU
1–3
86
53.1
49.0
62.9
59.4
Harborview
4–10
32
48.3
46.3
52.2
55.5
> 10
12
42.6
46.4
51.9
51.9
1–3
341
42.1
45.7
53.2
52.1
SNUBH
4–10
124
30.1
37.7
46.0
45.0
> 10
97
11.0
15.2
17.7
24.8
Table 3: Recovery under detector-missed intervals. Gaps are maximal labelled-frame runs where YOLO has IoU < 0.5. Recovery is the percentage of gap frames re-localized at IoU ≥ 0.5; Mean IoU is the IoU of LPT (default) averaged over all gap frames in the bucket. Bold = best recovery.
Components
Harborview
SNUBH
CoTr.
Re-anc.
Score
Snap
mAP 50↑
mAP 50:95↑
AUC ↑
MR ↓
mAP 50↑
mAP 50:95↑
AUC ↑
MR ↓
81.65
54.68
74.84
10.58
54.20
29.27
47.27
41.88
✓
83.64
65.07
81.72
8.13
56.91
41.82
55.71
38.68
✓
✓
✓
88.32
57.04
78.48
6.36
59.96
30.31
54.91
34.13
✓
✓
✓
85.26
65.87
84.00
5.72
61.04
43.97
61.42
33.42
✓
✓
✓
88.10
67.58
82.38
7.39
60.46
42.54
60.40
34.89
Table 4: Module ablation and sanity controls. Rows toggle temporal point tracking ( CoTr. ), detector re-anchoring, deterministic candidate scoring, and final SAM2 box snap. Bold = best, underline = second.
Point source
Recovery
Harborview
SNUBH
mAP 50
mAP 50:95
mAP 50
mAP 50:95
SAM mask
blend
88.32
57.04
59.96
30.31
SAM mask
quantile
88.08
57.05
60.15
30.54
SAM mask
affine
87.93
56.86
59.68
30.23
grid
blend
87.71
56.76
59.72
30.17
grid
quantile
87.97
56.87
59.91
30.39
Table 5: Point-sampling and box-recovery choices. All rows use the full pre-snap pipeline, isolating query-point source and point-to-box recovery. The main experiments use SAM-mask-supported query initialization with the visibility-weighted blend; regular-grid sampling is included as an ablation. mAP 50 stays within about 0.6 points across variants.
Method
ms/frame ↓
FPS ↑
HV mAP 50 ↑
HV mAP 50:95 ↑
YOLO only
12.9
77.5
81.65
54.68
CL-MC
51.5
19.4
84.32
50.95
LPT (w/o box snap)
19.4
51.4
88.32
57.04
LPT (default)
70.0
14.3
89.06
67.87
Table 6: Runtime/efficiency at 1440 × 1080 on a single NVIDIA H100 (80 GB) GPU, tested end-to-end. Accuracy columns reuse Harborview values from Table 1 .