Detector-in-the-Loop Tracking: Active Memory Rectification for Stable Glottic Opening Localization

Huayu Wang; Bahaa Alattar; Cheng-Yen Yang; Hsiang-Wei Huang; Jung Heon Kim; Linda Shapiro; Nathan White; Jenq-Neng Hwang

cs.CVFeb 22, 2026

Detector-in-the-Loop Tracking: Active Memory Rectification for Stable Glottic Opening Localization

Authors: Huayu Wang, Bahaa Alattar, Cheng-Yen Yang, Hsiang-Wei Huang, Jung Heon Kim, Linda Shapiro, Nathan White, Jenq-Neng Hwang

Organizations: University of Washington, Seattle, WA · Ajou University School of Medicine, Suwon, Republic of Korea · Harborview Medical Center, Seattle, WA · Airlift Northwest, Seattle, WA

Abstract

Temporal stability in glottic opening localization remains challenging due to the complementary weaknesses of single-frame detectors and foundation-model trackers: the former lacks temporal context, while the latter suffers from memory drift. Specifically, in video laryngoscopy, rapid tissue deformation, occlusions, and visual ambiguities in emergency settings require a robust, temporally aware solution that can prevent progressive tracking errors. We propose Closed-Loop Memory Correction (CL-MC), a detector-in-the-loop framework that supervises Segment Anything Model 2(SAM2) through confidence-aligned state decisions and active memory rectification. High-confidence detections trigger semantic resets that overwrite corrupted tracker memory, effectively mitigating drift accumulation with a training-free foundation tracker in complex endoscopic scenes. On emergency intubation videos, CL-MC achieves state-of-the-art performance, significantly reducing drift and missing rate compared with the SAM2 variants and open loop based methods. Our results establish memory correction as a crucial component for reliable clinical video tracking. Our code will be available in https://github.com/huayuww/CL-MR.

Figures & tables

Data	mAP 50 ( $\uparrow$ )		mAP 50:95 ( $\uparrow$ )		Miss Rate ( $\downarrow$ )
Data	YOLO	Ours	YOLO	Ours	YOLO	Ours
30%	64.16	64.79 (+0.6)	35.09	36.21 (+1.1)	21.08	23.55 (+2.5)
70%	81.28	82.29 (+1.0)	45.98	46.61 (+0.6)	11.21	8.31 (-2.9)
100%	82.41	84.32 (+1.9)	51.05	50.95 (-0.1)	8.81	6.85 (-2.0)

Table 3: Performance Comparison. Our method is compared against the YOLO baseline across different data scales. The improvement relative to baseline is shown in parentheses.

(a) Temporal Window $T_{win}$ (Fixed: $\tau_{drift}=0.7,\tau_{iou}=0.5,\tau_{lost}=0.1$ )
Varying Parameter	Value	Metrics
Varying Parameter	Value	mAP 50	mAP 50:95	AUC	Missing Rate
Window Size	20	83.81	50.58	76.71	6.53
	30	84.87	51.22	76.67	6.40
	40	85.29	51.43	76.53	6.55
(b) Drift Threshold $\tau_{drift}$ (Fixed: $T_{win}=30,\tau_{iou}=0.5,\tau_{lost}=0.1$ )

Table 4: Comprehensive Hyperparameter Analysis. We systematically evaluate four hyperparameters (

T_{win},\tau_{drift},\tau_{iou},\tau_{lost}

). The method demonstrates high stability across all experimental configurations, achieving an average mAP 50 of

84.36{\scriptstyle\pm 0.43}

, mAP 50:95 of

50.97{\scriptstyle\pm 0.29}

, AUC of

76.53{\scriptstyle\pm 0.12}

and Missing Rate of

6.77{\scriptstyle\pm 0.23}

Visual Object Tracking

Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.

See all 175 papers

LiAM-SAM: Lifecycle-Aware Memory for Robust SAM2-Based MOT
Sep 23, 2026Grégoire Francisco, Alessandro D'Amico, Samuele Costantini +2Visual Object Tracking Multi-Object Tracking
Toyota Motor Europe, Zaventem, Belgium
When Trackers Fail: VLM-Guided Verification and Recovery for Robust Video Object Segmentation
Oct 11, 2026Valay Bundele, Susmit Agrawal, Mehran Hosseinzadeh +2Visual Object Tracking Video Object Segmentation
University of Tübingen, Germany · Woven by Toyota, Japan
SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking
Jun 23, 2026Mohamad Alansari, Yonathan Michael, Hasan AlMarzouqi +3Memory-Augmented Neural Networks Visual Object Tracking
Khalifa University, Abu Dhabi, UAE · University of Western Australia, Australia

Sep 23, 2026J/K move · Enter open · S save

LiAM-SAM: Lifecycle-Aware Memory for Robust SAM2-Based MOT

Grégoire Francisco, Alessandro D'Amico, Samuele Costantini, Gianpiero Francesca+1

Toyota Motor Europe, Zaventem, Belgium

Visual Object Tracking Multi-Object Tracking

Open paper AskPDF arXiv

Segmentation-based multi-object tracking (MOT) with foundation video models such as SAM2 offers strong localization quality, yet remains fragile in crowded, real-world scenes. In detector-prompted SAM2 pipelines, failures typically arise at three stages of the object lifecycle: (i) erroneous or duplicate track initiation, (ii) memory drift during close interactions, and (iii) unreliable re-identification after long occlusions or re-entry. These errors corrupt object memory and accumulate over time, making long-horizon tracking unstable. In this paper, we reframe MOT as a lifecycle memory integrity problem. We present LiAM-SAM, a Lifecycle-Aware Memory (LiAM) framework with targeted mechanisms for each of the three failure modes. At track birth, to prevent faulty or duplicate initiations, we apply contrastive track initiation, which conditions each prompt on existing nearby tracked instances. To preserve memory integrity during strong interactions, we introduce motion- and geometry-grounded memory correction that resolves interaction confusions and suppresses drift. For reliable re-identification after disappearance, we maintain an adaptive context memory that promotes diverse and trustworthy references as long-term identity anchors. Finally, similarity aware spatial pruning optionally selects the memory tokens to retain at cross-attention time, improving efficiency with minimal accuracy loss. LiAM-SAM represents a modular, detector-agnostic, SAM2-based MOT system that achieves state-of-the-art HOTA and IDF1 on the evaluated benchmarks. In association-challenging environments, our ablations show that LiAM improves a detector+SAM2 baseline by +10.5 HOTA, +17.4 AssA, and reduces identity switches by 96%.

Abstract

Visual Object Tracking

Explore similar work

LiAM-SAM: Lifecycle-Aware Memory for Robust SAM2-Based MOT

When Trackers Fail: VLM-Guided Verification and Recovery for Robust Video Object Segmentation

SENTRY: SAM2-Enhanced Neighbor-Aware and Temporally Reasoned Memory for Visual Tracking