Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challenge
Organizations: Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute, China
Abstract
In the Complex Video Object Segmentation task, researchers are required to track and segment specific targets within cluttered environments, which rigorously tests a method's capability for target comprehension and environmental adaptability. Although SAM3, the current state-of-the-art solution, exhibits unparalleled segmentation performance and robustness on conventional targets, it underperforms on tiny and semantic-dominated objects. The root cause of this limitation lies in SAM3's insufficient comprehension of these specific target types. To address this issue, we propose TEP: Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompts. As a training-free approach, TEP leverages external tracking models and Multimodal Large Language Models to introduce tracking-enhanced prompts, thereby alleviating the difficulty SAM3 faces in understanding these challenging targets. Our method achieved first place (56.91%) on the test set of the PVUW Challenge 2026: Complex Video Object Segmentation Track.
Figures & tables
| Method | J | & | & | F | J&F | ||
|---|---|---|---|---|---|---|---|
| SAM3 | 46.63 | 44.76 | 48.50 | 56.94 | 31.54 | 51.04 | 47.90 |
| TEP (MLLM) | 55.57 | 53.44 | 57.70 | 61.36 | 42.45 | 60.75 | 57.10 |
| TEP (Tracking Model & MLLM) | 56.91 | 54.71 | 59.12 | 62.09 | 44.11 | 62.39 | 58.55 |
Video Object Segmentation
Video object segmentation assigns pixel-level masks to objects across frames while preserving their identities through motion, occlusion, and change.
See all 110 papers