In the Complex Video Object Segmentation task, researchers are required to track and segment specific targets within cluttered environments, which rigorously tests a method's capability for target comprehension and environmental adaptability. Although SAM3, the current state-of-the-art solution, exhibits unparalleled segmentation performance and robustness on conventional targets, it underperforms on tiny and semantic-dominated objects. The root cause of this limitation lies in SAM3's insufficient comprehension of these specific target types. To address this issue, we propose TEP: Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompts. As a training-free approach, TEP leverages external tracking models and Multimodal Large Language Models to introduce tracking-enhanced prompts, thereby alleviating the difficulty SAM3 faces in understanding these challenging targets. Our method achieved first place (56.91%) on the test set of the PVUW Challenge 2026: Complex Video Object Segmentation Track.
Figures & tables
Figure 1: Visualization of the three target types in the MOSE v2 dataset, highlighting the challenges posed by tiny targets and semantic-dominated targets in complex environments. Tiny targets are visually minuscule and easily overwhelmed by background interference, while semantic-dominated targets possess highly similar visual features that can be easily confused with intra-class instances.
Figure 2: The architecture of the proposed TEP framework, which consists of three main stages: Target Classification, Tracking Enhancement, and Prompt Fusion. The Target Classification stage categorizes video targets into Regular, Tiny, and Semantic-Dominated types using mask area calculations and MLLM. The Tracking Enhancement stage employs tracking methods for Tiny Targets and MLLM for Semantic-Dominated Targets to generate bounding box prompts. The Prompt Fusion stage dynamically integrates these prompts into the SAM3 pipeline based on IoU and confidence scores to enhance segmentation accuracy and stability.
Method
J&F˙
J
F˙
J & F˙disappear
J & F˙reappear
F
J&F
SAM3
46.63
44.76
48.50
56.94
31.54
51.04
47.90
TEP (MLLM)
55.57
53.44
57.70
61.36
42.45
60.75
57.10
TEP (Tracking Model & MLLM)
56.91
54.71
59.12
62.09
44.11
62.39
58.55
Table 1: Quantitative comparison of our TEP framework against the SAM3 baseline on the MOSE v2 test set. The metrics include overall J&F, J, F, and specific evaluations for disappearance and reappearance scenarios.
Figure 3: Qualitative visualization of segmentation results on the MOSE v2 test set. The visualizations demonstrate that the tracking and segmentation contours for both tiny and semantic-dominated targets remain clean, tight, and temporally stable, validating the efficacy of our prompt-enhancement strategy.
This technical report explores the MOSEv2 track of the PVUW 2026 Challenge, which targets complex semi-supervised video object segmentation. Built on SAM3, we develop an automatic re-prompting framework to improve robustness under target disappearance and reappearance, severe transformation, and strong same-category distractors. Our method first applies the SAM3 detector to later frames to identify same-category object candidates, and then performs DINOv3-based object-level matching with a transformation-aware target feature pool to retrieve reliable target anchors. These anchors are injected back into the SAM~3 tracker together with the first-frame mask, enabling multi-anchor propagation rather than relying solely on the initial prompt. This simple directly benefits several core challenges of MOSEv2. Our solution achieves a J&F of 51.17% on the test set, ranking 3rd in the MOSEv2 track.
Mingqi Gao, Sijie Li, Jungong Han
School of Computer Science, University of Sheffield · Department of Automation, Tsinghua University
This report presents our winning solution to the 5th PVUW MeViS-Text Challenge. The track studies referring video object segmentation under motion-centric language expressions, where the model must jointly understand appearance, temporal behavior, and object interactions. To address this problem, we build a fully training-free pipeline that combines strong multimodal large language models with SAM3. Our method contains three stages. First, Gemini-3.1 Pro decomposes each target event into instance-level grounding targets, selects the frame where the target is most clearly visible, and generates a discriminative description. Second, SAM3-agent produces a precise seed mask on the selected frame, and the official SAM3 tracker propagates the mask through the whole video. Third, a refinement stage uses Qwen3.5-Plus and behavior-level verification to correct ambiguous or semantically inconsistent predictions. Without task-specific fine-tuning, our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897. The code is available at https://github.com/Moujuruo/MeViSv2_Track_Solution_2026.
Xusheng He, Canyang Wu, Jinrong Zhang +3
Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute, China
Complex video object segmentation requires robust target propagation under severe occlusion, disappearance and reappearance. Although SAM3 provides strong promptable mask propagation, a uniform inference path remains unreliable for tiny targets with insufficient visual evidence and semantic-dominated targets whose identities depend on explicit attributes. To this end, we present VOS-Agent, a collaborative framework that retains SAM3 as the shared dense segmentation module and conditionally activates specialized agents according to target characteristics. A Target Perception and Routing Agent assigns each sequence to a regular, tiny, or semantic-dominated route. Tiny targets are supported by a Visual Tracking Agent through confidence-aware box prompts, while semantic-dominated targets are handled by an MLLM-based Semantic Agent through description-guided localization and candidate verification. On the MOSEv2 test set, VOS-Agent achieves 69.82% on the official J&F˙ metric and ranks first in the MOSEv2 Track of the 8th LSVOS Challenge at ECCV 2026.
Canyang Wu, Jinrong Zhang, Xusheng He +3
Harbin Institute of Technology, Shenzhen · Nanyang Technological University · Shenzhen Loop Area Institute