Key-Frame Reasoning with SAM3: Third Place Solution for the MeViS-Text Track of the 8th LSVOS Challenge
Organizations: Harbin Institute of Technology, Shenzhen · Nanyang Technological University · Shenzhen Loop Area Institute
Abstract
This report presents a two-stage, training-free solution for the MeViS-Text track of the 8th LSVOS Challenge. The task requires a model to localize and segment the object specified by a natural-language expression throughout a video. Such expressions often depend on temporal cues, including actions, interactions, directions, and relative positions. Our first stage uses Gemini-3.1 Pro via API to decompose a video-level event into instance-level targets, select a key frame for each target, and generate a discriminative description aligned with that frame. In the second stage, SAM3-agent produces a pixel-level seed mask on the selected frame, and the SAM3 video tracker propagates the mask bidirectionally through the video. Valid instances are grounded and propagated independently before their frame-wise masks are merged. All local SAM3 processing runs on a single NVIDIA GeForce RTX 4090 without task-specific training or model ensembling. Our method ranked third on the challenge test set, obtaining J&F, J, F, N-acc., T-acc., and Final scores of 0.761, 0.7367, 0.7852, 0.8333, 0.9755, and 0.856593, respectively.
Figures & tables
| Rank | J&F | J | F | N-acc. | T-acc. | Final |
|---|---|---|---|---|---|---|
| 1 | 0.7922 | 0.7657 | 0.8187 | 0.9444 | 0.9877 | 0.908134 |
| 2 | 0.7163 | 0.6894 | 0.7432 | 0.9722 | 0.9755 | 0.888004 |
| 3 | 0.761 | 0.7367 | 0.7852 | 0.8333 | 0.9755 | 0.856593 |
Referring Video Object Segmentation
Referring video object segmentation grounds language descriptions in video frames and produces temporally consistent masks for the described objects.
See all 42 papers