cs.MMAug 23, 2026

Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026

Authors: Jinxing Zhou, Suiyi Zhao, Yanghao Zhou, Ruohao Guo

Organizations: Mohamed bin Zayed University of Artificial Intelligence · Anhui University of Science and Technology · National University of Singapore · China Agricultural University

Abstract

Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here, speech carries a linguistic instruction rather than acoustic evidence from a sounding object, so a solution must connect speech recognition, motion-centric temporal grounding, mask tracking, and explicit no-target handling. We introduce Speech2MaskTrack, our approach for the MeViS-Audio track of the 8th LSVOS Challenge. Speech2MaskTrack transcribes the spoken query and compiles it into structured constraints over category, count, direction, interaction role, and temporal phase. SAM3.1 enumerates multiple instance tracks, which TRACE ranks using complete-trajectory motion and relation evidence. A frozen lexical presence gate may suppress the ranked SAM3.1 base prediction. When the gate predicts that a target is present, an available full-expression-conditioned SaSaSa2VA track replaces the SAM3.1 mask. Only outputs that remain empty enter GPT-assisted recovery, which invokes SaSaSa2VA again under query- and mask-level verification. Speech2MaskTrack achieved second place in the official challenge ranking.

Figures & tables

Explore similar work

CardsList
  1. Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge

    Aug 10, 2026Yiwen Ren, Jianing Liu, Yingxin Wang +4Audio-Visual UnderstandingReferring Video Object Segmentation

  2. APRVOS: 1st Place Winner of 5th PVUW MeViS-Audio Track

    Apr 20, 2026Deshui Miao, Yameng Gu, Chao Yang +3Audio-Visual UnderstandingVideo Object Segmentation

  3. The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation

    Apr 1, 2026Xusheng He, Canyang Wu, Jinrong Zhang +3Referring Video Object Segmentation