cs.AINov 4, 2026

Which One Is Sounding? Binaural Sounding Instance Disambiguation for Audio-Visual Segmentation

Authors: Haoze Ma, Yidi Li, Yu Ma, Pengyuan Zhao, Yi Shi, Bin Ren, Yanwen Guo, Dimitrios Kanoulas

Organizations: Taiyuan University of Technology, China · University College London, United Kingdom · EPFL, Switzerland

Abstract

Existing audio-visual segmentation methods mainly predict sound-related regions or category masks and often fail to identify the sounding instance when multiple same-class candidates coexist. Recent audio-visual instance segmentation methods address instance-level prediction but rely largely on motion and cross-frame association, which may become unreliable under sparse observations or inconspicuous motion. To this end, we introduce binaural audio, whose interaural differences encode source-direction information, to provide spatial constraints for distinguishing the sounding instance among same-class visual candidates. We propose Binaural Sounding Instance Disambiguation for Audio-Visual Segmentation (BiSID-AVS), which progressively refines category-level candidate responses into a mask of the true sounding instance. BiSID-AVS first grounds category-consistent candidate regions through audio prompting, then projects binaural cues into an image-plane spatial response map, and finally refines the true sounding instance through dual-level spatial conditioning at the feature and decoder levels. Experiments on the binaurally extended AVS-Synthetic benchmark and Habitat-AVS dataset for robotic indoor perception show that BiSID-AVS achieves the best performance on the multi-candidate subsets, improving instance-level mIoU by 35.19 and 41.21 percentage points, respectively, while substantially reducing false activation on non-sounding same-class regions.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Binaural Audio-Visual Instance Segmentation

    Sep 26, 2026Saijun Wang, Guanfeng Tang, Hongbo Zhao +4Audio-Visual UnderstandingInstance Segmentation

  2. AVPS-360: A Spatial Audio-Visual Benchmark for Instance Segmentation and Tracking of Sound-Producing Persons

    Oct 29, 2026Yifu Sun, Parthasaarathy Sudarsanam, Tuomas Virtanen +1Audio-Visual UnderstandingInstance Segmentation

  3. Hear to See: Discerning Stateful Listening for Audio-Visual Instance Segmentation

    Aug 4, 2026Leiye Liu, Miao Zhang, Jiahong Jiang +7Video Object SegmentationInstance Segmentation