Audio-Visual Segmentation (AVS) aims to precisely outline audible objects in
a visual scene at the pixel level. Existing AVS methods require fine-grained
annotations of audio-mask pairs in supervised learning fashion. This limits
their scalability since it is time consuming and tedious to acquire such
cross-modality pixel level labels. To overcome this obstacle, in this work we
introduce unsupervised audio-visual segmentation with no need for task-specific
data annotations and model training. For tackling this newly proposed problem,
we formulate a novel Cross-Modality Semantic Filtering (CMSF) approach to
accurately associate the underlying audio-mask pairs by leveraging the
off-the-shelf multi-modal foundation models (e.g., detection [1], open-world
segmentation [2] and multi-modal alignment [3]). Guiding the proposal
generation by either audio or visual cues, we design two training-free
variants: AT-GDINO-SAM and OWOD-BIND. Extensive experiments on the AVS-Bench
dataset show that our unsupervised approach can perform well in comparison to
prior art supervised counterparts across complex scenarios with multiple
auditory objects. Particularly, in situations where existing supervised AVS
methods struggle with overlapping foreground objects, our models still excel in
accurately segmenting overlapped auditory objects. Our code will be publicly
released.

语音 - 视觉分割（AVS）通过像素级精确在视觉场景中描绘可听到的对象。我们提出了一种新的跨模态语义过滤（CMSF）方法，以解决这个问题，并利用现成的多模态基础模型来准确地关联潜在的音频 - 掩码对。我们的无监督方法在复杂情景中的多个听觉对象上表现优异，特别是在现有监督 AVS 方法在重叠前景对象方面表现困难的情况下，仍能准确地分割重叠的听觉对象。