Unsupervised audio-visual source localization aims at localizing visible
sound sources in a video without relying on ground-truth localization for
training. Previous works often seek high audio-visual similarities for likely
positive (sounding) regions and low similarities for likely negative regions.
However, accurately distinguishing between sounding and non-sounding regions is
challenging without manual annotations. In this work, we propose a simple yet
effective approach for Easy Visual Sound Localization, namely EZ-VSL, without
relying on the construction of positive and/or negative regions during
training. Instead, we align audio and visual spaces by seeking audio-visual
representations that are aligned in, at least, one location of the associated
image, while not matching other images, at any location. We also introduce a
novel object guided localization scheme at inference time for improved
precision. Our simple and effective framework achieves state-of-the-art
performance on two popular benchmarks, Flickr SoundNet and VGG-Sound Source. In
particular, we improve the CIoU of the Flickr SoundNet test set from 76.80% to
83.94%, and on the VGG-Sound Source dataset from 34.60% to 38.85%. The code is
available at this https URL

本文提出了一种名为 EZ-VSL 的简单而有效的无监督音频 - 视觉源定位方法，旨在识别视频中的可见声源，其采用对齐音频和视觉空间的方法来实现，在 Flickr SoundNet 和 VGG-Sound Source 数据集上均取得了良好的表现，特别是在 CIoU 方面从 76.80% 提高到了 83.94%。