Our objective is audio-visual synchronization with a focus on 'in-the-wild'
videos, such as those on YouTube, where synchronization cues can be sparse. Our
contributions include a novel audio-visual synchronization model, and training
that decouples feature extraction from synchronization modelling through
multi-modal segment-level contrastive pre-training. This approach achieves
state-of-the-art performance in both dense and sparse settings. We also extend
synchronization model training to AudioSet a million-scale 'in-the-wild'
dataset, investigate evidence attribution techniques for interpretability, and
explore a new capability for synchronization models: audio-visual
synchronizability.

我们的研究目标是在 YouTube 等 ' 野外 ' 视频中实现音频和视觉的同步，我们提出了一种新颖的音视频同步模型，并通过多模式片段级对比预训练将特征提取与同步建模分离开来。这种方法在密集和稀疏环境下都实现了最先进的性能，并将同步模型训练扩展到了 AudioSet 这个年度规模的 ' 野外 ' 数据集，并研究了可解释性的证据归因技术，探索了同步模型的新能力：音视频同步能力。