Active speaker detection requires a solid integration of multi-modal cues.
While individual modalities can approximate a solution, accurate predictions
can only be achieved by explicitly fusing the audio and visual features and
modeling their temporal progression. Despite its inherent muti-modal nature,
current methods still focus on modeling and fusing short-term audiovisual
features for individual speakers, often at frame level. In this paper we
present a novel approach to active speaker detection that directly addresses
the multi-modal nature of the problem, and provides a straightforward strategy
where independent visual features from potential speakers in the scene are
assigned to a previously detected speech event. Our experiments show that, an
small graph data structure built from a single frame, allows to approximate an
instantaneous audio-visual assignment problem. Moreover, the temporal extension
of this initial graph achieves a new state-of-the-art on the AVA-ActiveSpeaker
dataset with a mAP of 88.8\%.

本文提出了将图形数据结构应用于音频和视觉信号融合的新方法，将多模式的音视频特征分配给先前检测到的语音事件，实现了 88.8％的 AV-ActiveSpeaker 数据集性能最优。