In this paper we address the problem of tracking multiple speakers via the
fusion of visual and auditory information. We propose to exploit the
complementary nature of these two modalities in order to accurately estimate
smooth trajectories of the tracked persons, to deal with the partial or total
absence of one of the modalities over short periods of time, and to estimate
the acoustic status -- either speaking or silent -- of each tracked person
along time. We propose to cast the problem at hand into a generative
audio-visual fusion (or association) model formulated as a latent-variable
temporal graphical model. This may well be viewed as the problem of maximizing
the posterior joint distribution of a set of continuous and discrete latent
variables given the past and current observations, which is intractable. We
propose a variational inference model which amounts to approximate the joint
distribution with a factorized distribution. The solution takes the form of a
closed-form expectation maximization procedure. We describe in detail the
inference algorithm, we evaluate its performance and we compare it with several
baseline methods. These experiments show that the proposed audio-visual tracker
performs well in informal meetings involving a time-varying number of people.

本文提出了一种基于视听信息融合技术框架的多说话人跟踪系统，利用可变因素推断方法近似求解了连续和离散潜变量的后验联合分布，实现了跟踪对象的平滑轨迹估计和说话状态的判断。实验结果表明该方法在非正式会议中表现出较好的性能。