Capturing the `mutual gaze' of people is essential for understanding and
interpreting the social interactions between them. To this end, this paper
addresses the problem of detecting people Looking At Each Other (LAEO) in video
sequences. For this purpose, we propose LAEO-Net, a new deep CNN for
determining LAEO in videos. In contrast to previous works, LAEO-Net takes
spatio-temporal tracks as input and reasons about the whole track. It consists
of three branches, one for each character's tracked head and one for their
relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and
AVA-LAEO. A thorough experimental evaluation demonstrates the ability of
LAEONet to successfully determine if two people are LAEO and the temporal
window where it happens. Our model achieves state-of-the-art results on the
existing TVHID-LAEO video dataset, significantly outperforming previous
approaches. Finally, we apply LAEO-Net to social network analysis, where we
automatically infer the social relationship between pairs of people based on
the frequency and duration that they LAEO.

本文基于 LAEO-Net 提出了一种新的深度 CNN 模型用于检测人们的相互凝视，将空间时间轨迹作为输入，由三个分支构成，分别针对每个角色的追踪头部和相对位置。通过实验证明该模型在分析人们的社交关系中可以成功地确定两个人是否进行相互注视以及注视的时间窗口，达到了最先进的技术水平。