Social relations are the foundation of human daily life. Developing techniques to analyze such relations from visual data bears great potential to build machines that better understand us and are capable of interacting with us at a social level. Previous investigations have remained partial due to the overwhelming diversity and complexity of the topic and consequently have only focused on a handful of social relations. In this paper, we argue that the domain-based theory from social psychology is a great starting point to systematically approach this problem. The theory provides coverage of all aspects of social relations and equally is concrete and predictive about the visual attributes and behaviors defining the relations included in each domain. We provide the first dataset built on this holistic conceptualization of social life that is composed of a hierarchical label space of social domains and social relations. We also contribute the first models to recognize such domains and relations and find superior performance for attribute based features. Beyond the encouraging performance of the attribute based approach, we also find interpretable features that are in accordance with the predictions from social psychology literature. Beyond our findings, we believe that our contributions more tightly interleave visual recognition and social psychology theory that has the potential to complement the theoretical work in the area with empirical and data-driven models of social life.

本文提出使用社会心理学中的基于领域的理论作为系统研究社会关系的切入点，并开发了第一个综合的社会生活概念数据集以及识别社会关系和领域的模型，其中表现最好的特征是基于属性的特征。除了性能良好的基于属性的方法外，还发现了一些可解释的特征，这些特征与社会心理学文献的预测一致。我们相信，我们的贡献将更加紧密地将视觉识别和社会心理学理论相结合，从而为社会生活提供基于经验和数据驱动的模型。

基于领域的社交关系识别方法