The problem of estimating subjective visual properties (SVP) of images (e.g., Shoes A is more comfortable than B) is gaining rising attention. Due to its highly subjective nature, different annotators often exhibit different interpretations of scales when adopting absolute value tests. Therefore, recent investigations turn to collect pairwise comparisons via crowdsourcing platforms. However, crowdsourcing data usually contains outliers. For this purpose, it is desired to develop a robust model for learning SVP from crowdsourced noisy annotations. In this paper, we construct a deep SVP prediction model which not only leads to better detection of annotation outliers but also enables learning with extremely sparse annotations. Specifically, we construct a comparison multi-graph based on the collected annotations, where different labeling results correspond to edges with different directions between two vertexes. Then, we propose a generalized deep probabilistic framework which consists of an SVP prediction module and an outlier modeling module that work collaboratively and are optimized jointly. Extensive experiments on various benchmark datasets demonstrate that our new approach guarantees promising results.

本文提出了一种基于深度学习、概率建模和排除异常值技术的主观可视属性(SVP)模型。该模型不仅可以从众包数据中检测到标注异常值，并且能够学习带有极度稀疏注释的数据。特别的，其构造了一个基于标注比较的多图，提出了一个广义概率框架，包括了一个SVP预测模块和一个异常值建模模块，从而在各种基准数据集上取得了相当不错的效果。

众包中深度鲁棒主观视觉属性预测