We develop a novel visual model which can recognize protesters, describe their activities by visual attributes and estimate the level of perceived violence in an image. Studies of social media and protests use natural language processing to track how individuals use hashtags and links, often with a focus on those items' diffusion. These approaches, however, may not be effective in fully characterizing actual real-world protests (e.g., violent or peaceful) or estimating the demographics of participants (e.g., age, gender, and race) and their emotions. Our system characterizes protests along these dimensions. We have collected geotagged tweets and their images from 2013-2017 and analyzed multiple major protest events in that period. A multi-task convolutional neural network is employed in order to automatically classify the presence of protesters in an image and predict its visual attributes, perceived violence and exhibited emotions. We also release the UCLA Protest Image Dataset, our novel dataset of 40,764 images (11,659 protest images and hard negatives) with various annotations of visual attributes and sentiments. Using this dataset, we train our model and demonstrate its effectiveness. We also present experimental results from various analysis on geotagged image data in several prevalent protest events. Our dataset will be made accessible at https://www.sscnet.ucla.edu/comm/jjoo/mm-protest/.

开发了一种新的视觉模型，该模型能够通过视觉属性识别抗议者、描述他们的活动并估计在图像中所表现的暴力水平。通过分析从2013年到2017年的地理标记推文以及它们的图像，采用多任务卷积神经网络自动分类出图像中的抗议者，并预测视觉属性、感知的暴力和展示的情绪，并发布了包含40,764张图像的 UCLA 抗议图像数据集及相关注释。

社交媒体图像的抗议活动检测和暴力评估