Visual Emotion Analysis (VEA) is attracting increasing attention. One of the
biggest challenges of VEA is to bridge the affective gap between visual clues
in a picture and the emotion expressed by the picture. As the granularity of
emotions increases, the affective gap increases as well. Existing deep
approaches try to bridge the gap by directly learning discrimination among
emotions globally in one shot without considering the hierarchical relationship
among emotions at different affective levels and the affective level of
emotions to be classified. In this paper, we present the Multi-level Dependent
Attention Network (MDAN) with two branches, to leverage the emotion hierarchy
and the correlation between different affective levels and semantic levels. The
bottom-up branch directly learns emotions at the highest affective level and
strictly follows the emotion hierarchy while predicting emotions at lower
affective levels. In contrast, the top-down branch attempt to disentangle the
affective gap by one-to-one mapping between semantic levels and affective
levels, namely, Affective Semantic Mapping. At each semantic level, a local
classifier learns discrimination among emotions at the corresponding affective
level. Finally, We integrate global learning and local learning into a unified
deep framework and optimize the network simultaneously. Moreover, to properly
extract and leverage channel dependencies and spatial attention while
disentangling the affective gap, we carefully designed two attention modules:
the Multi-head Cross Channel Attention module and the Level-dependent Class
Activation Map module. Finally, the proposed deep framework obtains new
state-of-the-art performance on six VEA benchmarks, where it outperforms
existing state-of-the-art methods by a large margin, e.g., +3.85% on the WEBEmo
dataset at 25 classes classification accuracy.

本文提出了一个名为 “多层依赖注意力网络”（MDAN）的深度学习框架，通过融合全局学习和局部学习优化网络，采用底层朴素贝叶斯分类器、自底而上的分类器和自顶而下的情感语义映射分类器来消除视觉线索和图片表达情感之间的 “情感差距”，同时设计两种注意力模块以恰当地提取和利用通道相关性和空间注意力，相比现有技术在 6 个视觉情感分析基准上取得了新的最优性能。