Sarcasm is a linguistic phenomenon indicating a discrepancy between literal
meanings and implied intentions. Due to its sophisticated nature, it is usually
challenging to be detected from the text itself. As a result, multi-modal
sarcasm detection has received more attention in both academia and industries.
However, most existing techniques only modeled the atomic-level inconsistencies
between the text input and its accompanying image, ignoring more complex
compositions for both modalities. Moreover, they neglected the rich information
contained in external knowledge, e.g., image captions. In this paper, we
propose a novel hierarchical framework for sarcasm detection by exploring both
the atomic-level congruity based on multi-head cross attention mechanism and
the composition-level congruity based on graph neural networks, where a post
with low congruity can be identified as sarcasm. In addition, we exploit the
effect of various knowledge resources for sarcasm detection. Evaluation results
on a public multi-modal sarcasm detection dataset based on Twitter demonstrate
the superiority of our proposed model.

本文提出了一种基于层次结构的检测方法，通过多头交叉注意力机制和图神经网络同时探索文本和图像之间的原子级一致性和组成级一致性，利用丰富的外部知识资源进行讽刺检测，评估结果表明该模型优于现有的技术。