The prevalence of sarcasm in social media, conveyed through text-image
combinations, presents significant challenges for sentiment analysis and
intention mining. Current multi-modal sarcasm detection methods have been
proven to struggle with biases from spurious cues, leading to a superficial
understanding of the complex interactions between text and image. To address
these issues, we propose InterCLIP-MEP, a robust framework for multi-modal
sarcasm detection. InterCLIP-MEP introduces a refined variant of CLIP,
Interactive CLIP (InterCLIP), as the backbone, enhancing sample representations
by embedding cross-modality information in each encoder. Furthermore, a novel
training strategy is designed to adapt InterCLIP for a Memory-Enhanced
Predictor (MEP). MEP uses dynamic dual-channel memory to store valuable
historical knowledge of test samples and then leverages this memory as a
non-parametric classifier to derive the final prediction. By using InterCLIP to
encode text-image interactions more effectively and incorporating MEP,
InterCLIP-MEP offers a more robust recognition of multi-modal sarcasm.
Experiments demonstrate that InterCLIP-MEP achieves state-of-the-art
performance on the MMSD2.0 benchmark. Code and data are available at
[https://github.com/CoderChen01/InterCLIP-MEP](this https URL).

通过引入 InterCLIP-MEP 框架，结合 InterCLIP 和 MEP，并改进文本 - 图像交互编码，实现了对多模态讽刺的更强鲁棒性的识别。