Classification of social media data is an important approach in understanding
user behavior on the Web. Although information on social media can be of
different modalities such as texts, images, audio or videos, traditional
approaches in classification usually leverage only one prominent modality.
Techniques that are able to leverage multiple modalities are often complex and
susceptible to the absence of some modalities. In this paper, we present simple
models that combine information from different modalities to classify social
media content and are able to handle the above problems with existing
techniques. Our models combine information from different modalities using a
pooling layer and an auxiliary learning task is used to learn a common feature
space. We demonstrate the performance of our models and their robustness to the
missing of some modalities in the emotion classification domain. Our
approaches, although being simple, can not only achieve significantly higher
accuracies than traditional fusion approaches but also have comparable results
when only one modality is available.

本研究使用池化层和辅助学习任务进行多模态社交媒体数据分类，具有良好的鲁棒性和高准确性，并与传统融合方法相比有着显著的优势。