Multimodal movie genre classification has always been regarded as a demanding
multi-label classification task due to the diversity of multimodal data such as
posters, plot summaries, trailers and metadata. Although existing works have
made great progress in modeling and combining each modality, they still face
three issues: 1) unutilized group relations in metadata, 2) unreliable
attention allocation, and 3) indiscriminative fused features. Given that the
knowledge graph has been proven to contain rich information, we present a novel
framework that exploits the knowledge graph from various perspectives to
address the above problems. As a preparation, the metadata is processed into a
domain knowledge graph. A translate model for knowledge graph embedding is
adopted to capture the relations between entities. Firstly we retrieve the
relevant embedding from the knowledge graph by utilizing group relations in
metadata and then integrate it with other modalities. Next, we introduce an
Attention Teacher module for reliable attention allocation based on
self-supervised learning. It learns the distribution of the knowledge graph and
produces rational attention weights. Finally, a Genre-Centroid Anchored
Contrastive Learning module is proposed to strengthen the discriminative
ability of fused features. The embedding space of anchors is initialized from
the genre entities in the knowledge graph. To verify the effectiveness of our
framework, we collect a larger and more challenging dataset named MM-IMDb 2.0
compared with the MM-IMDb dataset. The experimental results on two datasets
demonstrate that our model is superior to the state-of-the-art methods. We will
release the code in the near future.

利用知识图谱从多个角度探索解决多模态电影类型分类的问题，包括未利用的元数据组关系、可靠的注意力分配以及不可区分的特征融合，通过准备元数据的领域知识图谱，整合不同模态数据，构建可靠的注意力分配和增强特征区分能力的框架，并在两个数据集上进行实验证明本模型优于现有方法。