Link prediction aims to identify potential missing triples in knowledge
graphs. To get better results, some recent studies have introduced multimodal
information to link prediction. However, these methods utilize multimodal
information separately and neglect the complicated interaction between
different modalities. In this paper, we aim at better modeling the
inter-modality information and thus introduce a novel Interactive Multimodal
Fusion (IMF) model to integrate knowledge from different modalities. To this
end, we propose a two-stage multimodal fusion framework to preserve
modality-specific knowledge as well as take advantage of the complementarity
between different modalities. Instead of directly projecting different
modalities into a unified space, our multimodal fusion module limits the
representations of different modalities independent while leverages bilinear
pooling for fusion and incorporates contrastive learning as additional
constraints. Furthermore, the decision fusion module delivers the learned
weighted average over the predictions of all modalities to better incorporate
the complementarity of different modalities. Our approach has been demonstrated
to be effective through empirical evaluations on several real-world datasets.
The implementation code is available online at
this https URL.

该研究提出了一种交互式多模态融合技术，可在保留各自模态特征的同时，更好地融合来自不同模态的知识，以实现更好的链接预测效果，并在多个真实世界数据集上进行了实证评估。