The advancement of Multi-modal Pre-training highlights the necessity for a
robust Multi-Modal Knowledge Graph (MMKG) representation learning framework.
This framework is crucial for integrating structured knowledge into multi-modal
Large Language Models (LLMs) at scale, aiming to alleviate issues like
knowledge misconceptions and multi-modal hallucinations. In this work, to
evaluate models' ability to accurately embed entities within MMKGs, we focus on
two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and
Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a
novel SNAG method that utilizes a Transformer-based architecture equipped with
modality-level noise masking for the robust integration of multi-modal entity
features in KGs. By incorporating specific training objectives for both MKGC
and MMEA, our approach achieves SOTA performance across a total of ten datasets
(three for MKGC and seven for MEMA), demonstrating its robustness and
versatility. Besides, SNAG can not only function as a standalone model but also
enhance other existing methods, providing stable performance improvements. Our
code and data are available at: this https URL

通过提出一种基于 Transformer 架构的 SNAG 方法，结合多模态实体特征在知识图谱中进行鲁棒的集成，本研究在多模态预训练的基础上，展示了在多模态知识图谱完成和多模态实体对齐两个任务中的领先性能，同时还具有增强其他现有方法的稳定性和性能改进的功能。