Multimodal Knowledge Graph Construction (MMKC) refers to the process of
creating a structured representation of entities and relationships through
multiple modalities such as text, images, videos, etc. However, existing MMKC
models have limitations in handling the introduction of new entities and
relations due to the dynamic nature of the real world. Moreover, most
state-of-the-art studies in MMKC only consider entity and relation extraction
from text data while neglecting other multi-modal sources. Meanwhile, the
current continual setting for knowledge graph construction only consider entity
and relation extraction from text data while neglecting other multi-modal
sources. Therefore, there arises the need to explore the challenge of
continuous multimodal knowledge graph construction to address the phenomenon of
catastrophic forgetting and ensure the retention of past knowledge extracted
from different forms of data. This research focuses on investigating this
complex topic by developing lifelong multimodal benchmark datasets. Based on
the empirical findings that several state-of-the-art MMKC models, when trained
on multimedia data, might unexpectedly underperform compared to those solely
utilizing textual resources in a continual setting, we propose a Lifelong
MultiModal Consistent Transformer Framework (LMC) for continuous multimodal
knowledge graph construction. By combining the advantages of consistent KGC
strategies within the context of continual learning, we achieve greater balance
between stability and plasticity. Our experiments demonstrate the superior
performance of our method over prevailing continual learning techniques or
multimodal approaches in dynamic scenarios. Code and datasets can be found at
this https URL

本研究的重点是通过开发终身多模式基准数据集，研究探索多模式知识图构建的持续挑战，提出了一种基于 Lifelong MultiModal Consistent Transformer 框架（LMC）的方法，并通过实验证明，与现有技术相比，在动态场景中具有更好的性能。