Multimodal large language models (MLLMs) are prone to non-factual or outdated
knowledge issues, which can manifest as misreading and misrecognition errors
due to the complexity of multimodal knowledge. Previous benchmarks have not
systematically analyzed the performance of editing methods in correcting these
two error types. To better represent and correct these errors, we decompose
multimodal knowledge into its visual and textual components. Different error
types correspond to different editing formats, which edits distinct part of the
multimodal knowledge. We present MC-MKE, a fine-grained Multimodal Knowledge
Editing benchmark emphasizing Modality Consistency. Our benchmark facilitates
independent correction of misreading and misrecognition errors by editing the
corresponding knowledge component. We evaluate three multimodal knowledge
editing methods on MC-MKE, revealing their limitations, particularly in terms
of modality consistency. Our work highlights the challenges posed by multimodal
knowledge editing and motivates further research in developing effective
techniques for this task.

多模态大型语言模型 (MLLMs) 存在非事实或过时知识问题，通过将多模态知识分解为其视觉和文本组成部分，我们提出了 MC-MKE，一个注重模态一致性的细粒度多模态知识编辑基准，评估了三种多模态知识编辑方法在 MC-MKE 上的性能，并揭示了它们在模态一致性方面的局限性。我们的工作强调了多模态知识编辑所面临的挑战，并激励进一步的研究以开发有效的技术来解决这个任务。

MC-MKE: 一个注重模态一致性的细粒度多模态知识编辑基准

MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark  Emphasizing Modality Consistency

Multimodal knowledge editing represents a critical advancement in enhancing
the capabilities of Multimodal Large Language Models (MLLMs). Despite its
potential, current benchmarks predominantly focus on coarse-grained knowledge,
leaving the intricacies of fine-grained (FG) multimodal entity knowledge
largely unexplored. This gap presents a notable challenge, as FG entity
recognition is pivotal for the practical deployment and effectiveness of MLLMs
in diverse real-world scenarios. To bridge this gap, we introduce MIKE, a
comprehensive benchmark and dataset specifically designed for the FG multimodal
entity knowledge editing. MIKE encompasses a suite of tasks tailored to assess
different perspectives, including Vanilla Name Answering, Entity-Level Caption,
and Complex-Scenario Recognition. In addition, a new form of knowledge editing,
Multi-step Editing, is introduced to evaluate the editing efficiency. Through
our extensive evaluations, we demonstrate that the current state-of-the-art
methods face significant challenges in tackling our proposed benchmark,
underscoring the complexity of FG knowledge editing in MLLMs. Our findings
spotlight the urgent need for novel approaches in this domain, setting a clear
agenda for future research and development efforts within the community.

通过我们的广泛评估，我们展示了目前最先进的方法在解决我们提出的基准测试中面临着重大挑战，凸显了多模态大型语言模型中细粒度知识编辑的复杂性，这进一步凸显了在这个领域中需要创新方法的迫切需求，为未来的研究和开发工作设定了明确的议程。