Multimodal large language models (MLLMs) are prone to non-factual or outdated
knowledge issues, which can manifest as misreading and misrecognition errors
due to the complexity of multimodal knowledge. Previous benchmarks have not
systematically analyzed the performance of editing methods in correcting these
two error types. To better represent and correct these errors, we decompose
multimodal knowledge into its visual and textual components. Different error
types correspond to different editing formats, which edits distinct part of the
multimodal knowledge. We present MC-MKE, a fine-grained Multimodal Knowledge
Editing benchmark emphasizing Modality Consistency. Our benchmark facilitates
independent correction of misreading and misrecognition errors by editing the
corresponding knowledge component. We evaluate three multimodal knowledge
editing methods on MC-MKE, revealing their limitations, particularly in terms
of modality consistency. Our work highlights the challenges posed by multimodal
knowledge editing and motivates further research in developing effective
techniques for this task.

多模态大型语言模型 (MLLMs) 存在非事实或过时知识问题，通过将多模态知识分解为其视觉和文本组成部分，我们提出了 MC-MKE，一个注重模态一致性的细粒度多模态知识编辑基准，评估了三种多模态知识编辑方法在 MC-MKE 上的性能，并揭示了它们在模态一致性方面的局限性。我们的工作强调了多模态知识编辑所面临的挑战，并激励进一步的研究以开发有效的技术来解决这个任务。

MC-MKE: 一个注重模态一致性的细粒度多模态知识编辑基准

MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark  Emphasizing Modality Consistency

We study the appropriateness of Large Language Models (LLMs) as knowledge
repositories. We focus on the challenge of maintaining LLMs' factual knowledge
up-to-date over time. Motivated by the lack of studies on identifying outdated
knowledge within LLMs, we design and develop a dynamic benchmark with
up-to-date ground truth answers for each target factual question. We evaluate
eighteen open-source and closed-source state-of-the-art LLMs on time-sensitive
knowledge retrieved in real-time from Wikidata. We select time-sensitive domain
facts in politics, sports, and organizations, and estimate the recency of the
information learned by the model during pre-training\fine-tuning. In the second
contribution, we evaluate the effectiveness of knowledge editing methods for
aligning LLMs with up-to-date factual knowledge and compare their performance
with Retrieval Augmented Generation. The dynamic benchmark is designed to be
used as-is to assess LLMs's up-to-dateness, as well as to be extended to other
domains by sharing the code, the dataset, as well as evaluation and
visualization scripts.

将大规模语言模型作为知识库，我们研究了其随时间推移保持事实知识的适当性。我们设计了一个动态基准来识别过时的知识，并通过实时从 Wikidata 检索的时间敏感领域政治、体育和组织的信息评估了 18 种开源和闭源的最先进大规模语言模型。我们评估了知识编辑方法与检索增强生成在与最新事实知识对齐方面的效果，并通过共享代码、数据集以及评估和可视化脚本将这个动态基准用于评估大规模语言模型的及时性，并可扩展至其他领域。

您的 LLM 已过时吗？时间敏感知识的 LLM 与对齐算法的基准测试

Is Your LLM Outdated? Benchmarking LLMs & Alignment Algorithms for  Time-Sensitive Knowledge

Addressing the issue of hallucinations and outdated knowledge in large
language models is critical for their reliable application. Model Editing
presents a promising avenue for mitigating these challenges in a cost-effective
manner. However, existing methods often suffer from unsatisfactory
generalization and unintended effects on unrelated samples. To overcome these
limitations, we introduce a novel approach: Scalable Model Editing via
Customized Expert Networks (SCEN), which is a two-stage continuous training
paradigm. Specifically, in the first stage, we train lightweight expert
networks individually for each piece of knowledge that needs to be updated.
Subsequently, we train a corresponding neuron for each expert to control the
activation state of that expert. Our experiments on two different sizes of
open-source large language models, the Llama2 7B and 13B, achieve
state-of-the-art results compared to existing mainstream Model Editing methods.
Our code is available at https: //github.com/TAL-auroraX/SCEN

通过引入一种新的方法，即 SCEN（通过定制化专家网络的可扩展性模型编辑），我们在两个不同大小的开源大语言模型 Llama2 7B 和 13B 上取得了与现有主流模型编辑方法相比的最新成果。