Large language models are known for encoding a vast amount of factual
knowledge, but they often becomes outdated due to the ever-changing nature of
external information. A promising solution to this challenge is the utilization
of model editing methods to update the knowledge in an efficient manner.
However, the majority of existing model editing techniques are limited to
monolingual frameworks, thus failing to address the crucial issue of
cross-lingual knowledge synchronization for multilingual models. To tackle this
problem, we propose a simple yet effective method that trains multilingual
patch neuron to store cross-lingual knowledge. It can be easily adapted to
existing approaches to enhance their cross-lingual editing capabilities. To
evaluate our method, we conduct experiments using both the XNLI dataset and a
self-constructed XFEVER dataset. Experimental results demonstrate that our
proposed method achieves improved performance in cross-lingual editing tasks
without requiring excessive modifications to the original methodology, thereby
showcasing its user-friendly characteristics. Codes will be released soon.

提出了一种简单而有效的方法，使用多语言修补神经元来存储跨语言知识，以提升现有方法的跨语言编辑能力，并在两个数据集上进行实验证明其在跨语言编辑任务中表现出了提高的性能。

MPN: 利用多语言修补神经元进行跨语言模型编辑

MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model  Editing

Recently Large Language Models (LLMs) have demonstrated their amazing text
understanding and generation capabilities. However, even stronger LLMs may
still learn incorrect knowledge from the training corpus, as well as some
knowledge that is outdated over time. Direct secondary fine-tuning with data
containing new knowledge may be ineffective in updating knowledge due to the
conflict between old and new knowledge. In this paper, we propose a new
paradigm for fine-tuning called F-Learning (Forgetting before Learning), which
is based on parametric arithmetic to achieve forgetting of old knowledge and
learning of new knowledge. Experimental results on two publicly available
datasets demonstrate that our proposed F-Learning can obviously improve the
knowledge updating performance of both full fine-tuning and LoRA fine-tuning.
Moreover, we have also discovered that forgetting old knowledge by subtracting
the parameters of LoRA can achieve a similar effect to subtracting the
parameters of full fine-tuning, and sometimes even surpass it significantly.

通过参数算术实现旧知识遗忘和新知识学习的新的精调范式 F-Learning 在两个公开数据集上的实验结果表明，该方法明显改善了完全精调和 LoRA 精调的知识更新性能，通过减去 LoRA 参数来忘记旧知识即可达到与完全精调相似的效果，有时甚至更好。

遗忘学习前：在大型语言模型中利用参数算术进行知识更新

Forgetting before Learning: Utilizing Parametric Arithmetic for  Knowledge Updating in Large Language Models

The open domain knowledge base is very important. It is usually extracted
from encyclopedia websites and is widely used in knowledge retrieval systems,
question answering systems, or recommendation systems. In practice, the key
challenge is to maintain an up-to-date knowledge base. Different from Unwieldy
fetching all of the data from the encyclopedia dumps, to enlarge the freshness
of the knowledge base as big as possible while avoiding invalid fetching, the
current knowledge base updating methods usually determine whether entities need
to be updated by building a prediction model. However, these methods can only
be defined in some specific fields and the result turns out to be obvious bias,
due to the problem of data source and data structure. The users' query
intentions are often diverse as to the open domain knowledge, so we construct a
topic-aware graph network for knowledge updating based on the user query log.
Our methods can be summarized as follow: 1. Extract entities through the user's
log and select them as seeds 2. Scrape the attributes of seed entities in the
encyclopedia website, and self-supervised construct the entity attribute graph
for each entity. 3. Use the entity attribute graph to train the GNN entity
update model to determine whether the entity needs to be synchronized. 4.Use
the encyclopedia knowledge to match and update the filtered entity with the
entity in the knowledge base according to the minimum edit times algorithm.

基于用户查询日志，构造主题感知的图网络进行知识更新，通过实体属性图训练 GNN 实体更新模型以确定实体是否需要同步，并使用最小编辑次数算法将过滤的实体与知识库中的实体进行匹配和更新。