Addressing the issue of hallucinations and outdated knowledge in large
language models is critical for their reliable application. Model Editing
presents a promising avenue for mitigating these challenges in a cost-effective
manner. However, existing methods often suffer from unsatisfactory
generalization and unintended effects on unrelated samples. To overcome these
limitations, we introduce a novel approach: Scalable Model Editing via
Customized Expert Networks (SCEN), which is a two-stage continuous training
paradigm. Specifically, in the first stage, we train lightweight expert
networks individually for each piece of knowledge that needs to be updated.
Subsequently, we train a corresponding neuron for each expert to control the
activation state of that expert. Our experiments on two different sizes of
open-source large language models, the Llama2 7B and 13B, achieve
state-of-the-art results compared to existing mainstream Model Editing methods.
Our code is available at https: //github.com/TAL-auroraX/SCEN

通过引入一种新的方法，即 SCEN（通过定制化专家网络的可扩展性模型编辑），我们在两个不同大小的开源大语言模型 Llama2 7B 和 13B 上取得了与现有主流模型编辑方法相比的最新成果。