Model editing has recently gained widespread attention. Current model editing
methods primarily involve modifying model parameters or adding additional
modules to the existing model. However, the former causes irreversible damage
to LLMs, while the latter incurs additional inference overhead and fuzzy vector
matching is not always reliable. To address these issues, we propose an
expandable Subject Word Embedding Altering (SWEA) framework, which modifies the
representation of subjects and achieve the goal of editing knowledge during the
inference stage. SWEA uses precise key matching outside the model and performs
reliable subject word embedding altering, thus protecting the original weights
of the model without increasing inference overhead. We then propose optimizing
then suppressing fusion method, which first optimizes the embedding vector for
the editing target and then suppresses the Knowledge Embedding Dimension (KED)
to obtain the final fused embedding. We thus propose SWEAOS method for editing
factual knowledge in LLMs. We demonstrate the state-of-the-art performance of
SWEAOS on the COUNTERFACT and zsRE datasets. To further validate the reasoning
ability of SWEAOS in editing knowledge, we evaluate it on the more complex
RIPPLEEDITS benchmark. The results on two subdatasets demonstrate that our
SWEAOS possesses state-of-the-art reasoning ability.

我们提出了一种可扩展的主题词嵌入修改（SWEA）框架，它修改了主题的表示并在推理阶段实现了知识的编辑目标。我们进一步提出了优化然后抑制融合方法，用于编辑 LLMs 中的事实知识。我们在多个数据集上展示了 SWEAOS 的最先进推理能力。