This study presents a targeted model editing analysis focused on the latest
large language model, Llama-3. We explore the efficacy of popular model editing
techniques - ROME, MEMIT, and EMMET, which are designed for precise layer
interventions. We identify the most effective layers for targeted edits through
an evaluation that encompasses up to 4096 edits across three distinct
strategies: sequential editing, batch editing, and a hybrid approach we call as
sequential-batch editing. Our findings indicate that increasing edit
batch-sizes may degrade model performance more significantly than using smaller
edit batches sequentially for equal number of edits. With this, we argue that
sequential model editing is an important component for scaling model editing
methods and future research should focus on methods that combine both batched
and sequential editing. This observation suggests a potential limitation in
current model editing methods which push towards bigger edit batch sizes, and
we hope it paves way for future investigations into optimizing batch sizes and
model editing performance.

本研究重点针对最新的大型语言模型 Llama-3 进行了定向模型编辑分析。我们探讨了针对精确层次干预设计的流行模型编辑技术 ——ROME、MEMIT 和 EMMET 的有效性。通过对三种不同策略（顺序编辑、批次编辑和顺序 - 批次编辑）的 4096 次编辑进行评估，我们确定了最有效的编辑层次。我们的发现表明，增加编辑批次大小可能会比使用较小的编辑批次按顺序执行相同数量的编辑更明显地降低模型性能。基于此，我们认为顺序模型编辑在扩展模型编辑方法方面是一个重要组成部分，并且未来的研究应该专注于将批量编辑和顺序编辑相结合的方法。这一观察表明了当前模型编辑方法在推动更大的编辑批次大小方面存在潜在限制，我们希望这为优化批次大小和模型编辑性能的未来研究铺平道路。