Large language models (LLMs) have had a huge impact on society due to their
impressive capabilities and vast knowledge of the world. Various applications
and tools have been created that allow users to interact with these models in a
black-box scenario. However, one limitation of this scenario is that users
cannot modify the internal knowledge of the model, and the only way to add or
modify internal knowledge is by explicitly mentioning it to the model during
the current interaction. This learning process is called in-context training,
and it refers to training that is confined to the user's current session or
context. In-context learning has significant applications, but also has
limitations that are seldom studied. In this paper, we present a study that
shows how the model can suffer from interference between information that
continually flows in the context, causing it to forget previously learned
knowledge, which can reduce the model's performance. Along with showing the
problem, we propose an evaluation benchmark based on the bAbI dataset.

大型语言模型对社会产生了巨大影响，但黑盒子情景下，用户只能在当前交互中明确提及来添加或修改内部知识，而不具备修改模型内部知识的能力。本文提出了一个研究，展示了模型在不同上下文中持续流动的信息之间可能出现干扰，从而忘记先前学到的知识，导致模型性能下降，并提出了一个基于 bAbI 数据集的评估基准。