We propose MemoChat, a pipeline for refining instructions that enables large language models (LLMs) to effectively employ self-composed memos for maintaining consistent long-range open-domain conversations. We demonstrate a long-range open-domain conversation through iterative "memorization-retrieval-response" cycles. This requires us to carefully design tailored tuning instructions for each distinct stage. The instructions are reconstructed from a collection of public datasets to teach the LLMs to memorize and retrieve past dialogues with structured memos, leading to enhanced consistency when participating in future conversations. We invite experts to manually annotate a test set designed to evaluate the consistency of long-range conversations questions. Experiments on three testing scenarios involving both open-source and API-accessible chatbots at scale verify the efficacy of MemoChat, which outperforms strong baselines.

为了实现保持一致的长距离开放领域对话，我们提出了MemoChat，一种用于细化指令的流程，使得大规模的语言模型能够有效地使用自组成的备忘录。经过演示我们通过“记忆-检索-回应”循环实现的长距离开放领域对话，这要求我们为每个独特的阶段精心设计定制化的调优指令。这些指令是从一个公共数据集的收集中重建而来，用于教导语言模型去记忆并检索过去的对话，通过结构化的备忘录，提高在未来对话中的一致性。我们邀请专家对一个测试集进行手动注释，以评估长距离对话的一致性。在涵盖开源和可访问API聊天机器人的三个测试场景上的实验证明了MemoChat的有效性，它优于强基线模型。

MemoChat: 调整LLMs以利用备忘录进行一致的长距离开放领域对话