Most open-domain dialogue systems suffer from forgetting important
information, especially in a long-term conversation. Existing works usually
train the specific retriever or summarizer to obtain key information from the
past, which is time-consuming and highly depends on the quality of labeled
data. To alleviate this problem, we propose to recursively generate summaries/
memory using large language models (LLMs) to enhance long-term memory ability.
Specifically, our method first stimulates LLMs to memorize small dialogue
contexts and then recursively produce new memory using previous memory and
following contexts. Finally, the LLM can easily generate a highly consistent
response with the help of the latest memory. We evaluate our method using
ChatGPT and text-davinci-003, and the experiments on the widely-used public
dataset show that our method can generate more consistent responses in a
long-context conversation. Notably, our method is a potential solution to
enable the LLM to model the extremely long context. Code and scripts will be
released later.

使用大型语言模型递归生成摘要 / 记忆，从而提高长期记忆能力，进而解决开放领域对话系统中遗忘重要信息的问题。实验证明，该方法可以在长对话环境中生成更加一致的回应。

在大型语言模型中，递归总结实现了长期的对话记忆

Recursively Summarizing Enables Long-Term Dialogue Memory in Large  Language Models

Dialogue systems without consistent responses are not fascinating. In this
study, we build a dialogue system that can respond based on a given character
setting (persona) to bring consistency. Considering the trend of the rapidly
increasing scale of language models, we propose an approach that uses
prompt-tuning, which has low learning costs, on pre-trained large-scale
language models. The results of automatic and manual evaluations in English and
Japanese show that it is possible to build a dialogue system with more natural
and personalized responses using less computational resources than fine-tuning.

本研究通过使用 persona 来建立具有一致性反应的对话系统，提出了一种基于预训练的大型语言模型的 prompt 调节方法，在英语和日语方面的自动和手动评估结果表明，这种方法可以建立具有更自然和个性化反应的对话系统，而且计算资源较少。

使用 Prompt-Tuning 构建个性化对话系统

Building a Personalized Dialogue System with Prompt-Tuning

Generating responses that are consistent with the dialogue context is one of
the central challenges in building engaging conversational agents. We
demonstrate that neural conversation models can be geared towards generating
consistent responses by maintaining certain features related to topics and
personas throughout the conversation. Past work has required external
supervision that exploits features such as user identities that are often
unavailable. In our approach, topic and persona feature extractors are trained
using a contrastive training scheme that utilizes the natural structure of
dialogue data. We further adopt a feature disentangling loss which, paired with
controllable response generation techniques, allows us to promote or demote
certain learned topics and persona features. Evaluation results demonstrate the
model's ability to capture meaningful topics and persona features. The
incorporation of the learned features brings significant improvement in terms
of the quality of generated responses on two dialogue datasets.

该研究使用神经会话模型和特征提取器生成与对话上下文一致的响应，并通过对话数据上的对比训练方案和特征分离损失增强特定的话题和角色特征以提高响应质量。