Personalized chatbots focus on endowing the chatbots with a consistent
personality to behave like real users and further act as personal assistants.
Previous studies have explored generating implicit user profiles from the
user's dialogue history for building personalized chatbots. However, these
studies only use the response generation loss to train the entire model, thus
it is prone to suffer from the problem of data sparsity. Besides, they
overemphasize the final generated response's quality while ignoring the
correlations and fusions between the user's dialogue history, leading to rough
data representations and performance degradation. To tackle these problems, we
propose a self-supervised learning framework MCP for capturing better
representations from users' dialogue history for personalized chatbots.
Specifically, we apply contrastive sampling methods to leverage the supervised
signals hidden in user dialog history, and generate the pre-training samples
for enhancing the model. We design three pre-training tasks based on three
types of contrastive pairs from user dialogue history, namely response pairs,
sequence augmentation pairs, and user pairs. We pre-train the utterance encoder
and the history encoder towards the contrastive objectives and use these
pre-trained encoders for generating user profiles while personalized response
generation. Experimental results on two real-world datasets show a significant
improvement in our proposed model MCP compared with the existing methods.

该研究提出了一种自主学习框架 MCP，它使用对比采样方法从用户对话历史中获取更好的表征，包括回应对、序列扩增对和用户对，它有效提高了个性化聊天机器人生成用户概要和相应的表现。

MCP：基于多级对比采样的个性化聊天机器人自学习预训练

MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling

Personalized chatbots focus on endowing chatbots with a consistent
personality to behave like real users, give more informative responses, and
further act as personal assistants. Existing personalized approaches tried to
incorporate several text descriptions as explicit user profiles. However, the
acquisition of such explicit profiles is expensive and time-consuming, thus
being impractical for large-scale real-world applications. Moreover, the
restricted predefined profile neglects the language behavior of a real user and
cannot be automatically updated together with the change of user interests. In
this paper, we propose to learn implicit user profiles automatically from
large-scale user dialogue history for building personalized chatbots.
Specifically, leveraging the benefits of Transformer on language understanding,
we train a personalized language model to construct a general user profile from
the user's historical responses. To highlight the relevant historical responses
to the input post, we further establish a key-value memory network of
historical post-response pairs, and build a dynamic post-aware user profile.
The dynamic profile mainly describes what and how the user has responded to
similar posts in history. To explicitly utilize users' frequently used words,
we design a personalized decoder to fuse two decoding strategies, including
generating a word from the generic vocabulary and copying one word from the
user's personalized vocabulary. Experiments on two real-world datasets show the
significant improvement of our model compared with existing methods. Our code
is available at this https URL

本文提出了一种从用户对话历史中自动学习隐式用户配置文件以构建个性化聊天机器人的方法，采用 Transformer 进行语言理解训练，并通过建立历史帖子 - 响应对的键值存储网络来建立动态的贴子感知用户配置文件。同时，采用个性化解码器融合两种解码策略，包括生成通用词汇表中的单词和从用户个性化词汇表中复制一个单词。实验结果表明，该模型较现有方法具有显著的改进。