Building an intelligent dialogue system with the ability to select a proper
response according to a multi-turn context is a great challenging task.
Existing studies focus on building a context-response matching model with
various neural architectures or PLMs and typically learning with a single
response prediction task. These approaches overlook many potential training
signals contained in dialogue data, which might be beneficial for context
understanding and produce better features for response prediction. Besides, the
response retrieved from existing dialogue systems supervised by the
conventional way still faces some critical challenges, including incoherence
and inconsistency. To address these issues, in this paper, we propose learning
a context-response matching model with auxiliary self-supervised tasks designed
for the dialogue data based on pre-trained language models. Specifically, we
introduce four self-supervised tasks including next session prediction,
utterance restoration, incoherence detection and consistency discrimination,
and jointly train the PLM-based response selection model with these auxiliary
tasks in a multi-task manner. By this means, the auxiliary tasks can guide the
learning of the matching model to achieve a better local optimum and select a
more proper response. Experiment results on two benchmarks indicate that the
proposed auxiliary self-supervised tasks bring significant improvement for
multi-turn response selection in retrieval-based dialogues, and our model
achieves new state-of-the-art results on both datasets.

本研究提出了一种基于预训练语言模型的上下文相应匹配模型，引入了四个自我监督任务，并与 auxiliary tasks 一起多任务训练 PLM-based 响应选择模型，旨在提高多轮响应选择的准确性。实验结果表明，在检索式对话中，引入辅助自我监督任务对多轮响应选择带来了显著的改进，本模型在两个数据集上都达到了最新的最优结果。