We study multi-turn response generation for open-domain dialogues. The
existing state-of-the-art addresses the problem with deep neural architectures.
While these models improved response quality, their complexity also hinders the
application of the models in real systems. In this work, we pursue a model that
has a simple structure yet can effectively leverage conversation contexts for
response generation. To this end, we propose four auxiliary tasks including
word order recovery, utterance order recovery, masked word recovery, and masked
utterance recovery, and optimize the objectives of these tasks together with
maximizing the likelihood of generation. By this means, the auxiliary tasks
that relate to context understanding can guide the learning of the generation
model to achieve a better local optimum. Empirical studies with three
benchmarks indicate that our model can significantly outperform
state-of-the-art generation models in terms of response quality on both
automatic evaluation and human judgment, and at the same time enjoys a much
faster decoding process.

本研究探讨多轮开放域对话的响应生成，并通过辅助任务实现上下文理解，提高了生成模型的质量，同时具有更快的解码速度。

使用辅助任务训练简单有效的多轮对话生成模型

Learning a Simple and Effective Model for Multi-turn Response Generation  with Auxiliary Tasks

We study multi-turn response generation in chatbots where a response is
generated according to a conversation context. Existing work has modeled the
hierarchy of the context, but does not pay enough attention to the fact that
words and utterances in the context are differentially important. As a result,
they may lose important information in context and generate irrelevant
responses. We propose a hierarchical recurrent attention network (HRAN) to
model both aspects in a unified framework. In HRAN, a hierarchical attention
mechanism attends to important parts within and among utterances with word
level attention and utterance level attention respectively. With the word level
attention, hidden vectors of a word level encoder are synthesized as utterance
vectors and fed to an utterance level encoder to construct hidden
representations of the context. The hidden vectors of the context are then
processed by the utterance level attention and formed as context vectors for
decoding the response. Empirical studies on both automatic evaluation and human
judgment show that HRAN can significantly outperform state-of-the-art models
for multi-turn response generation.

研究聊天机器人中的多轮对话应答生成，提出一种基于分层循环注意力网络的方法，该方法使用单词级别和语句级别的注意机制来捕捉对话上下文中重要部分，能够明显优于现有的多轮应答生成模型。