Sequence to sequence models attempt to capture the correlation between all
the words in the input and output sequences. While this is quite useful for
machine translation where the correlation among the words is indeed quite
strong, it becomes problematic for conversation modelling where the correlation
is often at a much abstract level. In contrast, humans tend to focus on the
essential concepts discussed in the conversation context and generate responses
accordingly. In this paper, we attempt to mimic this response generating
mechanism by learning the essential concepts in the context and response in an
unsupervised manner. The proposed model, referred to as Mask \& Focus maps the
input context to a sequence of concepts which are then used to generate the
response concepts. Together, the context and the response concepts generate the
final response. In order to learn context concepts from the training data
automatically, we \emph{mask} words in the input and observe the effect of
masking on response generation. We train our model to learn those response
concepts that have high mutual information with respect to the context
concepts, thereby guiding the model to \emph{focus} on the context concepts.
Mask \& Focus achieves significant improvement over the existing baselines in
several established metrics for dialogues.

本文介绍一种通过无监督学习的方式，学习上下文中的关键概念并根据其生成回复，从而在对话建模中取得显著进展。运用的模型被称为 Mask&Focus，并通过对输入进行掩码和观察响应生成效果的方式实现对上下文概念的自动化学习，从而指导模型关注所述概念。该方法已在多个评测指标下实现比现有基线方法更好的效果。