Generative models for dialog systems have gained much interest because of the
recent success of RNN and Transformer based models in tasks like question
answering and summarization. Although the task of dialog response generation is
generally seen as a sequence-to-sequence (Seq2Seq) problem, researchers in the
past have found it challenging to train dialog systems using the standard
Seq2Seq models. Therefore, to help the model learn meaningful utterance and
conversation level features, Sordoni et al. (2015b); Serban et al. (2016)
proposed Hierarchical RNN architecture, which was later adopted by several
other RNN based dialog systems. With the transformer-based models dominating
the seq2seq problems lately, the natural question to ask is the applicability
of the notion of hierarchy in transformer based dialog systems. In this paper,
we propose a generalized framework for Hierarchical Transformer Encoders and
show how a standard transformer can be morphed into any hierarchical encoder,
including HRED and HIBERT like models, by using specially designed attention
masks and positional encodings. We demonstrate that Hierarchical Encoding helps
achieve better natural language understanding of the contexts in
transformer-based models for task-oriented dialog systems through a wide range
of experiments.

本文提出了一种广义的分层 Transformer 编码器框架，并通过一系列实验展示了分层编码如何帮助任务导向的对话系统中的 transformer-based 模型更好地理解上下文从而实现更好的自然语言理解。