Although many pretrained models exist for text or images, there have been
relatively fewer attempts to train representations specifically for dialog
understanding. Prior works usually relied on finetuned representations based on
generic text representation models like BERT or GPT-2. But such language
modeling pretraining objectives do not take the structural information of
conversational text into consideration. Although generative dialog models can
learn structural features too, we argue that the structure-unaware word-by-word
generation is not suitable for effective conversation modeling. We empirically
demonstrate that such representations do not perform consistently across
various dialog understanding tasks. Hence, we propose a structure-aware Mutual
Information based loss-function DMI (Discourse Mutual Information) for training
dialog-representation models, that additionally captures the inherent
uncertainty in response prediction. Extensive evaluation on nine diverse dialog
modeling tasks shows that our proposed DMI-based models outperform strong
baselines by significant margins.

为了实现有效的会话建模，本文提出了基于结构的相互信息损失函数 DMI 来训练对话表示模型，并在九个不同对话建模任务上展开了广泛评估，结果表明我们提出的 DMI-based 模型表现超过了强基线。