Multi-role dialogue understanding comprises a wide range of diverse tasks
such as question answering, act classification, dialogue summarization etc.
While dialogue corpora are abundantly available, labeled data, for specific
learning tasks, can be highly scarce and expensive. In this work, we
investigate dialogue context representation learning with various types
unsupervised pretraining tasks where the training objectives are given
naturally according to the nature of the utterance and the structure of the
multi-role conversation. Meanwhile, in order to locate essential information
for dialogue summarization/extraction, the pretraining process enables external
knowledge integration. The proposed fine-tuned pretraining mechanism is
comprehensively evaluated via three different dialogue datasets along with a
number of downstream dialogue-mining tasks. Result shows that the proposed
pretraining mechanism significantly contributes to all the downstream tasks
without discrimination to different encoders.

该研究基于各类无监督预训练任务探究对话上下文表示的学习，并通过精细调整的预训练机制，有效整合了外部知识，能够大幅提高对话提取 / 摘要等下游任务的效果。