Pre-trained conversation models (PCMs) have achieved promising progress in
recent years. However, existing PCMs for Task-oriented dialog (TOD) are
insufficient for capturing the sequential nature of the TOD-related tasks, as
well as for learning dialog policy information. To alleviate these problems,
this paper proposes a task-progressive PCM with two policy-aware pre-training
tasks. The model is pre-trained through three stages where TOD-related tasks
are progressively employed according to the task logic of the TOD system. A
global policy consistency task is designed to capture the multi-turn dialog
policy sequential relation, and an act-based contrastive learning task is
designed to capture similarities among samples with the same dialog policy. Our
model achieves better results on both MultiWOZ and In-Car end-to-end dialog
modeling benchmarks with only 18\% parameters and 25\% pre-training data
compared to the previous state-of-the-art PCM, GALAXY.

本论文提出了一种任务递进的预训练对话模型，通过两个具有策略感知的预训练任务来解决任务导向型对话模型中的时序性问题和学习对话策略信息的不足。该模型通过三个阶段的逐步任务处理来进行预训练，其中设计了一个全局策略一致性任务用于捕捉多轮对话策略的时序关系，并设计了一个基于行为的对比学习任务用于捕捉相同对话策略样本的相似性。与之前最先进的 PCM（GALAXY）相比，我们的模型仅使用 18％的参数和 25％的预训练数据，在 MultiWOZ 和 In-Car 的端到端对话建模基准上取得了更好的结果。