Sequence-to-sequence models have been applied to a wide variety of NLP tasks, but how to properly use them for dialogue state tracking has not been systematically investigated. In this paper, we study this problem from the perspectives of pre-training objectives as well as the formats of context representations. We demonstrate that the choice of pre-training objective makes a significant difference to the state tracking quality. In particular, we find that masked span prediction is more effective than auto-regressive language modeling. We also explore using Pegasus, a span prediction-based pre-training objective for text summarization, for the state tracking model. We found that pre-training for the seemingly distant summarization task works surprisingly well for dialogue state tracking. In addition, we found that while recurrent state context representation works also reasonably well, the model may have a hard time recovering from earlier mistakes. We conducted experiments on the MultiWOZ 2.1-2.4 data sets with consistent observations.

研究了如何正确使用序列到序列模型进行对话状态跟踪，从预训练目标和上下文表示格式的角度探讨了此问题，并发现遮掩跨度预测比自回归语言建模更有效。同时使用基于跨度预测的Pegasus预训练目标进行状态跟踪模型的预训练，并发现似乎相距很远的摘要任务预训练能够意外地为对话状态跟踪发挥良好作用。此外，我们在MultiWOZ 2.1-2.4，WOZ 2.0和DSTC2数据集上进行了实验证明了以上结论。

有效的序列到序列对话状态跟踪