User Satisfaction Estimation (USE) is an important yet challenging task in
goal-oriented conversational systems. Whether the user is satisfied with the
system largely depends on the fulfillment of the user's needs, which can be
implicitly reflected by users' dialogue acts. However, existing studies often
neglect the sequential transitions of dialogue act or rely heavily on annotated
dialogue act labels when utilizing dialogue acts to facilitate USE. In this
paper, we propose a novel framework, namely USDA, to incorporate the sequential
dynamics of dialogue acts for predicting user satisfaction, by jointly learning
User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific,
we first employ a Hierarchical Transformer to encode the whole dialogue
context, with two task-adaptive pre-training strategies to be a second-phase
in-domain pre-training for enhancing the dialogue modeling ability. In terms of
the availability of dialogue act labels, we further develop two variants of
USDA to capture the dialogue act information in either supervised or
unsupervised manners. Finally, USDA leverages the sequential transitions of
both content and act features in the dialogue to predict the user satisfaction.
Experimental results on four benchmark goal-oriented dialogue datasets across
different applications show that the proposed method substantially and
consistently outperforms existing methods on USE, and validate the important
role of dialogue act sequences in USE.

本文提出了一种新的框架 USDA，采用分层 Transformer 编码整个对话上下文，并发展了两种 USDA 变体，以捕捉有监督或无监督方式的对话行为信息，在预测用户满意度时，结合了内容和行为特征的时序转换。在四个基准目标导向的对话数据集上的实验证明，所提出的方法在用户满意度估计方面显著且一致地优于现有方法，并验证了对话行为序列在 USE 中的重要作用。

基于序列化对话行为建模的目标导向会话系统用户满意度评估

User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems

We propose a novel methodology to address dialog learning in the context of
goal-oriented conversational systems. The key idea is to quantize the dialog
space into clusters and create a language model across the clusters, thus
allowing for an accurate choice of the next utterance in the conversation. The
language model relies on n-grams associated with clusters of utterances. This
quantized-dialog language model methodology has been applied to the end-to-end
goal-oriented track of the latest Dialog System Technology Challenges (DSTC6).
The objective is to find the correct system utterance from a pool of candidates
in order to complete a dialog between a user and an automated
restaurant-reservation system. Our results show that the technique proposed in
this paper achieves high accuracy regarding selection of the correct candidate
utterance, and outperforms other state-of-the-art approaches based on neural
networks.

使用基于簇的语言模型方法解决面向目标对话系统中的对话学习问题，在 DSTC6 挑战赛中应用于餐厅预订系统；结果表明，该技术在选择正确候选话语方面达到了高准确性，并且优于基于神经网络的其他现有方法。