We present a data-driven, end-to-end approach to transaction-based dialog
systems that performs at near-human levels in terms of verbal response quality
and factual grounding accuracy. We show that two essential components of the
system produce these results: a sufficiently large and diverse, in-domain
labeled dataset, and a neural network-based, pre-trained model that generates
both verbal responses and API call predictions. In terms of data, we introduce
TicketTalk, a movie ticketing dialog dataset with 23,789 annotated
conversations. The movie ticketing conversations range from completely
open-ended and unrestricted to more structured, both in terms of their
knowledge base, discourse features, and number of turns. In qualitative human
evaluations, model-generated responses trained on just 10,000 TicketTalk
dialogs were rated to "make sense" 86.5 percent of the time, almost the same as
human responses in the same contexts. Our simple, API-focused annotation schema
results in a much easier labeling task making it faster and more cost
effective. It is also the key component for being able to predict API calls
accurately. We handle factual grounding by incorporating API calls in the
training data, allowing our model to learn which actions to take and when.
Trained on the same 10,000-dialog set, the model's API call predictions were
rated to be correct 93.9 percent of the time in our evaluations, surpassing the
ratings for the corresponding human labels. We show how API prediction and
response generation scores improve as the dataset size incrementally increases
from 5000 to 21,000 dialogs. Our analysis also clearly illustrates the benefits
of pre-training. We are publicly releasing the TicketTalk dataset with this
paper to facilitate future work on transaction-based dialogs.

研究通过数据驱动和端到端的方法，使用 TicketTalk 数据集和基于神经网络的预训练模型构建了一个可以实现近乎人类级别回应质量和 API 调用准确性的交易对话系统，并通过预训练提高了系统的 API 预测和响应生成得分.