With increasing demand for and adoption of virtual assistants, recent work
has investigated ways to accelerate bot schema design through the automatic
induction of intents or the induction of slots and dialogue states. However, a
lack of dedicated benchmarks and standardized evaluation has made progress
difficult to track and comparisons between systems difficult to make. This
challenge track, held as part of the Eleventh Dialog Systems Technology
Challenge, introduces a benchmark that aims to evaluate methods for the
automatic induction of customer intents in a realistic setting of customer
service interactions between human agents and customers. We propose two
subtasks for progressively tackling the automatic induction of intents and
corresponding evaluation methodologies. We then present three datasets suitable
for evaluating the tasks and propose simple baselines. Finally, we summarize
the submissions and results of the challenge track, for which we received
submissions from 34 teams.

本文提出一个针对客户服务交互的自动感知意图的基准测试，探讨了虚拟助手、bot 架构设计、意图识别、客户服务交互和评估方法方面的研究成果，并在挑战中收到了来自 34 个团队的提交和结果。