Traditionally, industry solutions for building a task-oriented dialog system
have relied on helping dialog authors define rule-based dialog managers,
represented as dialog flows. While dialog flows are intuitively interpretable
and good for simple scenarios, they fall short of performance in terms of the
flexibility needed to handle complex dialogs. On the other hand, purely
machine-learned models can handle complex dialogs, but they are considered to
be black boxes and require large amounts of training data. In this
demonstration, we showcase Conversation Learner, a machine teaching tool for
building dialog managers. It combines the best of both approaches by enabling
dialog authors to create a dialog flow using familiar tools, converting the
dialog flow into a parametric model (e.g., neural networks), and allowing
dialog authors to improve the dialog manager (i.e., the parametric model) over
time by leveraging user-system dialog logs as training data through a machine
teaching interface.

该研究论文提出了 Conversation Learner，一种结合了规则导向对话管理和基于机器学习的参数模型的机器教学工具，可以通过利用用户和系统之间的对话日志作为训练数据，帮助对话作者构建灵活应对复杂对话的对话管理器。

对话学习器 -- 用于构建任务导向对话系统的对话管理器的机器教学工具

Conversation Learner -- A Machine Teaching Tool for Building Dialog  Managers for Task-Oriented Dialog Systems

Today, most large-scale conversational AI agents (e.g. Alexa, Siri, or Google
Assistant) are built using manually annotated data to train the different
components of the system. Typically, the accuracy of the ML models in these
components are improved by manually transcribing and annotating data. As the
scope of these systems increase to cover more scenarios and domains, manual
annotation to improve the accuracy of these components becomes prohibitively
costly and time consuming. In this paper, we propose a system that leverages
user-system interaction feedback signals to automate learning without any
manual annotation. Users here tend to modify a previous query in hopes of
fixing an error in the previous turn to get the right results. These
reformulations, which are often preceded by defective experiences caused by
errors in ASR, NLU, ER or the application. In some cases, users may not
properly formulate their requests (e.g. providing partial title of a song), but
gleaning across a wider pool of users and sessions reveals the underlying
recurrent patterns. Our proposed self-learning system automatically detects the
errors, generate reformulations and deploys fixes to the runtime system to
correct different types of errors occurring in different components of the
system. In particular, we propose leveraging an absorbing Markov Chain model as
a collaborative filtering mechanism in a novel attempt to mine these patterns.
We show that our approach is highly scalable, and able to learn reformulations
that reduce Alexa-user errors by pooling anonymized data across millions of
customers. The proposed self-learning system achieves a win/loss ratio of 11.8
and effectively reduces the defect rate by more than 30% on utterance level
reformulations in our production A/B tests. To the best of our knowledge, this
is the first self-learning large-scale conversational AI system in production.

本研究提出了一种自学习系统，使用用户 - 系统交互反馈信号自动纠正对话式人工智能系统中出现的各种组件错误，并通过吸收马尔可夫链模型挖掘这些反馈信号中的共同模式，进行可扩展的改进，实现了大规模对话式人工智能系统的自主学习，能显著降低过多次错误和缺陷。