Traditional dialog systems used in goal-oriented applications require a lot
of domain-specific handcrafting, which hinders scaling up to new domains.
End-to-end dialog systems, in which all components are trained from the dialogs
themselves, escape this limitation. But the encouraging success recently
obtained in chit-chat dialog may not carry over to goal-oriented settings. This
paper proposes a testbed to break down the strengths and shortcomings of
end-to-end dialog systems in goal-oriented applications. Set in the context of
restaurant reservation, our tasks require manipulating sentences and symbols,
so as to properly conduct conversations, issue API calls and use the outputs of
such calls. We show that an end-to-end dialog system based on Memory Networks
can reach promising, yet imperfect, performance and learn to perform
non-trivial operations. We confirm those results by comparing our system to a
hand-crafted slot-filling baseline on data from the second Dialog State
Tracking Challenge (Henderson et al., 2014a). We show similar result patterns
on data extracted from an online concierge service.

本文提出了一个测试平台，以研究在特定场景下 end-to-end 对话系统的性能，通过实验比较其与手工设计的系统的相似性及差异性。结果表明，end-to-end 系统基于记忆网络能够实现目标，但仍存在缺陷。