Scene-aware dialog systems will be able to have conversations with users
about the objects and events around them. Progress on such systems can be made
by integrating state-of-the-art technologies from multiple research areas
including end-to-end dialog systems visual dialog, and video description. We
introduce the Audio Visual Scene Aware Dialog (AVSD) challenge and dataset. In
this challenge, which is one track of the 7th Dialog System Technology
Challenges (DSTC7) workshop1, the task is to build a system that generates
responses in a dialog about an input video

该研究论文介绍了一种基于多个研究领域的技术相互整合的新型场景感知对话系统，同时提出了基于音视频的场景感知对话系统 (AVSD) 挑战与数据集，该挑战要求参赛者构建一个能够对输入视频进行对话响应的系统。

DSTC7 的音视频场景感知对话（AVSD）竞赛

Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7

Traditional dialog systems used in goal-oriented applications require a lot
of domain-specific handcrafting, which hinders scaling up to new domains.
End-to-end dialog systems, in which all components are trained from the dialogs
themselves, escape this limitation. But the encouraging success recently
obtained in chit-chat dialog may not carry over to goal-oriented settings. This
paper proposes a testbed to break down the strengths and shortcomings of
end-to-end dialog systems in goal-oriented applications. Set in the context of
restaurant reservation, our tasks require manipulating sentences and symbols,
so as to properly conduct conversations, issue API calls and use the outputs of
such calls. We show that an end-to-end dialog system based on Memory Networks
can reach promising, yet imperfect, performance and learn to perform
non-trivial operations. We confirm those results by comparing our system to a
hand-crafted slot-filling baseline on data from the second Dialog State
Tracking Challenge (Henderson et al., 2014a). We show similar result patterns
on data extracted from an online concierge service.

本文提出了一个测试平台，以研究在特定场景下 end-to-end 对话系统的性能，通过实验比较其与手工设计的系统的相似性及差异性。结果表明，end-to-end 系统基于记忆网络能够实现目标，但仍存在缺陷。