Responding with multi-modal content has been recognized as an essential
capability for an intelligent conversational agent. In this paper, we introduce
the MMDialog dataset to better facilitate multi-modal conversation. MMDialog is
composed of a curated set of 1.08 million real-world dialogues with 1.53
million unique images across 4,184 topics. MMDialog has two main and unique
advantages. First, it is the largest multi-modal conversation dataset by the
number of dialogues by 88x. Second, it contains massive topics to generalize
the open-domain. To build engaging dialogue system with this dataset, we
propose and normalize two response producing tasks based on retrieval and
generative scenarios. In addition, we build two baselines for above tasks with
state-of-the-art techniques and report their experimental performance. We also
propose a novel evaluation metric MM-Relevance to measure the multi-modal
responses. Our dataset and scripts are available in
this https URL.

本文介绍了 MMDialog 数据集、提出并规范了两个基于检索和生成场景的反应生成任务，还构建了两个基线，并提出了一种新的评估指标 MM-Relevance，用于衡量多模态反应。