Recent work in open-domain conversational agents has demonstrated that
significant improvements in model engagingness and humanness metrics can be
achieved via massive scaling in both pre-training data and model size
(Adiwardana et al., 2020; Roller et al., 2020). However, if we want to build
agents with human-like abilities, we must expand beyond handling just text. A
particularly important topic is the ability to see images and communicate about
what is perceived. With the goal of engaging humans in multi-modal dialogue, we
investigate combining components from state-of-the-art open-domain dialogue
agents with those from state-of-the-art vision models. We study incorporating
different image fusion schemes and domain-adaptive pre-training and fine-tuning
strategies, and show that our best resulting model outperforms strong existing
models in multi-modal dialogue while simultaneously performing as well as its
predecessor (text-only) BlenderBot (Roller et al., 2020) in text-based
conversation. We additionally investigate and incorporate safety components in
our final model, and show that such efforts do not diminish model performance
with respect to engagingness metrics.

本文旨在研究结合先进的开放域对话代理和视觉模型，以实现多模态对话的目标，研究不同的图像融合方案和域自适应预训练和微调策略，并展示了我们最好的模型不仅在多模态对话方面优于现有的强模型，而且在纯文本对话方面表现也和其前身 BlenderBot 一样好，同时我们还研究并整合了安全组件，展示这种努力不会削弱模型性能方面的吸引力度量。