Over the last several years, end-to-end neural conversational agents have
vastly improved in their ability to carry a chit-chat conversation with humans.
However, these models are often trained on large datasets from the internet,
and as a result, may learn undesirable behaviors from this data, such as toxic
or otherwise harmful language. Researchers must thus wrestle with the issue of
how and when to release these models. In this paper, we survey the problem
landscape for safety for end-to-end conversational AI and discuss recent and
related work. We highlight tensions between values, potential positive impact
and potential harms, and provide a framework for making decisions about whether
and how to release these models, following the tenets of value-sensitive
design. We additionally provide a suite of tools to enable researchers to make
better-informed decisions about training and releasing end-to-end
conversational AI models.

本文讨论了在训练端到端对话 AI 模型时会遇到的关于安全性的困境。提供了一个基于价值观的设计框架，以帮助研究人员在决定何时和如何发布这些模型时做出更合理的决策，并提供了一套工具来帮助他们进行更明智的选择。