The majority of conversations a dialogue agent sees over its lifetime occur
after it has already been trained and deployed, leaving a vast store of
potential training signal untapped. In this work, we propose the self-feeding
chatbot, a dialogue agent with the ability to extract new training examples
from the conversations it participates in. As our agent engages in
conversation, it also estimates user satisfaction in its responses. When the
conversation appears to be going well, the user's responses become new training
examples to imitate. When the agent believes it has made a mistake, it asks for
feedback; learning to predict the feedback that will be given improves the
chatbot's dialogue abilities further. On the PersonaChat chit-chat dataset with
over 131k training examples, we find that learning from dialogue with a
self-feeding chatbot significantly improves performance, regardless of the
amount of traditional supervision.

本研究提出自我反馈聊天机器人，通过从参与的对话中提取新的训练样本和估计用户满意度来改进聊天机器人的对话能力，并在 PersonaChat chit-chat 数据集上进行实验得到了显著的性能提升。