State-of-the-art dialogue models still often stumble with regards to factual
accuracy and self-contradiction. Anecdotally, they have been observed to fail
to maintain character identity throughout discourse; and more specifically, may
take on the role of their interlocutor. In this work we formalize and quantify
this deficiency, and show experimentally through human evaluations that this is
indeed a problem. In contrast, we show that discriminative models trained
specifically to recognize who is speaking can perform well; and further, these
can be used as automated metrics. Finally, we evaluate a wide variety of
mitigation methods, including changes to model architecture, training protocol,
and decoding strategy. Our best models reduce mistaken identity issues by
nearly 65% according to human annotators, while simultaneously improving
engagingness. Despite these results, we find that maintaining character
identity still remains a challenging problem.

通过对话模型进行定量化和形式化，并且使用人类评估实验证明了角色身份维持的问题，提出了识别讲话者的判别模型，并评估了改变模型结构、训练协议和译码策略等多种缓解方法，发现最佳模型能够将身份混淆问题降低近 65% 同时提高参与度，但是维护角色身份仍然是一个具有挑战性的问题。