Prompting is a standard tool for customizing language-model chatbots,
enabling them to take on a specific "persona". An implicit assumption in the
use of prompts is that they will be stable, so the chatbot will continue to
generate text according to the stipulated persona for the duration of a
conversation. We propose a quantitative benchmark to test this assumption,
evaluating persona stability via self-chats between two personalized chatbots.
Testing popular models like LLaMA2-chat-70B, we reveal a significant persona
drift within eight rounds of conversations. An empirical and theoretical
analysis of this phenomenon suggests the transformer attention mechanism plays
a role, due to attention decay over long exchanges. To combat attention decay
and persona drift, we propose a lightweight method called split-softmax, which
compares favorably against two strong baselines.

通过自我会话测试，我们评估了个性化聊天机器人之间的身份稳定性，并揭示了频繁的身份转化现象。通过经验和理论分析，我们发现 Transformer 注意机制的衰退是造成这一现象的原因。为了解决注意力衰退和身份转化问题，我们提出了一种轻量级方法称为 split-softmax，并与两个强基线模型进行了比较。