Recent advances in interactive large language models like ChatGPT have revolutionized various domains; however, their behavior in natural and role-play conversation settings remains underexplored. In our study, we address this gap by deeply investigating how ChatGPT behaves during conversations in different settings by analyzing its interactions in both a normal way and a role-play setting. We introduce a novel dataset of broad range of human-AI conversations annotated with user motives and model naturalness to examine (i) how humans engage with the conversational AI model, and (ii) how natural are AI model responses. Our study highlights the diversity of user motives when interacting with ChatGPT and variable AI naturalness, showing not only the nuanced dynamics of natural conversations between humans and AI, but also providing new avenues for improving the effectiveness of human-AI communication.

通过分析ChatGPT在不同设定下的交互行为，我们深入研究了该模型在自然对话和角色扮演对话环境中的行为。我们引入了一个新的人工智能和人类对话的数据集，注释了用户动机和模型自然度，从而探究了人类如何与对话型人工智能模型进行交互以及人工智能模型的自然度。我们的研究突出了用户与ChatGPT进行交互时的动机多样性和人工智能模型的自然度差异，不仅展示了人与人工智能之间自然对话的微妙动态，也为提高人工智能与人类沟通的效果提供了新的途径。

ChatGPT角色扮演数据集：用户动机和模型自然度分析