This study explores linguistic differences between human and LLM-generated
dialogues, using 19.5K dialogues generated by ChatGPT-3.5 as a companion to the
EmpathicDialogues dataset. The research employs Linguistic Inquiry and Word
Count (LIWC) analysis, comparing ChatGPT-generated conversations with human
conversations across 118 linguistic categories. Results show greater
variability and authenticity in human dialogues, but ChatGPT excels in
categories such as social processes, analytical style, cognition, attentional
focus, and positive emotional tone, reinforcing recent findings of LLMs being
"more human than human." However, no significant difference was found in
positive or negative affect between ChatGPT and human dialogues. Classifier
analysis of dialogue embeddings indicates implicit coding of the valence of
affect despite no explicit mention of affect in the conversations. The research
also contributes a novel, companion ChatGPT-generated dataset of conversations
between two independent chatbots, which were designed to replicate a corpus of
human conversations available for open access and used widely in AI research on
language modeling. Our findings increase understanding of ChatGPT's linguistic
capabilities and inform ongoing efforts to distinguish between human and
LLM-generated text, which is critical in detecting AI-generated fakes,
misinformation, and disinformation.

研究探讨人类和 LLM 生成的对话之间的语言差异，并使用 ChatGPT-3.5 生成的 19.5K 对话作为 EmpathicDialogues 数据集的补充。研究使用了语言查询和词频统计（LIWC）分析，在 118 个语言类别上比较 ChatGPT 生成的对话和人类对话。结果显示人类对话在可变性和真实性方面更出色，但 ChatGPT 在社交过程、分析风格、认知、注意力焦点和积极情绪方面表现优异，加强了 LLM “更接近人类” 的最新发现。然而，在正面或负面情感方面，ChatGPT 和人类对话之间没有显著差异。对话嵌入的分类器分析表明，尽管对话中未明确提及情感，但存在着情感价值的隐式编码。该研究还提供了一个新颖的 ChatGPT 生成的对话数据集，其中包含两个独立的聊天机器人之间的对话，这些对话旨在复制一个开放可访问的人类对话语料库，广泛应用于语言建模的 AI 研究。我们的研究结果增加了对 ChatGPT 的语言能力的理解，并为区分人类和 LLM 生成的文本的持续努力提供了信息，这对于检测由 AI 生成的虚假信息、错误信息和误导信息至关重要。