Role-playing agents (RPA) have been a popular application area for large
language models (LLMs), attracting significant interest from both industry and
academia.While existing RPAs well portray the characters' knowledge and tones,
they face challenges in capturing their minds, especially for small
role-playing language models (RPLMs). In this paper, we propose to enhance
RPLMs via personality-indicative data. Specifically, we leverage questions from
psychological scales and distill advanced RPAs to generate dialogues that grasp
the minds of characters. Experimental results validate that RPLMs trained with
our dataset exhibit advanced role-playing capabilities for both general and
personality-related evaluations. Code and data are available at
\href{this https URL}{this URL}.

我们通过应用特征性数据提升小型角色扮演语言模型，实验证明使用我们的数据集训练的模型在一般和与个性相关的评估中展现了高级角色扮演能力。

捕捉思维，而非语言：使用个性指示数据增强角色扮演语言模型

Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models  with Personality-Indicative Data

While Large Language Models (LLMs) can serve as agents to simulate human
behaviors (i.e., role-playing agents), we emphasize the importance of
point-in-time role-playing. This situates characters at specific moments in the
narrative progression for three main reasons: (i) enhancing users' narrative
immersion, (ii) avoiding spoilers, and (iii) fostering engagement in fandom
role-playing. To accurately represent characters at specific time points,
agents must avoid character hallucination, where they display knowledge that
contradicts their characters' identities and historical timelines. We introduce
TimeChara, a new benchmark designed to evaluate point-in-time character
hallucination in role-playing LLMs. Comprising 10,895 instances generated
through an automated pipeline, this benchmark reveals significant hallucination
issues in current state-of-the-art LLMs (e.g., GPT-4o). To counter this
challenge, we propose Narrative-Experts, a method that decomposes the reasoning
steps and utilizes narrative experts to reduce point-in-time character
hallucinations effectively. Still, our findings with TimeChara highlight the
ongoing challenges of point-in-time character hallucination, calling for
further study.

使用大型语言模型（LLMs）进行角色扮演可以增强用户的叙事沉浸感，避免剧透，并促进粉丝的参与，在此基础上，引入 TimeChara 作为评估角色幻觉问题的新基准，并提出 Narrative-Experts 方法来有效减少角色幻觉问题，但该研究发现角色幻觉问题仍然存在挑战，需要进一步研究。

TimeChara：评估角色扮演大型语言模型的时点角色幻象

TimeChara: Evaluating Point-in-Time Character Hallucination of  Role-Playing Large Language Models

Large language models (LLMs) have demonstrated impressive performance and
spurred numerous AI applications, in which role-playing agents (RPAs) are
particularly popular, especially for fictional characters. The prerequisite for
these RPAs lies in the capability of LLMs to understand characters from
fictional works. Previous efforts have evaluated this capability via basic
classification tasks or characteristic imitation, failing to capture the
nuanced character understanding with LLMs. In this paper, we propose evaluating
LLMs' character understanding capability via the character profiling task,
i.e., summarizing character profiles from corresponding materials, a widely
adopted yet understudied practice for RPA development. Specifically, we
construct the CroSS dataset from literature experts and assess the generated
profiles by comparing ground truth references and their applicability in
downstream tasks. Our experiments, which cover various summarization methods
and LLMs, have yielded promising results. These results strongly validate the
character understanding capability of LLMs. We believe our constructed resource
will promote further research in this field. Resources are available at
this https URL

我们提出使用角色概况任务来评估大型语言模型（LLMs）的角色理解能力，通过从对应的材料中总结角色概况，构建 CroSS 数据集并比较与下游任务的适用性，我们的实验结果强有力地验证了 LLMs 的角色理解能力，并且我们相信我们构建的资源将促进该领域的进一步研究。