Can Large Language Models substitute humans in making important decisions?
Recent research has unveiled the potential of LLMs to role-play assigned
personas, mimicking their knowledge and linguistic habits. However, imitative
decision-making requires a more nuanced understanding of personas. In this
paper, we benchmark the ability of LLMs in persona-driven decision-making.
Specifically, we investigate whether LLMs can predict characters' decisions
provided with the preceding stories in high-quality novels. Leveraging
character analyses written by literary experts, we construct a dataset
LIFECHOICE comprising 1,401 character decision points from 395 books. Then, we
conduct comprehensive experiments on LIFECHOICE, with various LLMs and methods
for LLM role-playing. The results demonstrate that state-of-the-art LLMs
exhibit promising capabilities in this task, yet there is substantial room for
improvement. Hence, we further propose the CHARMAP method, which achieves a
6.01% increase in accuracy via persona-based memory retrieval. We will make our
datasets and code publicly available.

大型语言模型在决策中是否可以替代人类是一个近期的研究课题。本研究中，我们通过使用高质量小说中的人物分析数据构建了 LIFECHOICE 数据集，并进行了多项实验，研究了 LLMs 在以人物为驱动的决策中的能力。结果表明，目前的 LLMs 在此任务中显示出有希望的能力，但仍有很大的改进空间。因此，我们进一步提出了基于人物记忆检索的 CHARMAP 方法，通过该方法可以获得 6.01％的准确率提升。我们将公开提供我们的数据集和代码。