Persona-driven role-playing (PRP) aims to build AI characters that can
respond to user queries by faithfully sticking with all persona statements.
Unfortunately, existing faithfulness criteria for PRP are limited to
coarse-grained LLM-based scoring without a clear definition or formulation.
This paper presents a pioneering exploration to quantify PRP faithfulness as a
fine-grained and explainable criterion, which also serves as a reliable
reference for optimization. Our criterion first discriminates persona
statements into active and passive constraints by identifying the
query-statement relevance. Then, we incorporate all constraints following the
principle that the AI character's response should be (a) entailed by active
(relevant) constraints and (b) not contradicted by passive (irrelevant)
constraints. We translate this principle mathematically into a novel
Active-Passive-Constraint (APC) score, a constraint-wise sum of natural
language inference (NLI) scores weighted by relevance scores. In practice, we
build the APC scoring system by symbolically distilling small discriminators
from GPT-4 for efficiency. We validate the quality of the APC score against
human evaluation based on example personas with tens of statements, and the
results show a high correlation. We further leverage it as a reward system in
direct preference optimization (DPO) for better AI characters. Our experiments
offer a fine-grained and explainable comparison between existing PRP
techniques, revealing their advantages and limitations. We further find
APC-based DPO to be one of the most competitive techniques for sticking with
all constraints and can be well incorporated with other techniques. We then
extend the scale of the experiments to real persons with hundreds of statements
and reach a consistent conclusion.

该研究提出了一种量化 PRP 忠实度的创新方法，通过从 Active-Passive-Constraint（APC）得分入手，将所有约束合并成一种解释性评估标准。同时，在实验中验证了此评分系统的质量，并将其用于直接偏好优化（DPO）中，以获得更好的 AI 角色。结果发现，APC-DPO 是一种精确均衡所有约束的竞争性技术，可以与其他技术结合使用。

量化和优化基于人物的角色扮演中的全球忠实度

Quantifying and Optimizing Global Faithfulness in Persona-driven  Role-playing

Neuro-symbolic AI attempts to integrate neural and symbolic architectures in
a manner that addresses strengths and weaknesses of each, in a complementary
fashion, in order to support robust strong AI capable of reasoning, learning,
and cognitive modeling. In this paper we consider the intensional First Order
Logic (IFOL) as a symbolic architecture of modern robots, able to use natural
languages to communicate with humans and to reason about their own knowledge
with self-reference and abstraction language property. We intend to obtain the
grounding of robot's language by experience of how it uses its neuronal
architectures and hence by associating this experience with the mining (sense)
of non-defined language concepts (particulars/individuals and universals) in
PRP (Properties/Relations/propositions) theory of IFOL. We consider three
natural language levels: The syntax of particular natural language (Italian,
French, etc..), and two universal language properties: its semantic logic
structure (based on virtual predicates of FOL and logic connectives), and its
corresponding conceptual PRP structure which universally represents the
composite mining of FOL formulae grounded on the robot's neuro system.

本文考虑自然语言符号结构与基于神经元结构的机器人语言之间的联系，探讨如何利用自然语言语法及两个通用语言属性，即基于 FOL 虚拟谓词和逻辑连接符的语义逻辑结构和相应的概念 PRP 结构，以智能化方法把自然语言言语接地到基于 IFOL 的机器人语言之中。