The rapid advancement of language models (LMs) necessitates robust alignment with diverse user values. However, current preference optimization approaches often fail to capture the plurality of user opinions, instead reinforcing majority viewpoints and marginalizing minority perspectives. We introduce PERSONA, a reproducible test bed designed to evaluate and improve pluralistic alignment of LMs. We procedurally generate diverse user profiles from US census data, resulting in 1,586 synthetic personas with varied demographic and idiosyncratic attributes. We then generate a large-scale evaluation dataset containing 3,868 prompts and 317,200 feedback pairs obtained from our synthetic personas. Leveraging this dataset, we systematically evaluate LM capabilities in role-playing diverse users, verified through human judges, and the establishment of both a benchmark, PERSONA Bench, for pluralistic alignment approaches as well as an extensive dataset to create new and future benchmarks. The full dataset and benchmarks are available here: https://www.synthlabs.ai/research/persona.

本研究解决了当前语言模型在优化用户偏好时忽视多样性的问题，主要聚焦于用户意见的多元性。提出的PERSONA为评估和改进语言模型的多元对齐提供了一种新颖的方法，生成了1586个合成用户档案和大规模评估数据集。研究发现，PERSONA Bench可作为多元对齐方法的基准，从而推动语言模型的更广泛应用。

PERSONA：多元化对齐的可重复测试平台