As LLMs become capable of complex tasks, there is growing potential for personalized interactions tailored to the subtle and idiosyncratic preferences of the user. We present a public benchmark, PersonalLLM, focusing on adapting LLMs to provide maximal benefits for a particular user. Departing from existing alignment benchmarks that implicitly assume uniform preferences, we curate open-ended prompts paired with many high-quality answers over which users would be expected to display heterogeneous latent preferences. Instead of persona-prompting LLMs based on high-level attributes (e.g., user's race or response length), which yields homogeneous preferences relative to humans, we develop a method that can simulate a large user base with diverse preferences from a set of pre-trained reward models. Our dataset and generated personalities offer an innovative testbed for developing personalization algorithms that grapple with continual data sparsity--few relevant feedback from the particular user--by leveraging historical data from other (similar) users. We explore basic in-context learning and meta-learning baselines to illustrate the utility of PersonalLLM and highlight the need for future methodological development. Our dataset is available at https://huggingface.co/datasets/namkoong-lab/PersonalLLM

本研究解决了个性化LLM交互中缺乏对用户独特偏好的适应性的问题。通过构建PersonalLLM数据集，该研究提出了一种基于多样化激励模型的方法，以模拟用户的异质性偏好，推动个性化算法的发展。最重要的发现是，该方法能够有效利用相似用户的历史数据来应对数据稀疏性挑战，为个性化LLM提供了创新的测试平台。

个人化大型语言模型：根据个体偏好定制LLM