As large language models (LLMs) become increasingly integrated into daily
applications, it is essential to ensure they operate fairly across diverse user
demographics. In this work, we show that LLMs suffer from personalization bias,
where their performance is impacted when they are personalized to a user's
identity. We quantify personalization bias by evaluating the performance of
LLMs along two axes - safety and utility. We measure safety by examining how
benign LLM responses are to unsafe prompts with and without personalization. We
measure utility by evaluating the LLM's performance on various tasks, including
general knowledge, mathematical abilities, programming, and reasoning skills.
We find that various LLMs, ranging from open-source models like Llama (Touvron
et al., 2023) and Mistral (Jiang et al., 2023) to API-based ones like GPT-3.5
and GPT-4o (Ouyang et al., 2022), exhibit significant variance in performance
in terms of safety-utility trade-offs depending on the user's identity.
Finally, we discuss several strategies to mitigate personalization bias using
preference tuning and prompt-based defenses.

大型语言模型（LLMs）在日常应用中变得越来越普遍，因此需要确保它们在各种用户群体之间公正地运行。本文揭示了 LLMs 存在个性化偏差的问题，也就是当 LLMs 根据用户的身份进行个性化设置时，它们的性能会受到影响。我们通过评估 LLMs 在安全性和效用两个维度上的表现来量化个性化偏差。结论发现，不同的 LLMs 在安全性和效用的权衡方面存在显著的性能差异，这取决于用户的身份。最后，我们探讨了一些使用偏好调优和基于提示的防御策略来减轻个性化偏差的方法。