As large language models (LLMs) improve in their capacity to serve as personal AI assistants, their ability to output uniquely tailored, personalized responses that align with the soft preferences of their users is essential for enhancing user satisfaction and retention. However, untrained lay users have poor prompt specification abilities and often struggle with conveying their latent preferences to AI assistants. To address this, we leverage activation steering to guide LLMs to align with interpretable preference dimensions during inference. In contrast to memory-based personalization methods that require longer user history, steering is extremely lightweight and can be easily controlled by the user via an linear strength factor. We embed steering into three different interactive chatbot interfaces and conduct a within-subjects user study (n=14) to investigate how end users prefer to personalize their conversations. The results demonstrate the effectiveness of preference-based steering for aligning real-world conversations with hidden user preferences, and highlight further insights on how diverse values around control, usability, and transparency lead users to prefer different interfaces.

本研究解决了用户在使用大型语言模型（LLM）时难以表达个人偏好的问题。通过利用激活引导的方法，研究提供了一种轻量级的个性化用户互动方式，使得用户能够更好地调整和控制与聊天机器人的对话。研究结果表明，基于偏好的引导可以有效匹配用户的隐性需求，并揭示了用户在控制、可用性和透明度方面的多样化价值观对接口偏好的影响。

可操控聊天机器人：基于偏好的激活引导个性化大语言模型