This paper explores the effectiveness of using large language models (LLMs)
for personalized movie recommendations from users' perspectives in an online
field experiment. Our study involves a combination of between-subject prompt
and historic consumption assessments, along with within-subject recommendation
scenario evaluations. By examining conversation and survey response data from
160 active users, we find that LLMs offer strong recommendation explainability
but lack overall personalization, diversity, and user trust. Our results also
indicate that different personalized prompting techniques do not significantly
affect user-perceived recommendation quality, but the number of movies a user
has watched plays a more significant role. Furthermore, LLMs show a greater
ability to recommend lesser-known or niche movies. Through qualitative
analysis, we identify key conversational patterns linked to positive and
negative user interaction experiences and conclude that providing personal
context and examples is crucial for obtaining high-quality recommendations from
LLMs.

本文通过在线实验的方式，从用户的角度探讨了使用大型语言模型（LLMs）进行个性化电影推荐的有效性。研究结果显示，LLMs 提供较强的推荐解释能力，但缺乏整体个性化、多样性和用户信任。此外，不同的个性化提示技术对用户感知的推荐质量影响不大，而用户观看的电影数量起到了更重要的作用。与此同时，LLMs 展示了更好地推荐知名度较低或小众化电影的能力。通过定性分析，我们发现与用户交互体验中的积极和消极关联的关键对话模式，并得出结论：为了获得高质量的 LLMs 推荐，提供个人背景和示例至关重要。