Large Language Models (LLMs) have recently emerged as promising tools for recommendation thanks to their advanced textual understanding ability and context-awareness. Despite the current practice of training and evaluating LLM-based recommendation (LLM4Rec) models under a sequential recommendation scenario, we found that whether these models understand the sequential information inherent in users' item interaction sequences has been largely overlooked. In this paper, we first demonstrate through a series of experiments that existing LLM4Rec models do not fully capture sequential information both during training and inference. Then, we propose a simple yet effective LLM-based sequential recommender, called LLM-SRec, a method that enhances the integration of sequential information into LLMs by distilling the user representations extracted from a pre-trained CF-SRec model into LLMs. Our extensive experiments show that LLM-SRec enhances LLMs' ability to understand users' item interaction sequences, ultimately leading to improved recommendation performance. Furthermore, unlike existing LLM4Rec models that require fine-tuning of LLMs, LLM-SRec achieves state-of-the-art performance by training only a few lightweight MLPs, highlighting its practicality in real-world applications. Our code is available at https://github.com/Sein-Kim/LLM-SRec.

本研究探讨了大型语言模型（LLMs）在序列推荐中的不足，发现现有的LLM4Rec模型未能充分捕捉用户交互序列中的序列信息。通过提出一种新方法LLM-SRec，该方法通过提炼用户表示来增强LLM对序列信息的整合，最终在推荐性能上实现了显著提升，且只需训练少量轻量级的多层感知器，突显了其在实际应用中的可行性。

迷失在序列中：大型语言模型理解序列推荐吗？