Traditional recommender systems such as matrix factorization methods rely on
learning a shared dense embedding space to represent both items and user
preferences. Sequence models such as RNN, GRUs, and, recently, Transformers
have also excelled in the task of sequential recommendation. This task requires
understanding the sequential structure present in users' historical
interactions to predict the next item they may like. Building upon the success
of Large Language Models (LLMs) in a variety of tasks, researchers have
recently explored using LLMs that are pretrained on vast corpora of text for
sequential recommendation. To use LLMs in sequential recommendations, both the
history of user interactions and the model's prediction of the next item are
expressed in text form. We propose CALRec, a two-stage LLM finetuning framework
that finetunes a pretrained LLM in a two-tower fashion using a mixture of two
contrastive losses and a language modeling loss: the LLM is first finetuned on
a data mixture from multiple domains followed by another round of target domain
finetuning. Our model significantly outperforms many state-of-the-art baselines
(+37% in Recall@1 and +24% in NDCG@10) and systematic ablation studies reveal
that (i) both stages of finetuning are crucial, and, when combined, we achieve
improved performance, and (ii) contrastive alignment is effective among the
target domains explored in our experiments.

传统推荐系统如矩阵分解方法依赖于学习一个共享的稠密嵌入空间来表示物品和用户偏好。最近，序列模型如 RNN、GRU 和 Transformer 在序列推荐任务中也表现出色。研究人员最近开始探索使用在大量文本语料库上预训练的大型语言模型（LLMs）进行序列推荐。为了在序列推荐中使用 LLMs，用户交互的历史记录和模型预测的下一个物品都以文本形式表示。我们提出了 CALRec，一个两阶段的 LLM 微调框架，它使用两种对比损失和语言建模损失在两个塔式结构中微调预训练的 LLM：首先在多个领域的数据混合上微调 LLM，然后进行目标领域的微调。我们的模型显著优于许多最先进的对比方法（在 Recall@1 上增长 37%，在 NDCG@10 上增长 24%），系统的消融研究揭示了（i）两个微调阶段都是至关重要的，在结合起来时我们可以获得更好的性能，以及（ii）目标领域中的对比对齐在我们的实验中是有效的。