Adaptations of Transformer models, such as BERT4Rec and SASRec, achieve
state-of-the-art performance in the sequential recommendation task according to
accuracy-based metrics, such as NDCG. These models treat items as tokens and
then utilise a score-and-rank approach (Top-K strategy), where the model first
computes item scores and then ranks them according to this score. While this
approach works well for accuracy-based metrics, it is hard to use it for
optimising more complex beyond-accuracy metrics such as diversity. Recently,
the GPTRec model, which uses a different Next-K strategy, has been proposed as
an alternative to the Top-K models. In contrast with traditional Top-K
recommendations, Next-K generates recommendations item-by-item and, therefore,
can account for complex item-to-item interdependencies important for the
beyond-accuracy measures. However, the original GPTRec paper focused only on
accuracy in experiments and needed to address how to optimise the model for
complex beyond-accuracy metrics. Indeed, training GPTRec for beyond-accuracy
goals is challenging because the interaction training data available for
training recommender systems typically needs to be aligned with beyond-accuracy
recommendation goals. To solve the misalignment problem, we train GPTRec using
a 2-stage approach: in the first stage, we use a teacher-student approach to
train GPTRec, mimicking the behaviour of traditional Top-K models; in the
second stage, we use Reinforcement Learning to align the model for
beyond-accuracy goals. In particular, we experiment with increasing
recommendation diversity and reducing popularity bias. Our experiments on two
datasets show that in 3 out of 4 cases, GPTRec's Next-K generation approach
offers a better tradeoff between accuracy and secondary metrics than classic
greedy re-ranking techniques.

Transformer 模型在推荐系统的顺序推荐任务中应用广泛，BERT4Rec、SASRec 等模型通过使用 Top-K 策略，在准确度等度量指标上取得了最先进的性能。然而，GPTRec 作为替代 Top-K 模型的一个新方法，可以考虑复杂的物品间相互依赖关系，从而适应更多样化的度量指标。本研究提出了一个二阶段的训练方法，通过强化学习来对 GPTRec 进行训练，从而解决了推荐系统训练数据与度量指标之间的不匹配问题。在两个数据集上的实验证明，GPTRec 的 Next-K 生成方法在准确度和次要度量指标之间可以达到更好的权衡。