Recent studies suggest that the existing neural models have difficulty
handling repeated items in sequential recommendation tasks. However, our
understanding of this difficulty is still limited. In this study, we
substantially advance this field by identifying a major source of the problem:
the single hidden state embedding and static item embeddings in the output
softmax layer. Specifically, the similarity structure of the global item
embeddings in the softmax layer sometimes forces the single hidden state
embedding to be close to new items when copying is a better choice, while
sometimes forcing the hidden state to be close to the items from the input
inappropriately. To alleviate the problem, we adapt the recently-proposed
softmax alternatives such as softmax-CPR to sequential recommendation tasks and
demonstrate that the new softmax architectures unleash the capability of the
neural encoder on learning when to copy and when to exclude the items from the
input sequence. By only making some simple modifications on the output softmax
layer for SASRec and GRU4Rec, softmax-CPR achieves consistent improvement in 12
datasets. With almost the same model size, our best method not only improves
the average NDCG@10 of GRU4Rec in 5 datasets with duplicated items by 10%
(4%-17% individually) but also improves 7 datasets without duplicated items by
24% (8%-39%)!

最近的研究表明，现有的神经模型在处理顺序推荐任务中的重复项目时存在困难。然而，我们对这个困难的理解仍然有限。通过识别问题的主要来源 —— 输出 softmax 层中的单一隐藏状态嵌入和静态项目嵌入的相似性结构，本研究在该领域取得了显著的进展。具体地说，当复制是更好的选择时，全局项目嵌入在 softmax 层中的相似性结构有时会导致单一隐藏状态嵌入接近新项目，而有时会不适当地导致隐藏状态接近输入中的项目。为了缓解这个问题，我们将最近提出的 softmax 替代方法（如 softmax-CPR）应用于顺序推荐任务，并证明了新的 softmax 架构释放了神经编码器在学习何时复制和何时排除输入序列中的项目方面的能力。通过仅对 SASRec 和 GRU4Rec 的输出 softmax 层进行一些简单的修改，softmax-CPR 在 12 个数据集中实现了持续的改进。与几乎相同的模型大小相比，我们的最佳方法不仅将 GRU4Rec 在有重复项目的 5 个数据集中的平均 NDCG@10 提高了 10%（个别情况下提高了 4%-17%），还将没有重复项目的 7 个数据集的结果提高了 24%（提高了 8%-39%）！