Recent advancements of sequential deep learning models such as Transformer
and BERT have significantly facilitated the sequential recommendation. However,
according to our study, the distribution of item embeddings generated by these
models tends to degenerate into an anisotropic shape, which may result in high
semantic similarities among embeddings. In this paper, both empirical and
theoretical investigations of this representation degeneration problem are
first provided, based on which a novel recommender model DuoRec is proposed to
improve the item embeddings distribution. Specifically, in light of the
uniformity property of contrastive learning, a contrastive regularization is
designed for DuoRec to reshape the distribution of sequence representations.
Given the convention that the recommendation task is performed by measuring the
similarity between sequence representations and item embeddings in the same
space via dot product, the regularization can be implicitly applied to the item
embedding distribution. Existing contrastive learning methods mainly rely on
data level augmentation for user-item interaction sequences through item
cropping, masking, or reordering and can hardly provide semantically consistent
augmentation samples. In DuoRec, a model-level augmentation is proposed based
on Dropout to enable better semantic preserving. Furthermore, a novel sampling
strategy is developed, where sequences having the same target item are chosen
hard positive samples. Extensive experiments conducted on five datasets
demonstrate the superior performance of the proposed DuoRec model compared with
baseline methods. Visualization results of the learned representations validate
that DuoRec can largely alleviate the representation degeneration problem.

本文提出了一种新的推荐模型 DuoRec，通过对用户行为序列的表示形式的优化，解决了深度学习模型在生成物品表示时的退化问题，并在五个数据集上进行了实验验证。