Contrastive learning with Transformer-based sequence encoder has gained
predominance for sequential recommendation. It maximizes the agreements between
paired sequence augmentations that share similar semantics. However, existing
contrastive learning approaches in sequential recommendation mainly center upon
left-to-right unidirectional Transformers as base encoders, which are
suboptimal for sequential recommendation because user behaviors may not be a
rigid left-to-right sequence. To tackle that, we propose a novel framework
named \textbf{C}ontrastive learning with \textbf{Bi}directional
\textbf{T}ransformers for sequential recommendation (\textbf{CBiT}).
Specifically, we first apply the slide window technique for long user sequences
in bidirectional Transformers, which allows for a more fine-grained division of
user sequences. Then we combine the cloze task mask and the dropout mask to
generate high-quality positive samples and perform multi-pair contrastive
learning, which demonstrates better performance and adaptability compared with
the normal one-pair contrastive learning. Moreover, we introduce a novel
dynamic loss reweighting strategy to balance between the cloze task loss and
the contrastive loss. Experiment results on three public benchmark datasets
show that our model outperforms state-of-the-art models for sequential
recommendation.

本文提出了一种新框架，使用双向 Transformer 进行对比学习，针对顺序推荐中用户行为不是严格的从左到右顺序这一问题进行改进，同时采用滑动窗口技术，结合 Cloze 任务掩码和 Dropout 掩码来生成高质量正样本，并进行多对比学习，通过实验结果表明，相较于传统单一对比学习，该框架表现更优越。