Sequential recommendation can capture user chronological preferences from
their historical behaviors, yet the learning of short sequences (cold-start
problem) in many benchmark datasets is still an open challenge. Recently, data
augmentation with pseudo-prior items generated by Transformers has drawn
considerable attention. These methods can generate pseudo-prior items
sequentially in reverse chronological order to extend the original sequences.
Nevertheless, the performance may still dramatically degrade in very short
sequences; most notably, the generation of pseudo-prior items does not take
into account the forward direction (from the past to the future), and so the
underlying temporal correlations are not preserved in terms of conditional
probabilities. Motivated by this, we propose a Bidirectional Chronological
Augmentation of Transformer (BiCAT) that uses a forward learning constraint in
the reverse generative process to capture contextual information more
effectively. Then, self-knowledge distillation is adopted in augmented and
original sequences to bridge the gap between data augmentation and model
representation, which enhances the robustness of sequence encoder. Moreover, an
informative positive and negative sampling strategy is proposed to accelerate
optimization and prevent overfitting. Extensive experiments on two popular
real-world datasets demonstrate the efficacy of our method on very short
sequences (L < 3) and long sequences (20 < L < 50) as well, our approach
outperforms state-of-the-art models by an average of 35.04% and 8.76%
respectively, in terms of Recall@5. Source code is available at
this https URL

研究通过双向时间排序转换数据增强和自主知识蒸馏提高顺序推荐中对于短序列的精度

带双向时间增强的 Transformer 序列推荐

Sequential Recommendation with Bidirectional Chronological Augmentation  of Transformer

Sequential Recommendation characterizes the evolving patterns by modeling
item sequences chronologically. The essential target of it is to capture the
item transition correlations. The recent developments of transformer inspire
the community to design effective sequence encoders, \textit{e.g.,} SASRec and
BERT4Rec. However, we observe that these transformer-based models suffer from
the cold-start issue, \textit{i.e.,} performing poorly for short sequences.
Therefore, we propose to augment short sequences while still preserving
original sequential correlations. We introduce a new framework for
\textbf{A}ugmenting \textbf{S}equential \textbf{Re}commendation with
\textbf{P}seudo-prior items~(ASReP). We firstly pre-train a transformer with
sequences in a reverse direction to predict prior items. Then, we use this
transformer to generate fabricated historical items at the beginning of short
sequences. Finally, we fine-tune the transformer using these augmented
sequences from the time order to predict the next item. Experiments on two
real-world datasets verify the effectiveness of ASReP. The code is available on
https://github.com/DyGRec/ASReP.

本研究提出了一个新的框架 ASReP，其目的是增强短序列数据的特征，避免 transformer-based models 在冷启动问题上性能不佳。通过预训练将 transformer 引导生成历史虚拟数据，并在时间顺序上微调 transformer。实验证明 ASReP 的应用有效性。