Sequential recommendation (SR) models based on Transformers have achieved
remarkable successes. The self-attention mechanism of Transformers for computer
vision and natural language processing suffers from the oversmoothing problem,
i.e., hidden representations becoming similar to tokens. In the SR domain, we,
for the first time, show that the same problem occurs. We present pioneering
investigations that reveal the low-pass filtering nature of self-attention in
the SR, which causes oversmoothing. To this end, we propose a novel method
called Beyond Self-Attention for Sequential Recommendation (BSARec), which
leverages the Fourier transform to i) inject an inductive bias by considering
fine-grained sequential patterns and ii) integrate low and high-frequency
information to mitigate oversmoothing. Our discovery shows significant
advancements in the SR domain and is expected to bridge the gap for existing
Transformer-based SR models. We test our proposed approach through extensive
experiments on 6 benchmark datasets. The experimental results demonstrate that
our model outperforms 7 baseline methods in terms of recommendation
performance.

提出了一种名为 BSARec 的新方法，利用傅里叶变换注入归纳偏差并整合低频和高频信息以减轻过平滑问题，通过在 6 个基准数据集上进行广泛实验，结果表明该模型在推荐性能方面优于其他 7 种基线方法。