Recent studies have proposed unified user modeling frameworks that leverage
user behavior data from various applications. Many of them benefit from
utilizing users' behavior sequences as plain texts, representing rich
information in any domain or system without losing generality. Hence, a
question arises: Can language modeling for user history corpus help improve
recommender systems? While its versatile usability has been widely investigated
in many domains, its applications to recommender systems still remain
underexplored. We show that language modeling applied directly to task-specific
user histories achieves excellent results on diverse recommendation tasks.
Also, leveraging additional task-agnostic user histories delivers significant
performance benefits. We further demonstrate that our approach can provide
promising transfer learning capabilities for a broad spectrum of real-world
recommender systems, even on unseen domains and services.

本文探讨了如何通过语言建模提高推荐系统的效果，研究表明将语言建模应用到特定任务的用户历史记录可以带来显著的性能优势，并且可以提供广泛的迁移学习能力，适用于各种领域和系统。

语言建模在推荐系统中的关键作用：丰富任务特定和任务不可知的表示学习

Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning

Click-Through Rate (CTR) prediction plays an important role in many
industrial applications, such as online advertising and recommender systems.
How to capture users' dynamic and evolving interests from their behavior
sequences remains a continuous research topic in the CTR prediction. However,
most existing studies overlook the intrinsic structure of the sequences: the
sequences are composed of sessions, where sessions are user behaviors separated
by their occurring time. We observe that user behaviors are highly homogeneous
in each session, and heterogeneous cross sessions. Based on this observation,
we propose a novel CTR model named Deep Session Interest Network (DSIN) that
leverages users' multiple historical sessions in their behavior sequences. We
first use self-attention mechanism with bias encoding to extract users'
interests in each session. Then we apply Bi-LSTM to model how users' interests
evolve and interact among sessions. Finally, we employ the local activation
unit to adaptively learn the influences of various session interests on the
target item. Experiments are conducted on both advertising and production
recommender datasets and DSIN outperforms other state-of-the-art models on both
datasets.

论文提出了一种名为 Deep Session Interest Network (DSIN) 的模型，它利用了用户历史行为序列中的多个会话，并将自我注意机制与双向 LSTM 相结合，以建模用户的兴趣演化，并通过局部激活单元自适应地学习各种会话对目标项目的影响，从而在在线广告和推荐系统等工业应用中提高了点击率（CTR）预测的准确性。