Online forums and social media platforms provide noisy but valuable data
every day. In this paper, we propose a novel end-to-end neural network-based
user embedding system, Author2Vec. The model incorporates sentence
representations generated by BERT (Bidirectional Encoder Representations from
Transformers) with a novel unsupervised pre-training objective, authorship
classification, to produce better user embedding that encodes useful
user-intrinsic properties. This user embedding system was pre-trained on post
data of 10k Reddit users and was analyzed and evaluated on two user
classification benchmarks: depression detection and personality classification,
in which the model proved to outperform traditional count-based and
prediction-based methods. We substantiate that Author2Vec successfully encoded
useful user attributes and the generated user embedding performs well in
downstream classification tasks without further finetuning.

本文提出了一种基于神经网络和 BERT 模型的用户嵌入系统 ——Author2Vec，其通过使用 BERT 模型中的句子嵌入技术，结合新颖的无监督预训练目标，即作者分类，以生成更好的用户嵌入表示。该系统在 Reddit 平台的 1 万个用户的文章数据上进行预训练，并在抑郁症与人格分类等两个用户分类基准中进行了分析和评估，结果表明其性能优于传统的基于计数和基于预测的方法。Author2Vec 成功地编码了有用的用户属性，并在下游分类任务中表现出良好的性能，无需进一步的微调。