Online Social Networks serve as fertile ground for harmful behavior, ranging
from hate speech to the dissemination of disinformation. Malicious actors now
have unprecedented freedom to misbehave, leading to severe societal unrest and
dire consequences, as exemplified by events such as the Capitol assault during
the US presidential election and the Antivaxx movement during the COVID-19
pandemic. Understanding online language has become more pressing than ever.
While existing works predominantly focus on content analysis, we aim to shift
the focus towards understanding harmful behaviors by relating content to their
respective authors. Numerous novel approaches attempt to learn the stylistic
features of authors in texts, but many of these approaches are constrained by
small datasets or sub-optimal training losses. To overcome these limitations,
we introduce the Style Transformer for Authorship Representations (STAR),
trained on a large corpus derived from public sources of 4.5 x 10^6 authored
texts involving 70k heterogeneous authors. Our model leverages Supervised
Contrastive Loss to teach the model to minimize the distance between texts
authored by the same individual. This author pretext pre-training task yields
competitive performance at zero-shot with PAN challenges on attribution and
clustering. Additionally, we attain promising results on PAN verification
challenges using a single dense layer, with our model serving as an embedding
encoder. Finally, we present results from our test partition on Reddit. Using a
support base of 8 documents of 512 tokens, we can discern authors from sets of
up to 1616 authors with at least 80\% accuracy. We share our pre-trained model
at huggingface (this https URL) and our code is available
at (this https URL)

在线社交网络是有害行为的肥沃土壤，从仇恨言论到虚假信息的传播。我们提出了一种基于作者风格的转换模型 STAR，通过关联内容与其各自的作者来理解有害行为。我们的模型在与 PAN 挑战中的归因和聚类任务上表现出有竞争力的性能，并且在 PAN 验证挑战中使用单个密集层取得了有希望的结果。