Online Social Networks serve as fertile ground for harmful behavior, ranging
from hate speech to the dissemination of disinformation. Malicious actors now
have unprecedented freedom to misbehave, leading to severe societal unrest and
dire consequences, as exemplified by events such as the Capitol assault during
the US presidential election and the Antivaxx movement during the COVID-19
pandemic. Understanding online language has become more pressing than ever.
While existing works predominantly focus on content analysis, we aim to shift
the focus towards understanding harmful behaviors by relating content to their
respective authors. Numerous novel approaches attempt to learn the stylistic
features of authors in texts, but many of these approaches are constrained by
small datasets or sub-optimal training losses. To overcome these limitations,
we introduce the Style Transformer for Authorship Representations (STAR),
trained on a large corpus derived from public sources of 4.5 x 10^6 authored
texts involving 70k heterogeneous authors. Our model leverages Supervised
Contrastive Loss to teach the model to minimize the distance between texts
authored by the same individual. This author pretext pre-training task yields
competitive performance at zero-shot with PAN challenges on attribution and
clustering. Additionally, we attain promising results on PAN verification
challenges using a single dense layer, with our model serving as an embedding
encoder. Finally, we present results from our test partition on Reddit. Using a
support base of 8 documents of 512 tokens, we can discern authors from sets of
up to 1616 authors with at least 80\% accuracy. We share our pre-trained model
at huggingface (this https URL) and our code is available
at (this https URL)

在线社交网络是有害行为的肥沃土壤，从仇恨言论到虚假信息的传播。我们提出了一种基于作者风格的转换模型 STAR，通过关联内容与其各自的作者来理解有害行为。我们的模型在与 PAN 挑战中的归因和聚类任务上表现出有竞争力的性能，并且在 PAN 验证挑战中使用单个密集层取得了有希望的结果。

利用监督对比预训练转换器解析社交媒体中的写作风格

Understanding writing style in social media with a supervised  contrastively pre-trained transformer

Automatically disentangling an author's style from the content of their
writing is a longstanding and possibly insurmountable problem in computational
linguistics. At the same time, the availability of large text corpora furnished
with author labels has recently enabled learning authorship representations in
a purely data-driven manner for authorship attribution, a task that ostensibly
depends to a greater extent on encoding writing style than encoding content.
However, success on this surrogate task does not ensure that such
representations capture writing style since authorship could also be correlated
with other latent variables, such as topic. In an effort to better understand
the nature of the information these representations convey, and specifically to
validate the hypothesis that they chiefly encode writing style, we
systematically probe these representations through a series of targeted
experiments. The results of these experiments suggest that representations
learned for the surrogate authorship prediction task are indeed sensitive to
writing style. As a consequence, authorship representations may be expected to
be robust to certain kinds of data shift, such as topic drift over time.
Additionally, our findings may open the door to downstream applications that
require stylistic representations, such as style transfer.

自动从作者的写作风格中脱离其内容是计算语言学中一个长期存在且可能难以逾越的问题。然而，最近可用的带有作者标签的大型文本语料库使得可以通过完全基于数据驱动的方式来学习作者表征，用于作者归属的任务，这个任务似乎更多地依赖于编码写作风格而不是编码内容。然而，成功完成这个替代任务并不能确保这些表征能够捕捉到写作风格，因为作者归属也可能与其他潜在变量（如主题）相关。为了更好地理解这些表征传达的信息性质，特别是验证它们主要是否编码写作风格的假设，我们通过一系列有针对性的实验系统地探查了这些表征。这些实验的结果表明，学习用于替代作者归属预测任务的表征确实对写作风格敏感。因此，可预期作者表征对某些数据转换具有稳健性，例如随时间的主题漂移。此外，我们的发现可能为需要风格表征的下游应用开启大门，如风格转换。