In this paper, we introduce HateBERT, a re-trained BERT model for abusive
language detection in English. The model was trained on RAL-E, a large-scale
dataset of Reddit comments in English from communities banned for being
offensive, abusive, or hateful that we have collected and made available to the
public. We present the results of a detailed comparison between a general
pre-trained language model and the abuse-inclined version obtained by
retraining with posts from the banned communities on three English datasets for
offensive, abusive language and hate speech detection tasks. In all datasets,
HateBERT outperforms the corresponding general BERT model. We also discuss a
battery of experiments comparing the portability of the generic pre-trained
language model and its corresponding abusive language-inclined counterpart
across the datasets, indicating that portability is affected by compatibility
of the annotated phenomena.

介绍了 HateBERT，这是一个在大规模 Reddit 评论数据集（RAL-E）上重新训练过的 BERT 模型，用于检测英语中的辱骂语言。在三个英语数据集中，加入了被封禁社区的帖子进行重新训练，检测攻击性、辱骂性语言和仇恨言论，结果表明 HateBERT 优于通用 BERT 模型，并进行了相关的实验。

HateBERT: 用于英文恶意语言检测的 BERT 重新训练

HateBERT: Retraining BERT for Abusive Language Detection in English

Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying and
engaging conversations with a user; however, they typically exhibit either
inconsistent personality across conversations or the average personality of all
users. This paper addresses these issues by controlling an agent's persona upon
generation via conditioning on prior conversations of a target actor. In doing
so, we are able to utilize more abstract patterns within a person's speech and
better emulate them in generated responses. This work introduces the Generative
Conversation Control model, an augmented and fine-tuned GPT-2 language model
that conditions on past reference conversations to probabilistically model
multi-turn conversations in the actor's persona. We introduce an accompanying
data collection procedure to obtain 10.3M conversations from 6 months worth of
Reddit comments. We demonstrate that scaling model sizes from 117M to 8.3B
parameters yields an improvement from 23.14 to 13.14 perplexity on 1.7M held
out Reddit conversations. Increasing model scale yielded similar improvements
in human evaluations that measure preference of model samples to the held out
target distribution in terms of realism (31% increased to 37% preference),
style matching (37% to 42%), grammar and content quality (29% to 42%), and
conversation coherency (32% to 40%). We find that conditionally modeling past
conversations improves perplexity by 0.47 in automatic evaluations. Through
human trials we identify positive trends between conditional modeling and style
matching and outline steps to further improve persona control.

本文提出了一种新的对话代理模型 ——Generative Conversation Control，它能够控制对话代理的个性，并通过控制该模型上一次与目标演员的对话，实现对话代理个性的表达，该模型在 Reddit 评论数据上进行了验证和评估，其效果得到了显著提高。