Annotated data have traditionally been used to provide the input for training
a supervised machine learning (ML) model. However, current pre-trained ML
models for natural language processing (NLP) contain embedded linguistic
information that can be used to inform the annotation process. We use the BERT
neural language model to feed information back into an annotation task that
involves semantic labelling of dialog behavior in a question-asking game called
Emotion Twenty Questions (EMO20Q). First we describe the background of BERT,
the EMO20Q data, and assisted annotation tasks. Then we describe the methods
for fine-tuning BERT for the purpose of checking the annotated labels. To do
this, we use the paraphrase task as a way to check that all utterances with the
same annotation label are classified as paraphrases of each other. We show this
method to be an effective way to assess and revise annotations of textual user
data with complex, utterance-level semantic labels.

使用 BERT 神经语言模型对对话行为语义标注的辅助注释任务中的标注标签进行微调并使用复述任务检查其效果，以此有效评估和修订复杂的文本用户数据的语义标签。

使用 BERT 协助进行情感相关问题的语义注释校正

BERT-Assisted Semantic Annotation Correction for Emotion-Related Questions

This paper presents the first unsupervised approach to lexical semantic
change that makes use of contextualised word representations. We propose a
novel method that exploits the BERT neural language model to obtain
representations of word usages, clusters these representations into usage
types, and measures change along time with three proposed metrics. We create a
new evaluation dataset and show that the model representations and the detected
semantic shifts are positively correlated with human judgements. Our extensive
qualitative analysis demonstrates that our method captures a variety of
synchronic and diachronic linguistic phenomena. We expect our work to inspire
further research in this direction.

本文提出了第一种使用上下文词表示进行无监督词汇语义变化的方法。利用 BERT 神经语言模型来获取单词用法的表示，将这些表示聚类到用法类型，并用三种提出的度量方式衡量关于时间的变化。创造了一个新的评估数据集，并表明模型表示和检测到的语义转移与人类判断之间呈正相关。 extensive 的定性分析表明，我们的方法捕捉了各种同步和历时语言现象。我们期望我们的工作将激发进一步的研究。