Most Named Entity Recognition (NER) models operate under the assumption that
training datasets are fully labelled. While it is valid for established
datasets like CoNLL 2003 and OntoNotes, sometimes it is not feasible to obtain
the complete dataset annotation. These situations may occur, for instance,
after selective annotation of entities for cost reduction. This work presents
an approach to finetuning BERT on such partially labelled datasets using
self-supervision and label preprocessing. Our approach outperforms the previous
LSTM-based label preprocessing baseline, significantly improving the
performance on poorly labelled datasets. We demonstrate that following our
approach while finetuning RoBERTa on CoNLL 2003 dataset with only 10% of total
entities labelled is enough to reach the performance of the baseline trained on
the same dataset with 50% of the entities labelled.

本文提出一种基于 BERT 和自监督学习、标签预处理来优化部分标记数据集上的命名实体识别 (NER) 模型的方法。我们的方法在处理缺乏标记的数据集时显著优于基于 LSTM 的标签预处理基线，并证明在 CoNLL 2003 数据集上使用 RoBERTa 模型只标记 10％的实体即可达到与 50％实体标记的基线模型相同的性能。