Existing models for named entity recognition (NER) are mainly based on
large-scale labeled datasets, which always obtain using crowdsourcing. However,
it is hard to obtain a unified and correct label via majority voting from
multiple annotators for NER due to the large labeling space and complexity of
this task. To address this problem, we aim to utilize the original
multi-annotator labels directly. Particularly, we propose a Confidence-based
Partial Label Learning (CPLL) method to integrate the prior confidence (given
by annotators) and posterior confidences (learned by models) for
crowd-annotated NER. This model learns a token- and content-dependent
confidence via an Expectation-Maximization (EM) algorithm by minimizing
empirical risk. The true posterior estimator and confidence estimator perform
iteratively to update the true posterior and confidence respectively. We
conduct extensive experimental results on both real-world and synthetic
datasets, which show that our model can improve performance effectively
compared with strong baselines.

使用 CPLL 模型整合多注释者标签和先验、后验置信度，通过 EM 算法学习标记和内容相关的置信度，以解决命名实体识别中多注释者标签一致性问题，并在实际和合成数据集上得到有效提升。