Named Entity Recognition (NER) is a well-studied problem in NLP. However,
there is much less focus on studying NER datasets, compared to developing new
NER models. In this paper, we employed three simple techniques to detect
annotation errors in the OntoNotes 5.0 corpus for English NER, which is the
largest available NER corpus for English. Our techniques corrected ~10% of the
sentences in train/dev/test data. In terms of entity mentions, we corrected the
span and/or type of ~8% of mentions in the dataset, while
adding/deleting/splitting/merging a few more. These are large numbers of
changes, considering the size of OntoNotes. We used three NER libraries to
train, evaluate and compare the models trained with the original and the
re-annotated datasets, which showed an average improvement of 1.23% in overall
F-scores, with large (>10%) improvements for some of the entity types. While
our annotation error detection methods are not exhaustive and there is some
manual annotation effort involved, they are largely language agnostic and can
be employed with other NER datasets, and other sequence labelling tasks.

该研究通过使用三种简单技术在 OntoNotes 5.0 语料库中检测注释错误，对英文命名实体识别（Named Entity Recognition，NER）的训练集、开发集和测试集进行了大量修正，提高了 1.23% 的整体 F 分数，对某些实体类型的改进超过 10%。这些技术可应用于其他 NER 数据集和其他序列标记任务。

注释错误与 NER：OntoNotes 5.0 研究

Annotation Errors and NER: A Study with OntoNotes 5.0

We present WojoodNER-2023, the first Arabic Named Entity Recognition (NER)
Shared Task. The primary focus of WojoodNER-2023 is on Arabic NER, offering
novel NER datasets (i.e., Wojood) and the definition of subtasks designed to
facilitate meaningful comparisons between different NER approaches.
WojoodNER-2023 encompassed two Subtasks: FlatNER and NestedNER. A total of 45
unique teams registered for this shared task, with 11 of them actively
participating in the test phase. Specifically, 11 teams participated in
FlatNER, while $8$ teams tackled NestedNER. The winning teams achieved F1
scores of 91.96 and 93.73 in FlatNER and NestedNER, respectively.

WojoodNER-2023 是第一个阿拉伯命名实体识别（NER）共享任务，提供新的 NER 数据集（即 Wojood），定义了用于不同 NER 方法之间有意义比较的子任务。本次共享任务包含两个子任务：FlatNER 和 NestedNER。共有 45 个独特的团队注册参加，其中 11 个团队积极参与测试阶段。具体而言，11 个团队参与了 FlatNER，而 8 个团队解决了 NestedNER。获胜团队在 FlatNER 和 NestedNER 中分别取得了 91.96 和 93.73 的 F1 分数。

WojoodNER 2023: 首个阿拉伯命名实体识别共享任务

WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task

Nested entities are observed in many domains due to their compositionality,
which cannot be easily recognized by the widely-used sequence labeling
framework. A natural solution is to treat the task as a span classification
problem. To learn better span representation and increase classification
performance, it is crucial to effectively integrate heterogeneous factors
including inside tokens, boundaries, labels, and related spans which could be
contributing to nested entities recognition. To fuse these heterogeneous
factors, we propose a novel triaffine mechanism including triaffine attention
and scoring. Triaffine attention uses boundaries and labels as queries and uses
inside tokens and related spans as keys and values for span representations.
Triaffine scoring interacts with boundaries and span representations for
classification. Experiments show that our proposed method outperforms previous
span-based methods, achieves the state-of-the-art $F_1$ scores on nested NER
datasets GENIA and KBP2017, and shows comparable results on ACE2004 and
ACE2005.

使用 triaffine 机制来完成嵌套实体识别，包括三元注意力和打分机制，实验结果表明，该方法在嵌套命名实体识别任务中取得了最优结果。

基于 Triaffine 机制融合异构因素的嵌套命名实体识别

Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named  Entity Recognition

We consider a multilingual weakly supervised learning scenario where
knowledge from annotated corpora in a resource-rich language is transferred via
bitext to guide the learning in other languages. Past approaches project labels
across bitext and use them as features or gold labels for training. We propose
a new method that projects model expectations rather than labels, which
facilities transfer of model uncertainty across language boundaries. We encode
expectations as constraints and train a discriminative CRF model using
Generalized Expectation Criteria (Mann and McCallum, 2010). Evaluated on
standard Chinese-English and German-English NER datasets, our method
demonstrates F1 scores of 64% and 60% when no labeled data is used. Attaining
the same accuracy with supervised CRFs requires 12k and 1.5k labeled sentences.
Furthermore, when combined with labeled examples, our method yields significant
improvements over state-of-the-art supervised methods, achieving best reported
numbers to date on Chinese OntoNotes and German CoNLL-03 datasets.

本文提出了基于 bitext 资源将多语种低监督学习领域的已标注语料库知识转移的一种新方法，该方法使用推断的模型期望而不是标签来指导学习。在标注数据不足的情况下，评估结果表明，在标准的中英和德英 NER 数据集上，我们的方法表现出 64％和 60％的 F1 得分。与监督 CRFs 相比，实现相同准确性需要 12k 和 1.5k 标记句子。此外，当与标记示例结合使用时，我们的方法在中文 OntoNotes 和德语 CoNLL-03 数据集上获得了迄今为止报告的最佳结果。