Available training data for named entity recognition (NER) often contains a
significant percentage of incorrect labels for entity types and entity
boundaries. Such label noise poses challenges for supervised learning and may
significantly deteriorate model quality. To address this, prior work proposed
various noise-robust learning approaches capable of learning from data with
partially incorrect labels. These approaches are typically evaluated using
simulated noise where the labels in a clean dataset are automatically
corrupted. However, as we show in this paper, this leads to unrealistic noise
that is far easier to handle than real noise caused by human error or
semi-automatic annotation. To enable the study of the impact of various types
of real noise, we introduce NoiseBench, an NER benchmark consisting of clean
training data corrupted with 6 types of real noise, including expert errors,
crowdsourcing errors, automatic annotation errors and LLM errors. We present an
analysis that shows that real noise is significantly more challenging than
simulated noise, and show that current state-of-the-art models for noise-robust
learning fall far short of their theoretically achievable upper bound. We
release NoiseBench to the research community.

提出了一个包含 6 种真实噪音的 NER 基准，证明真实噪音比模拟噪音更具挑战性，并展示当前最先进的噪音鲁棒学习模型远未达到其理论可达的上限。

NoiseBench: 命名实体识别中真实标签噪声对性能的基准测试

NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity  Recognition

Named Entity Recognition (NER) is a fundamental NLP task, commonly formulated
as classification over a sequence of tokens. Morphologically-Rich Languages
(MRLs) pose a challenge to this basic formulation, as the boundaries of Named
Entities do not necessarily coincide with token boundaries, rather, they
respect morphological boundaries. To address NER in MRLs we then need to answer
two fundamental questions, namely, what are the basic units to be labeled, and
how can these units be detected and classified in realistic settings, i.e.,
where no gold morphology is available. We empirically investigate these
questions on a novel NER benchmark, with parallel tokenlevel and morpheme-level
NER annotations, which we develop for Modern Hebrew, a morphologically
rich-and-ambiguous language. Our results show that explicitly modeling
morphological boundaries leads to improved NER performance, and that a novel
hybrid architecture, in which NER precedes and prunes morphological
decomposition, greatly outperforms the standard pipeline, where morphological
decomposition strictly precedes NER, setting a new performance bar for both
Hebrew NER and Hebrew morphological decomposition tasks.

通过调查一个新的 NER 基准，我们研究了现代希伯来语中的 NER。 结果表明，显式建模形态的边界可以提高 NER 性能，而一个新的混合体系结构极大地超越了标准流水线，在这个流水线中形态分解严格先于 NER，为希伯来语 NER 和希伯来语形态分解任务设置了新的性能基准。