The number of biomedical literature on new biomedical concepts is rapidly
increasing, which necessitates a reliable biomedical named entity recognition
(BioNER) model for identifying new and unseen entity mentions. However, it is
questionable whether existing models can effectively handle them. In this work,
we systematically analyze the three types of recognition abilities of BioNER
models: memorization, synonym generalization, and concept generalization. We
find that although current best models achieve state-of-the-art performance on
benchmarks based on overall performance, they have limitations in identifying
synonyms and new biomedical concepts, indicating they are overestimated in
terms of their generalization abilities. We also investigate failure cases of
models and identify several difficulties in recognizing unseen mentions in
biomedical literature as follows: (1) models tend to exploit dataset biases,
which hinders the models' abilities to generalize, and (2) several biomedical
names have novel morphological patterns with weak name regularity, and models
fail to recognize them. We apply a statistics-based debiasing method to our
problem as a simple remedy and show the improvement in generalization to unseen
mentions. We hope that our analyses and findings would be able to facilitate
further research into the generalization capabilities of NER models in a domain
where their reliability is of utmost importance.

本文系统地分析了生物识别模型的三种识别能力：记忆，同义词泛化和概念泛化，并发现现有模型在识别同义词和新的生物概念方面存在局限性，这表明他们的泛化能力被高估了。作者调查了模型的失败案例，并确定了识别未见过的生物词汇的几个困难。最后，作者采用基于统计的去偏置方法作为简单的补救方法，显示出对未见过的提及的泛化改进。希望本文的分析和发现能够促进对 NER 模型泛化能力的进一步研究。