In this paper, we report our method for the Information Extraction task in
2019 Language and Intelligence Challenge. We incorporate BERT into the
multi-head selection framework for joint entity-relation extraction. This model
extends existing approaches from three perspectives. First, BERT is adopted as
a feature extraction layer at the bottom of the multi-head selection framework.
We further optimize BERT by introducing a semantic-enhanced task during BERT
pre-training. Second, we introduce a large-scale Baidu Baike corpus for entity
recognition pre-training, which is of weekly supervised learning since there is
no actual named entity label. Third, soft label embedding is proposed to
effectively transmit information between entity recognition and relation
extraction. Combining these three contributions, we enhance the information
extracting ability of the multi-head selection model and achieve F1-score 0.876
on testset-1 with a single model. By ensembling four variants of our model, we
finally achieve F1 score 0.892 (1st place) on testset-1 and F1 score 0.8924
(2nd place) on testset-2.

本文利用 BERT 作为特征提取层并引入语义增强的预训练任务，并使用大规模数据集进行实体识别预训练，提出软标签嵌入以有效传递实体识别和关系提取之间的信息，从而增强多头选择模型的信息提取能力，最终在测试集上获得了 0.892 的 F1 分数。

基于 BERT 的多头选择联合实体关系抽取

BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction

State-of-the-art models for joint entity recognition and relation extraction
strongly rely on external natural language processing (NLP) tools such as POS
(part-of-speech) taggers and dependency parsers. Thus, the performance of such
joint models depends on the quality of the features obtained from these NLP
tools. However, these features are not always accurate for various languages
and contexts. In this paper, we propose a joint neural model which performs
entity recognition and relation extraction simultaneously, without the need of
any manually extracted features or the use of any external tool. Specifically,
we model the entity recognition task using a CRF (Conditional Random Fields)
layer and the relation extraction task as a multi-head selection problem (i.e.,
potentially identify multiple relations for each entity). We present an
extensive experimental setup, to demonstrate the effectiveness of our method
using datasets from various contexts (i.e., news, biomedical, real estate) and
languages (i.e., English, Dutch). Our model outperforms the previous neural
models that use automatically extracted features, while it performs within a
reasonable margin of feature-based neural models, or even beats them.

该研究提出了一种基于神经网络的联合模型，能够同时执行实体识别和关系抽取任务，而无需人工提取特征或使用任何外部工具，并在包括新闻、生物医学和房地产等不同语境下进行了实验，而且表现优异。