Despite its popularity in sentence-level relation extraction, distantly
supervised data is rarely utilized by existing work in document-level relation
extraction due to its noisy nature and low information density. Among its
current applications, distantly supervised data is mostly used as a whole for
pertaining, which is of low time efficiency. To fill in the gap of efficient
and robust utilization of distantly supervised training data, we propose
Efficient Multi-Supervision for document-level relation extraction, in which we
first select a subset of informative documents from the massive dataset by
combining distant supervision with expert supervision, then train the model
with Multi-Supervision Ranking Loss that integrates the knowledge from multiple
sources of supervision to alleviate the effects of noise. The experiments
demonstrate the effectiveness of our method in improving the model performance
with higher time efficiency than existing baselines.

提出了高效的多监督关系抽取方法，通过融合远程监督和专家监督选择信息丰富的文档子集，并利用多方监督排名损失训练模型，以提高模型性能并具有较高的时间效率。

增强文档级关系抽取的高效多重监督

Augmenting Document-level Relation Extraction with Efficient  Multi-Supervision

In low-resource settings, the performance of supervised labeling models can
be improved with automatically annotated or distantly supervised data, which is
cheap to create but often noisy. Previous works have shown that significant
improvements can be reached by injecting information about the confusion
between clean and noisy labels in this additional training data into the
classifier training. However, for noise estimation, these approaches either do
not take the input features (in our case word embeddings) into account, or they
need to learn the noise modeling from scratch which can be difficult in a
low-resource setting. We propose to cluster the training data using the input
features and then compute different confusion matrices for each cluster. To the
best of our knowledge, our approach is the first to leverage feature-dependent
noise modeling with pre-initialized confusion matrices. We evaluate on
low-resource named entity recognition settings in several languages, showing
that our methods improve upon other confusion-matrix based methods by up to 9%.

在资源匮乏的情况下，使用自动注释或远程监督数据可以改善监督标注模型的性能，但这些数据往往信息有噪声，并且噪声估计需要考虑输入特征和预初始化的混淆矩阵。我们提出将训练数据使用输入特征进行聚类，为每个簇计算不同的混淆矩阵，并在多种语言的命名实体识别场景中进行评估，显示我们的方法比其他混淆矩阵方法提高了高达 9% 的性能。

基于特征的混淆矩阵用于低资源 NER 标记和噪声标记

Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with  Noisy Labels

Multiple entities in a document generally exhibit complex inter-sentence
relations, and cannot be well handled by existing relation extraction (RE)
methods that typically focus on extracting intra-sentence relations for single
entity pairs. In order to accelerate the research on document-level RE, we
introduce DocRED, a new dataset constructed from Wikipedia and Wikidata with
three features: (1) DocRED annotates both named entities and relations, and is
the largest human-annotated dataset for document-level RE from plain text; (2)
DocRED requires reading multiple sentences in a document to extract entities
and infer their relations by synthesizing all information of the document; (3)
along with the human-annotated data, we also offer large-scale distantly
supervised data, which enables DocRED to be adopted for both supervised and
weakly supervised scenarios. In order to verify the challenges of
document-level RE, we implement recent state-of-the-art methods for RE and
conduct a thorough evaluation of these methods on DocRED. Empirical results
show that DocRED is challenging for existing RE methods, which indicates that
document-level RE remains an open problem and requires further efforts. Based
on the detailed analysis on the experiments, we discuss multiple promising
directions for future research.

本文介绍 DocRED 数据集，该数据集为文档级别中关系抽取 (Relation Extraction) 提供了一个新的方法，并提供了大规模的远程监督数据，以满足超级 / 弱监督训练需求。同时我们实验了目前最先进的方法，结果表明文档级别中关系抽取仍需要进一步的研究。