Knowledge Graph (KG) and attention mechanism have been demonstrated effective
in introducing and selecting useful information for weakly supervised methods.
However, only qualitative analysis and ablation study are provided as evidence.
In this paper, we contribute a dataset and propose a paradigm to quantitatively
evaluate the effect of attention and KG on bag-level relation extraction (RE).
We find that (1) higher attention accuracy may lead to worse performance as it
may harm the model's ability to extract entity mention features; (2) the
performance of attention is largely influenced by various noise distribution
patterns, which is closely related to real-world datasets; (3) KG-enhanced
attention indeed improves RE performance, while not through enhanced attention
but by incorporating entity prior; and (4) attention mechanism may exacerbate
the issue of insufficient training data. Based on these findings, we show that
a straightforward variant of RE model can achieve significant improvements (6%
AUC on average) on two real-world datasets as compared with three
state-of-the-art baselines. Our codes and datasets are available at
this https URL

本文提供数据集和范例来定量评估注意力和知识图谱对于包级关系抽取的效应，发现高的准确率可能会导致更差的性能，并且性能会受到各种噪声分布模式的影响，增强型的 KG 注意力的确提高了关系抽取性能，但不是由于增强了注意力，而是通过引入实体先验知识。