We introduce a meta dataset for few-shot relation extraction, which includes
two datasets derived from existing supervised relation extraction datasets
NYT29 (Takanobu et al., 2019; Nayak and Ng, 2020) and WIKIDATA (Sorokin and
Gurevych, 2017) as well as a few-shot form of the TACRED dataset (Sabo et al.,
2021). Importantly, all these few-shot datasets were generated under realistic
assumptions such as: the test relations are different from any relations a
model might have seen before, limited training data, and a preponderance of
candidate relation mentions that do not correspond to any of the relations of
interest. Using this large resource, we conduct a comprehensive evaluation of
six recent few-shot relation extraction methods, and observe that no method
comes out as a clear winner. Further, the overall performance on this task is
low, indicating substantial need for future research. We release all versions
of the data, i.e., both supervised and few-shot, for future research.

我们引入了一个元数据集，其中包括两个从现有的监督关系抽取数据集 NYT29 和 WIKIDATA（以及 TACRED 的 few-shot 形式）中派生的数据集。我们对六种最近的 few-shot 关系抽取方法进行了全面评估，观察到没有一种方法被明确地确定为胜者。此任务的整体性能较低，表明未来研究有很大的需求。我们释放了所有数据版本，即监督和 few-shot 数据，供未来研究使用。