Relation extraction (RE) aims to identify relations between entities
mentioned in texts. Although large language models (LLMs) have demonstrated
impressive in-context learning (ICL) abilities in various tasks, they still
suffer from poor performances compared to most supervised fine-tuned RE
methods. Utilizing ICL for RE with LLMs encounters two challenges: (1)
retrieving good demonstrations from training examples, and (2) enabling LLMs
exhibit strong ICL abilities in RE. On the one hand, retrieving good
demonstrations is a non-trivial process in RE, which easily results in low
relevance regarding entities and relations. On the other hand, ICL with an LLM
achieves poor performance in RE while RE is different from language modeling in
nature or the LLM is not large enough. In this work, we propose a novel
recall-retrieve-reason RE framework that synergizes LLMs with retrieval corpora
(training examples) to enable relevant retrieving and reliable in-context
reasoning. Specifically, we distill the consistently ontological knowledge from
training datasets to let LLMs generate relevant entity pairs grounded by
retrieval corpora as valid queries. These entity pairs are then used to
retrieve relevant training examples from the retrieval corpora as
demonstrations for LLMs to conduct better ICL via instruction tuning. Extensive
experiments on different LLMs and RE datasets demonstrate that our method
generates relevant and valid entity pairs and boosts ICL abilities of LLMs,
achieving competitive or new state-of-the-art performance on sentence-level RE
compared to previous supervised fine-tuning methods and ICL-based methods.

通过提出一种召回 - 检索 - 推理关系抽取框架，将大型语言模型与检索语料库相结合，从而使语言模型能够在上下文中进行可靠的推理，以提高关系抽取的性能。