Retrieval-based methods have been shown to be effective in NLP tasks via
introducing external knowledge. However, the indexing and retrieving of
large-scale corpora bring considerable computational cost. Surprisingly, we
found that REtrieving from the traINing datA (REINA) only can lead to
significant gains on multiple NLG and NLU tasks. We retrieve the labeled
training instances most similar to the input text and then concatenate them
with the input to feed into the model to generate the output. Experimental
results show that this simple method can achieve significantly better
performance on a variety of NLU and NLG tasks, including summarization, machine
translation, language modeling, and question answering tasks. For instance, our
proposed method achieved state-of-the-art results on XSum, BigPatent, and
CommonsenseQA. Our code is released, this https URL .

本文提出了一种简单的 REtrieving from the traINing datA (REINA) 方法，通过从训练数据中检索最相似的实例并与输入进行拼接来提高自然语言处理任务的性能，实验证明该方法在多个 NLU 和 NLG 任务中都能取得最先进的结果。