Public datasets are often used to evaluate the efficacy and generalizability
of state-of-the-art methods for many tasks in natural language processing
(NLP). However, the presence of overlap between the train and test datasets can
lead to inflated results, inadvertently evaluating the model's ability to
memorize and interpreting it as the ability to generalize. In addition, such
data sets may not provide an effective indicator of the performance of these
methods in real world scenarios. We identify leakage of training data into test
data on several publicly available datasets used to evaluate NLP tasks,
including named entity recognition and relation extraction, and study them to
assess the impact of that leakage on the model's ability to memorize versus
generalize.

利用公共数据集来评估自然语言处理（NLP）中最先进方法的有效性和普适性时，训练和测试数据集之间的重叠会导致结果被高估，为了评估这些方法在实际场景中的性能，我们发现在用于评估 NLP 任务的几个公开数据集中，存在训练数据泄漏到测试数据中的情况，对此进行研究以评估泄漏对模型记忆与泛化能力的影响。