Continual Few-shot Relation Extraction (CFRE) is a practical problem that
requires the model to continuously learn novel relations while avoiding
forgetting old ones with few labeled training data. The primary challenges are
catastrophic forgetting and overfitting. This paper harnesses prompt learning
to explore the implicit capabilities of pre-trained language models to address
the above two challenges, thereby making language models better continual
few-shot relation extractors. Specifically, we propose a Contrastive Prompt
Learning framework, which designs prompt representation to acquire more
generalized knowledge that can be easily adapted to old and new categories, and
margin-based contrastive learning to focus more on hard samples, therefore
alleviating catastrophic forgetting and overfitting issues. To further remedy
overfitting in low-resource scenarios, we introduce an effective memory
augmentation strategy that employs well-crafted prompts to guide ChatGPT in
generating diverse samples. Extensive experiments demonstrate that our method
outperforms state-of-the-art methods by a large margin and significantly
mitigates catastrophic forgetting and overfitting in low-resource scenarios.

使用 Prompt Learning 框架和对抗式学习等技术，本文提出了一种解决持续的少样本关系抽取中灾难性遗忘和过拟合问题的方法，该方法通过引入更广义的知识表示和关注难例样本实现了对预训练语言模型的扩展和优化，同时引入有效的内存增强策略来缓解在资源匮乏情况下的过拟合问题，并通过大量实验证明了该方法在性能上超过了现有方法，并且在资源匮乏情景下显著降低了灾难性遗忘和过拟合。

改进预训练语言模型以提升持续少样本关系抽取能力

Making Pre-trained Language Models Better Continual Few-Shot Relation  Extractors

Continual few-shot relation extraction (RE) aims to continuously train a
model for new relations with few labeled training data, of which the major
challenges are the catastrophic forgetting of old relations and the overfitting
caused by data sparsity. In this paper, we propose a new model, namely SCKD, to
accomplish the continual few-shot RE task. Specifically, we design serial
knowledge distillation to preserve the prior knowledge from previous models and
conduct contrastive learning with pseudo samples to keep the representations of
samples in different relations sufficiently distinguishable. Our experiments on
two benchmark datasets validate the effectiveness of SCKD for continual
few-shot RE and its superiority in knowledge transfer and memory utilization
over state-of-the-art models.

本文提出了一种新模型 SCKD，通过串行知识蒸馏来保留以前模型的先前知识，并通过伪样本进行对比学习，以保持不同关系样本的表示具有足够的可区分性，实现持续的少样本关系抽取任务，并在两个基准数据集上进行了验证。