Fine-tuning pre-trained language models has recently become a common practice
in building NLP models for various tasks, especially few-shot tasks. We argue
that under the few-shot setting, formulating fine-tuning closer to the
pre-training objectives shall be able to unleash more benefits from the
pre-trained language models. In this work, we take few-shot named entity
recognition (NER) for a pilot study, where existing fine-tuning strategies are
much different from pre-training. We propose a novel few-shot fine-tuning
framework for NER, FFF-NER. Specifically, we introduce three new types of
tokens, "is-entity", "which-type" and bracket, so we can formulate the NER
fine-tuning as (masked) token prediction or generation, depending on the choice
of pre-trained language models. In our experiments, we apply FFF-NER to
fine-tune both BERT and BART for few-shot NER on several benchmark datasets and
observe significant improvements over existing fine-tuning strategies,
including sequence labeling, prototype meta-learning, and prompt-based
approaches. We further perform a series of ablation studies, showing few-shot
NER performance is strongly correlated with the similarity between fine-tuning
and pre-training.

本篇研究提出了一种基于预训练模型且更贴近预训练目标的新型 few-shot fine-tuning 框架 FFF-NER，应用于命名实体识别任务上，经过一系列实验及消融研究，确认其比现有序列标记、原型元学习和提示式方法等现有方法能够更有效改善 NER 性能。