Demonstration-based learning has shown great potential in stimulating
pretrained language models' ability under limited data scenario. Simply
augmenting the input with some demonstrations can significantly improve
performance on few-shot NER. However, why such demonstrations are beneficial
for the learning process remains unclear since there is no explicit alignment
between the demonstrations and the predictions. In this paper, we design
pathological demonstrations by gradually removing intuitively useful
information from the standard ones to take a deep dive of the robustness of
demonstration-based sequence labeling and show that (1) demonstrations composed
of random tokens still make the model a better few-shot learner; (2) the length
of random demonstrations and the relevance of random tokens are the main
factors affecting the performance; (3) demonstrations increase the confidence
of model predictions on captured superficial patterns. We have publicly
released our code at this https URL.

本研究旨在探明演示学习方法对于预训练语言模型在数据量有限的情况下的提升效果以及其原因。我们设计了不同类型的演示数据来研究其鲁棒性，发现无论演示数据的内容是随机的还是包含部分有用信息，都能使模型成为更好的 few-shot learner，证明了演示学习方法能够提升标记序列学习的能力。

有限数据情况下示范学习的鲁棒性

Robustness of Demonstration-based Learning Under Limited Data Scenario

Recent advances in prompt-based learning have shown strong results on
few-shot text classification by using cloze-style templates. Similar attempts
have been made on named entity recognition (NER) which manually design
templates to predict entity types for every text span in a sentence. However,
such methods may suffer from error propagation induced by entity span
detection, high cost due to enumeration of all possible text spans, and
omission of inter-dependencies among token labels in a sentence. Here we
present a simple demonstration-based learning method for NER, which lets the
input be prefaced by task demonstrations for in-context learning. We perform a
systematic study on demonstration strategy regarding what to include (entity
examples, with or without surrounding context), how to select the examples, and
what templates to use. Results on in-domain learning and domain adaptation show
that the model's performance in low-resource settings can be largely improved
with a suitable demonstration strategy (e.g., a 4-17% improvement on 25 train
instances). We also find that good demonstration can save many labeled examples
and consistency in demonstration contributes to better performance.

本文提出了一种演示为基础的命名实体识别学习方法，研究了演示策略对模型在低资源情况下性能的影响，并在领域内学习和领域适应中证明了良好的演示可以节省很多标记示例并促进更好的性能表现。