Recent advancements in language models (LMs) have led to the emergence of
powerful models such as Small LMs (e.g., T5) and Large LMs (e.g., GPT-4). These
models have demonstrated exceptional capabilities across a wide range of tasks,
such as name entity recognition (NER) in the general domain. (We define SLMs as
pre-trained models with fewer parameters compared to models like GPT-3/3.5/4,
such as T5, BERT, and others.) Nevertheless, their efficacy in the medical
section remains uncertain and the performance of medical NER always needs high
accuracy because of the particularity of the field. This paper aims to provide
a thorough investigation to compare the performance of LMs in medical few-shot
NER and answer How far is LMs from 100\% Few-shot NER in Medical Domain, and
moreover to explore an effective entity recognizer to help improve the NER
performance. Based on our extensive experiments conducted on 16 NER models
spanning from 2018 to 2023, our findings clearly indicate that LLMs outperform
SLMs in few-shot medical NER tasks, given the presence of suitable examples and
appropriate logical frameworks. Despite the overall superiority of LLMs in
few-shot medical NER tasks, it is important to note that they still encounter
some challenges, such as misidentification, wrong template prediction, etc.
Building on previous findings, we introduce a simple and effective method
called \textsc{RT} (Retrieving and Thinking), which serves as retrievers,
finding relevant examples, and as thinkers, employing a step-by-step reasoning
process. Experimental results show that our proposed \textsc{RT} framework
significantly outperforms the strong open baselines on the two open medical
benchmark datasets

本文旨在比较小型预训练模型和大型预训练模型在医疗 Few-shot NER 任务中的性能，并探索一种有效的实体识别器，称为检索和思考（RT）框架，该框架在 16 个 NER 模型上进行的实验表明，LLMs 在 Few-shot 医疗 NER 任务中的性能优于 SLMs，但其中仍存在一些挑战。