In the field of Natural Language Processing (NLP), Named Entity Recognition
(NER) is recognized as a critical technology, employed across a wide array of
applications. Traditional methodologies for annotating datasets for NER models
are challenged by high costs and variations in dataset quality. This research
introduces a novel hybrid annotation approach that synergizes human effort with
the capabilities of Large Language Models (LLMs). This approach not only aims
to ameliorate the noise inherent in manual annotations, such as omissions,
thereby enhancing the performance of NER models, but also achieves this in a
cost-effective manner. Additionally, by employing a label mixing strategy, it
addresses the issue of class imbalance encountered in LLM-based annotations.
Through an analysis across multiple datasets, this method has been consistently
shown to provide superior performance compared to traditional annotation
methods, even under constrained budget conditions. This study illuminates the
potential of leveraging LLMs to improve dataset quality, introduces a novel
technique to mitigate class imbalances, and demonstrates the feasibility of
achieving high-performance NER in a cost-effective way.

在自然语言处理（NLP）领域，命名实体识别（NER）被认为是一项关键技术，广泛应用于各种应用。本研究引入了一种新颖的混合标注方法，将人力与大型语言模型（LLMs）的能力相结合，旨在提高 NER 模型的性能，并以经济的方式解决传统标注方法存在的噪音和类别不平衡问题。通过多个数据集的分析，该方法在受限预算条件下始终显示出比传统标注方法更优越的性能，揭示了利用 LLMs 提高数据集质量的潜力，引入了一种减轻类别不平衡问题的新技术，并证明了以经济方式实现高性能 NER 的可行性。

利用 LLM 增加 NER 数据集：走向自动化和精确注释

Augmenting NER Datasets with LLMs: Towards Automated and Refined  Annotation

In this paper, we present a novel annotation approach to capture claims and
premises of arguments and their relations in student-written persuasive peer
reviews on business models in German language. We propose an annotation scheme
based on annotation guidelines that allows to model claims and premises as well
as support and attack relations for capturing the structure of argumentative
discourse in student-written peer reviews. We conduct an annotation study with
three annotators on 50 persuasive essays to evaluate our annotation scheme. The
obtained inter-rater agreement of $\alpha=0.57$ for argument components and
$\alpha=0.49$ for argumentative relations indicates that the proposed
annotation scheme successfully guides annotators to moderate agreement.
Finally, we present our freely available corpus of 1,000 persuasive
student-written peer reviews on business models and our annotation guidelines
to encourage future research on the design and development of argumentative
writing support systems for students.

本文提供了一种新颖的注释方法，以捕获德语商业模型学生写作的有关论点和前提以及它们之间的关系，并通过 50 篇说服性文章上的标注研究来评估我们的注释方案，呈现了我们的免费语料库以及指南，以鼓励未来针对学生的论述写作支撑系统的设计和开发的研究。

德语论证写作支持语料库

A Corpus for Argumentative Writing Support in German

We introduce a novel approach for annotating large quantity of in-the-wild
facial images with high-quality posterior age distribution as labels. Each
posterior provides a probability distribution of estimated ages for a face. Our
approach is motivated by observations that it is easier to distinguish who is
the older of two people than to determine the person's actual age. Given a
reference database with samples of known ages and a dataset to label, we can
transfer reliable annotations from the former to the latter via
human-in-the-loop comparisons. We show an effective way to transform such
comparisons to posterior via fully-connected and SoftMax layers, so as to
permit end-to-end training in a deep network. Thanks to the efficient and
effective annotation approach, we collect a new large-scale facial age dataset,
dubbed `MegaAge', which consists of 41,941 images. Data can be downloaded from
our project page mmlab.ie.cuhk.edu.hk/projects/MegaAge and
github.com/zyx2012/Age_estimation_BMVC2017. With the dataset, we train a
network that jointly performs ordinal hyperplane classification and posterior
distribution learning. Our approach achieves state-of-the-art results on
popular benchmarks such as MORPH2, Adience, and the newly proposed MegaAge.

介绍了一种新方法来标注大量的野外人脸图像，将高质量的后验年龄分布作为标签；通过人在循环比较，可以将可靠的注释从一个已知年龄样本的参考数据库转移到另一个标签数据集，并通过全连接和 SoftMax 层有效地将比较转换为后验概率分布，从而训练一种同时执行序数超平面分类和后验分布学习的网络，并达到了基准测试的最新成果。