Large language models (LLMs) exhibited powerful capability in various natural
language processing tasks. This work focuses on exploring LLM performance on
zero-shot information extraction, with a focus on the ChatGPT and named entity
recognition (NER) task. Inspired by the remarkable reasoning capability of LLM
on symbolic and arithmetic reasoning, we adapt the prevalent reasoning methods
to NER and propose reasoning strategies tailored for NER. First, we explore a
decomposed question-answering paradigm by breaking down the NER task into
simpler subproblems by labels. Second, we propose syntactic augmentation to
stimulate the model's intermediate thinking in two ways: syntactic prompting,
which encourages the model to analyze the syntactic structure itself, and tool
augmentation, which provides the model with the syntactic information generated
by a parsing tool. Besides, we adapt self-consistency to NER by proposing a
two-stage majority voting strategy, which first votes for the most consistent
mentions, then the most consistent types. The proposed methods achieve
remarkable improvements for zero-shot NER across seven benchmarks, including
Chinese and English datasets, and on both domain-specific and general-domain
scenarios. In addition, we present a comprehensive analysis of the error types
with suggestions for optimization directions. We also verify the effectiveness
of the proposed methods on the few-shot setting and other LLMs.

该论文主要研究大型语言模型在零 - shot 信息提取方面的性能，并提出针对命名实体识别的推理策略，包括分解式问答范式、句法增强和自一致性等方法，并在七个基准测试中取得了显著的改进，对错误类型进行了全面分析，并验证了这些方法在少样本情况和其他大型语言模型中的有效性。

ChatGPT 中零样本命名实体识别的实证研究

Empirical Study of Zero-Shot NER with ChatGPT

Zero-shot information extraction (IE) aims to build IE systems from the
unannotated text. It is challenging due to involving little human intervention.
Challenging but worthwhile, zero-shot IE reduces the time and effort that data
labeling takes. Recent efforts on large language models (LLMs, e.g., GPT-3,
ChatGPT) show promising performance on zero-shot settings, thus inspiring us to
explore prompt-based methods. In this work, we ask whether strong IE models can
be constructed by directly prompting LLMs. Specifically, we transform the
zero-shot IE task into a multi-turn question-answering problem with a two-stage
framework (ChatIE). With the power of ChatGPT, we extensively evaluate our
framework on three IE tasks: entity-relation triple extract, named entity
recognition, and event extraction. Empirical results on six datasets across two
languages show that ChatIE achieves impressive performance and even surpasses
some full-shot models on several datasets (e.g., NYT11-HRL). We believe that
our work could shed light on building IE models with limited resources.

本文探讨了零样本信息提取（Zero-shot information extraction）的挑战，通过大型语言模型和基于提示的方法构建具有强大信息提取能力的 ChatIE 框架，成功地实现了多轮问答式的信息提取任务，这为使用有限资源构建信息提取模型提供了研究思路和方法。