Large Language Models (LLMs) have ushered in a new era of text annotation, as their ease-of-use, high accuracy, and relatively low costs have meant that their use has exploded in recent months. However, the rapid growth of the field has meant that LLM-based annotation has become something of an academic Wild West: the lack of established practices and standards has led to concerns about the quality and validity of research. Researchers have warned that the ostensible simplicity of LLMs can be misleading, as they are prone to bias, misunderstandings, and unreliable results. Recognizing the transformative potential of LLMs, this paper proposes a comprehensive set of standards and best practices for their reliable, reproducible, and ethical use. These guidelines span critical areas such as model selection, prompt engineering, structured prompting, prompt stability analysis, rigorous model validation, and the consideration of ethical and legal implications. The paper emphasizes the need for a structured, directed, and formalized approach to using LLMs, aiming to ensure the integrity and robustness of text annotation practices, and advocates for a nuanced and critical engagement with LLMs in social scientific research.

本文提出了对大型语言模型（LLMs）的可靠、可重复和符合伦理的使用的全面标准和最佳实践，涵盖了模型选择、提示工程、结构化提示、提示稳定性分析、严格的模型验证以及伦理和法律影响等关键领域，强调了对LLMs的结构化、有导向性和格式化的使用的需求，以确保文本注释实践的完整性和鲁棒性，并倡导在社会科学研究中对LLMs进行细致而批判性的参与。

大型语言模型文本标注的最佳实践