Large Language Models (LLMs) are regularly being used to label data across
many domains and for myriad tasks. By simply asking the LLM for an answer, or
``prompting,'' practitioners are able to use LLMs to quickly get a response for
an arbitrary task. This prompting is done through a series of decisions by the
practitioner, from simple wording of the prompt, to requesting the output in a
certain data format, to jailbreaking in the case of prompts that address more
sensitive topics. In this work, we ask: do variations in the way a prompt is
constructed change the ultimate decision of the LLM? We answer this using a
series of prompt variations across a variety of text classification tasks. We
find that even the smallest of perturbations, such as adding a space at the end
of a prompt, can cause the LLM to change its answer. Further, we find that
requesting responses in XML and commonly used jailbreaks can have cataclysmic
effects on the data labeled by LLMs.

在这项研究中，我们探究了提示的构建方式对大型语言模型的决策是否产生影响。我们通过一系列针对不同文本分类任务的提示变化进行了实验。研究发现，即使是最小的扰动，如在提示的末尾添加一个空格，都可能导致大型语言模型改变其回答。此外，要求以 XML 格式回应和常用的破解行为可能对大型语言模型标注的数据产生灾难性影响。